+But if you do have the math, a moment of introspection will convince you that the analogy between category "boundaries" and national borders is shallow.
+
+A two-dimensional political map tells you which areas of the Earth's surface are under the jurisdiction of which government. In contrast, category "boundaries" tell you which regions of very high-dimensional configuration space correspond to a word/concept, which is useful _because_ that structure can be used to make probabilistic inferences. You can use your observations of some aspects of an entity (some of the coordinates of a point in configuration space) to infer category-membership, and then use category membership to make predictions about aspects that you haven't yet observed.
+
+But the trick only works to the extent that the category is a regular, non-squiggly region of configuration space: if you know that egg-shaped objects tend to be blue, and you see a black-and-white photo of an egg-shaped object, you can get close to picking out its color on a color wheel. But if egg-shaped objects tend to blue _or_ green _or_ red _or_ gray, you wouldn't know where to point to on the color wheel.
+
+The analogous algorithm applied to national borders on a political map would be to observe the longitude of a place, use that to guess what country the place is in, and then use the country to guess the latitude—which isn't typically what people do with maps. Category "boundaries" and national borders might both be illustrated similarly in a two-dimensional diagram, but philosophically, they're different entities. The fact that Scott Alexander was appealing to national borders to defend gerrymandered categories, suggested that he didn't understand this.
+
+I still had some deeper philosophical problems to resolve, though. If squiggly categories were less useful for inference, why would someone want a squiggly category boundary? Someone who said, "Ah, but I assign higher utility to doing it this way" had to be messing with you. Squiggly boundaries were less useful for inference; the only reason you would realistically want to use them would be to commit fraud, to pass off pyrite as gold by redefining the word "gold".
+
+That was my intuition. To formalize it, I wanted some sensible numerical quantity that would be maximized by using "nice" categories and get trashed by gerrymandering. [Mutual information](https://en.wikipedia.org/wiki/Mutual_information) was the obvious first guess, but that wasn't it, because mutual information lacks a "topology", a notion of "closeness" that would make some false predictions better than others by virtue of being "close".
+
+Suppose the outcome space of _X_ is `{H, T}` and the outcome space of _Y_ is `{1, 2, 3, 4, 5, 6, 7, 8}`. I wanted to say that if observing _X_=`H` concentrates _Y_'s probability mass on `{1, 2, 3}`, that's more useful than if it concentrates _Y_ on `{1, 5, 8}`. But that would require the numerals in _Y_ to be numbers rather than opaque labels; as far as elementary information theory was concerned, mapping eight states to three states reduced the entropy from log<sub>2</sub> 8 = 3 to log<sub>2</sub> 3 ≈ 1.58 no matter which three states they were.
+
+How could I make this rigorous? Did I want to be talking about the variance of my features conditional on category membership? Was "connectedness" what I wanted, or was it only important because it cut down the number of possibilities? (There are 8!/(6!2!) = 28 ways to choose two elements from `{1..8}`, but only 7 ways to choose two contiguous elements.) I thought connectedness was intrinsically important, because we didn't just want _few_ things, we wanted things that are similar enough to make similar decisions about.
+
+I put the question to a few friends in July 2020 (Subject: "rubber duck philosophy"), and Jessica said that my identification of the variance as the key quantity sounded right: it amounted to the expected squared error of someone trying to guess the values of the features given the category. It was okay that this wasn't a purely information-theoretic criterion, because for problems involving guessing a numeric quantity, bits that get you closer to the right answer were more valuable than bits that didn't.
+
+------
+
+I decided on "Unnatural Categories Are Optimized for Deception" as the title for my advanced categorization thesis. Writing it up was a major undertaking. There were a lot of nuances to address and potential objections to preëmpt, and I felt that I had to cover everything. (A reasonable person who wanted to understand the main ideas wouldn't need so much detail, but I wasn't up against reasonable people who wanted to understand.)
+
+In September 2020, Yudkowsky Tweeted [something about social media incentives prompting people to make nonsense arguments](https://twitter.com/ESYudkowsky/status/1304824253015945216), and something in me boiled over. The Tweet was fine in isolation, but I rankled at it given the absurdly disproportionate efforts I was undertaking to unwind his incentive-driven nonsense. I left [a snarky, pleading reply](/images/davis-snarky_pleading_reply.png) and [vented on my own timeline](https://twitter.com/zackmdavis/status/1304838346695348224) (with preview images from the draft of "Unnatural Categories Are Optimized for Deception"):
+
+> Who would have thought getting @ESYudkowsky's robot cult to stop trying to trick me into cutting my dick off (independently of the empirical facts determining whether or not I should cut my dick off) would involve so much math?? OK, I guess the math part isn't surprising, but—[^trying-to-trick-me]
+
+[^trying-to-trick-me]: I anticipate that some readers might object to the "trying to trick me into cutting my dick off" characterization. But as [Ben had pointed out earlier](/2023/Jul/a-hill-of-validity-in-defense-of-meaning/#physical-injuries), we have strong reason to believe that an information environment of ubiquitous propaganda was creating medical transitions on the margin. I think it made sense for me to use emphatic language to highlight what was actually at stake here!
+
+My rage-boil continued into staying up late writing him an angry email, which I mostly reproduce below (with a few redactions for either brevity or compliance with privacy norms, but I'm not going to clarify which).