During a phone call around early December 2019, Michael had pointed out that since [MIRI's 2019 fundraiser](https://intelligence.org/2019/12/02/miris-2019-fundraiser/) was going on, and we had information about how present-day MIRI differed from its marketing story, there was a time-sensitive opportunity to reach out to a perennial major donor, whom I'll call "Ethan", and fill him in on what we thought we knew about the Blight.
-I wrote to Jessica and Jack Gallagher (cc'ing Michael) on 14 December asking how we should organize this. (Jessica and Jack had relevant testimony about working at MIRI, which would be of more central interest to "Ethan" than my story about how the "rationalists" had lost their way.) Michael also mentioned "Tabitha", a lawyer who had been in the MIRI orbit for a long time, as another person to talk to.
+On 14 December 2019, I wrote to Jessica and Jack Gallagher, another disaffected ex-MIRI researcher, asking how we should organize this. (Jessica and Jack had relevant testimony about working at MIRI, which would be of more central interest to "Ethan" than my story about how the "rationalists" had lost their way.) Michael also mentioned "Tabitha", a lawyer who had been in the MIRI orbit for a long time, as another person to talk to.
-On 22 December, I apologized, saying that I wanted to postpone setting up the meeting, partially because I was on a roll with my productive blogging spree, and partially for a psychological reason: I was feeling subjective pressure to appease Michael by doing the thing that he explicitly suggested because of my loyalty to him, but that would be wrong, because Michael's ideology said that people should follow their sense of opportunity rather than obeying orders. I might feel motived to reach out to "Ethan" and "Tabitha" in January.
+About a week later, I apologized, saying that I wanted to postpone setting up the meeting, partially because I was on a roll with my productive blogging spree, and partially for a psychological reason: I was feeling subjective pressure to appease Michael by doing the thing that he explicitly suggested because of my loyalty to him, but that would be wrong, because Michael's ideology said that people should follow their sense of opportunity rather than obeying orders. I might feel motived to reach out to "Ethan" and "Tabitha" in January.
Michael said that implied that my sense of opportunity was driven by politics, and that I believed that simple honesty couldn't work; he only wanted me to acknowledge that. I was not inclined to affirm that characterization; it seemed like any conversation with "Ethan" and "Tabitha" would be partially optimized to move money, which I thought was politics.
Adherence to norms is fundamentally fraught for the same reason AI alignment is. That is, in [rich domains](https://arbital.com/p/rich_domain/), attempts to regulate behavior with explicit constraints face a lot of adversarial pressure from optimizers bumping up against the constraint and finding the [nearest unblocked strategies](https://arbital.greaterwrong.com/p/nearest_unblocked) that circumvent it. The intent of privacy norms is to conceal information. But [_information_ in Shannon's sense](https://en.wikipedia.org/wiki/Information_theory) is about what states of the world can be inferred given the states of communication signals; it's much more expansive than the denotative meaning of a text.
-If norms can only regulate the denotative meaning of a text (because trying to regulate subtext is too subjective for a norm-enforcing coalition to coordinate on), someone who would prefer to reveal private information but also wants to comply with privacy norms has an incentive to leak everything they possibly can as subtext—to imply it, and hope to escape punishment on grounds of not having "really said it." And if there's some sufficiently egregious letter-complying-but-spirit-violating evasion of the norm that a coalition _can_ coordinate on enforcing, the whistleblower has an incentive to stay _just_ shy of being that egregious.
+If norms can only regulate the denotative meaning of a text (because trying to regulate subtext is too subjective for a norm-enforcing coalition to coordinate on), someone who would prefer to reveal private information but also wants to comply with privacy norms has an incentive to leak everything they possibly can as subtext—to imply it, and hope to escape punishment on grounds of not having "really said it." And if there's some sufficiently egregious letter-complying-but-spirit-violating evasion of the norm that a coalition _can_ coordinate on enforcing, the whistleblower has an incentive to stay only just shy of being that egregious.
Thus, it's unclear how much mere adherence to norms helps, when people's wills are actually misaligned. If I'm furious at Yudkowsky for prevaricating about my Something to Protect, and am in fact _more_ furious rather than less that he managed to do it without violating the norm against lying, I should not be so foolish as to think myself innocent and beyond reproach for not having "really said it."
Previously, I had already thought it was nuts that trans ideology was exerting influence on the rearing of gender-non-conforming children—that is, children who are far outside the typical norm of behavior for their sex: very tomboyish girls and very effeminate boys.
-Under recent historical conditions in the West, these kids were mostly "pre-gay" rather than trans. (The stereotype about lesbians being masculine and gay men being feminine is, like most stereotypes, basically true: sex-atypical childhood behavior between gay and straight adults [has been meta-analyzed at _d_ ≈ 1.31 for men and _d_ ≈ 0.96 for women](/papers/bailey-zucker-childhood_sex-typed_behavior_and_sexual_orientation.pdf).) A solid supermajority of children diagnosed with gender dysphoria [ended up growing out of it by puberty](/papers/steensma_et_al-factors_associated_with_desistence_and_persistence.pdf). In the culture of the current year, it seemed likely that a lot of those kids would instead get affirmed into a cross-sex identity (and being a lifelong medical patient) at a young age, even though most of them would have otherwise (under [a "watchful waiting" protocol](/papers/de_vries-cohen-kettenis-clinical_management_of_gender_dysphoria_in_children.pdf)) grown up to be ordinary gay men and lesbians.
+Under recent historical conditions in the West, these kids were mostly "pre-gay" rather than trans. (The stereotype about lesbians being masculine and gay men being feminine is, like most stereotypes, basically true: sex-atypical childhood behavior between gay and straight adults [has been meta-analyzed at _d_ ≈ 1.31 for men and _d_ ≈ 0.96 for women](/papers/bailey-zucker-childhood_sex-typed_behavior_and_sexual_orientation.pdf).) A solid supermajority of children diagnosed with gender dysphoria [ended up growing out of it by puberty](/papers/steensma_et_al-factors_associated_with_desistence_and_persistence.pdf). In the culture of the current year, it seemed likely that a lot of those kids would instead get affirmed into a cross-sex identity at a young age, even though most of them would have otherwise (under [a "watchful waiting" protocol](/papers/de_vries-cohen-kettenis-clinical_management_of_gender_dysphoria_in_children.pdf)) grown up to be ordinary gay men and lesbians.
-What made this shift in norms crazy, in my view, was not just that transitioning younger children is a dubious treatment decision, but that it's a dubious treatment decision that was being made on the basis of the obvious falsehood that "trans" was one thing: the cultural phenomenon of "trans kids" was being used to legitimize trans _adults_, even though a supermajority of trans adults were in the AGP taxon and therefore had never resembled these HSTS-taxon kids. That is: pre-gay kids in our Society are being sterilized in order to affirm the narcissistic delusions of _guys like me_.
+What made this shift in norms crazy, in my view, was not just that transitioning younger children is a dubious treatment decision, but that it's a dubious treatment decision that was being made on the basis of the obvious falsehood that "trans" was one thing: the cultural phenomenon of "trans kids" was being used to legitimize trans _adults_, even though a supermajority of trans adults were in the late-onset/AGP taxon and therefore had never resembled these HSTS-taxon kids. That is: pre-gay kids in our Society are being sterilized in order to affirm the narcissistic delusions of guys like me.
That much was obvious to anyone who's had their Blanchardian enlightenment, and wouldn't have been worth the effort of writing a special private Document about. The disturbing hypothesis that occured to me in early 2020 was that, in the culture of the current year, affirmation of a cross-sex identity might happen to kids _who weren't even HSTS-taxon at all_.
-Very small children who are just learning what words mean say a lot of things that aren't true (I'm a grown-up; I'm a cat; I'm a dragon), and grownups tend to play along in the moment as a fantasy game, but they don't _coordinate to make that the permanent new social reality_. Ten years ago, if an otherwise ordinary three-year-old boy had occasionally claimed to be a girl, I think his nice smart liberal unschooling caretakers would treat it about the same way as when the kid claims to be a cat. (I'm going with the MtF case without too much loss of generality; I don't think the egregore is quite as eager to transition females at this age, but the dynamics would be broadly similar.)
+Very small children who are just learning what words mean say a lot of things that aren't true (I'm a grown-up; I'm a cat; I'm a dragon), and grownups tend to play along in the moment as a fantasy game, but they don't _coordinate to make that the permanent new social reality_. Ten years ago, if an otherwise ordinary three-year-old boy had occasionally claimed to be a girl, I think his nice smart liberal unschooling caretakers would treat it about the same way as when the kid claims to be a cat. (I'm going with the MtF case without too much loss of generality; I don't think the egregore is quite as eager to transition females at this age, but the dynamics are broadly similar.)
But if the grown-ups have been trained to believe that "trans kids know who they are"—if they're emotionally eager at the prospect of having a transgender child, or fearful of the damage they might do by not affirming—they might selectively attend to confirming evidence that the child "is trans", selectively ignore contrary evidence that the kid "is cis", and end up reinforcing a cross-sex identity that would not have existed if not for their belief in it.
But the harm I'm theorizing is _not_ that the child has an intrinsic male identity that hurts to not be respected. (What is an "identity", apart from the ordinary factual belief that one is of a particular sex?) Rather, the concern is that social transition prompts everyone, _including the child themself_, to use their mental models of girls (juvenile female humans) to make (mostly subconscious rather than deliberative) predictions and decisions about the child, which will be a systematically worse statistical fit than their models of boys (juvenile male humans), because the child is, in fact, a boy (juvenile male human), and those miscalibrated predictions and decisions will make the child's life worse in a complicated, illegible way that doesn't necessarily result in the child spontaneously verbally asserting, "I prefer that you call me a boy."
-Scott Alexander has written about how [concept-shaped holes can be impossible to notice](https://slatestarcodex.com/2017/11/07/concept-shaped-holes-can-be-impossible-to-notice/). A culture whose [civic religion](https://slatestarcodex.com/2019/07/08/gay-rites-are-civil-rites/) celebrates being trans, and denies that gender has truth conditions other than the individual's say-so, has concept-shaped holes that make it hard to notice the hypothesis "I'm having a systematically worse childhood than I otherwise would have because all the grown-ups in my life have agreed I was a girl since I was three years old, even though all of my actual traits are sampled from the joint distribution of juvenile male humans, not juvenile female humans", even if it's true.
+Scott Alexander has written about how [concept-shaped holes can be impossible to notice](https://slatestarcodex.com/2017/11/07/concept-shaped-holes-can-be-impossible-to-notice/). A culture whose [civic religion](https://slatestarcodex.com/2019/07/08/gay-rites-are-civil-rites/) celebrates being trans, and denies that gender has truth conditions other than the individual's say-so, has concept-shaped holes that would make it hard for a kid to notice the hypothesis "I'm having a systematically worse childhood than I otherwise would have because all the grown-ups in my life have agreed I was a girl since I was three years old, even though all of my actual traits are sampled from the joint distribution of juvenile male humans, not juvenile female humans", even if it's true.
-... anyway, that's just a hypothesis that occured to me in early 2020, about something that _could_ happen in the culture of the current year, hypothetically, as far as I know. I'm not a parent and I haven't studied child development. And even if the "Clever Hans" etiological pathway I conjectured is real, the extent to which it might apply to any particular case is complex; you could imagine a kid who _was_ "actually trans", whose social transition merely happened earlier than it otherwise would have due to these dynamics.
+The epistemic difficulties extend to grown-ups as well. I think people who are familiar with the relevant scientific literature or come from an older generation should find the story I've laid out above pretty compelling, but the parents themselves are likely to be more skeptical. They _know_ they didn't coach the child to say "I'm a girl." On what grounds could a stranger who wasn't there (or a skeptical family friend who sees the kid maybe once a month) assert that subconscious influence must be at work?
-For some reason, it seemed really important that I draft a Document about it with lots of citations to send to a few friends. If I get around to it, I might clean it up and publish it as its own blog post (working title: "Trans Kids on the Margin; and, Harms from Misleading Training Data").
+Apparently, people can have very different priors about these things. In the early 20th century, a German schoolteacher named Wilhelm von Osten claimed to have taught his horse, Clever Hans, to do arithmetic and other intellectual feats. One could ask, "How much is 2/5 plus 1/2?" and the stallion would first stomp his hoof nine times, and then ten times—representing 9/10ths, the correct answer. An investigation concluded that no deliberate trickery was involved—Hans could often give the correct answer when questioned by a stranger, demonstrating that von Osten couldn't be secretly signaling the horse when to stop stomping. But further careful experiments by Oskar Pfungst revealed that Hans was picking up on unconscious cues "leaked" by the questioner's body language as the number of stomps approached the correct answer: for instance, Hans couldn't answer questions that the questioner themself didn't know.[^pfungst]
-Given that I spent so many hours on this little research/writing project in May–July 2020, I think it makes sense for me to mention it at this point in my memoir, where it fits in chronologically. I have an inalienable right to talk about my own research interests, and talking about my research interests obviously doesn't violate any norm against leaking private information about someone else's family, or criticizing someone else's parenting decisions.
+[^pfungst]: Oskar Pfungst, [_Clever Hans (The Horse Of Mr. Von Osten): A Contribution To Experimental Animal and Human Psychology_](http://www.gutenberg.org/files/33936/33936-h/33936-h.htm), translated from the German by Carl L. Rahn
+
+Notably, von Osten didn't accept Pfungst's explanation, continuing to believe that his intensive tutoring had succeeded in teaching the horse arithmetic. It's hard to blame him, really. He had spent more time with Hans than anyone else. Hans observably _could_ stomp out the correct answers to questions. Absent a strong prior against the idea that a horse could learn arithmetic, why should he trust Pfungst's nitpicky experiments over the plain facts of his own intimately lived experience?
+
+But what was in question wasn't the observations of Hans's performance, only the interpretation of what those observations implied about Hans's psychology. As Pfungst put it: "that was looked for in the animal which should have been sought in the man."
+
+[TODO: the skeptical family friend's view—
+ * Similarly, in the case of the trans three-year-old, the skeptical family friend doesn't need to doubt that the child said "I'm a girl" and that it wasn't explicitly coached
+ * kid not having being told that boys are the ones with penises
+ * It's really salient that the grownups in the child's life are treating the gender assertion differently than everything else; "I'm a girl and I'm a vegetarian"
+ * Felix Fix-It, Jr. anecdote
+ * lemon anecdote
+ * fascination with forklifts at the same time the pronoun change is going on, but with wisdom of previous generations, and d=2.44 in the literature, has no weight
+ * http://zackmdavis.net/blog/2020/02/relationship-outcomes-are-not-particularly-sensitive-to-small-variations-in-verbal-ability/
+]
+
+Anyway, that's just a hypothesis that occured to me in early 2020, about something that _could_ happen in the culture of the current year, hypothetically, as far as I know. I'm not a parent and I'm not an expert on child development. And even if the "Clever Hans" etiological pathway I conjectured is real, the extent to which it might apply to any particular case is complex; you could imagine a kid who _was_ "actually trans", whose social transition merely happened earlier than it otherwise would have due to these dynamics.
+
+For some reason, it seemed important that I draft a Document about it with lots of citations to send to a few friends. If I get around to it, I might clean it up and publish it as a public blog post (working title: "Trans Kids on the Margin; and, Harms from Misleading Training Data"), but for some reason, that didn't seem as pressing.
+
+I put an epigraph at the top:
+
+> If you love someone, tell them the truth.
+>
+> —Anonymous
+
+Given that I spent so many hours on this little research and writing project in May–July 2020, I think it makes sense for me to mention it at this point in my memoir, where it fits in chronologically. I have an inalienable right to talk about my own research interests, and talking about my research interests obviously doesn't violate any norm against leaking private information about someone else's family, or criticizing someone else's parenting decisions.
(Only—[you two have such beautiful children](/2023/Nov/hrunkner-unnerby-and-the-shallowness-of-progress/)!)
-----
-On 1 June 2020, I received a Twitter DM from _New York Times_ reporter Cade Metz, who said he was "exploring a story about the intersection of the rationality community and Silicon Valley". I sent him an email saying that I would be happy to talk, but that I'd actually been pretty disappointed with the community lately: I was worried that the social pressures of trying to _be_ a "community" and protect the group's status (_e.g._, from _New York Times_ reporters who might portray us in an unflattering light??) incentivize people to compromise on the ideals of _systematically correct reasoning_ that made the community valuable in the first place.
+On 1 June 2020, I received a Twitter DM from _New York Times_ reporter Cade Metz, who said he was "exploring a story about the intersection of the rationality community and Silicon Valley". I sent him an email saying that I would be happy to talk, but that I'd actually been pretty disappointed with the community lately: I was worried that the social pressures of trying to _be_ a "community" and protect the group's status (_e.g._, from _New York Times_ reporters who might portray us in an unflattering light?) incentivize people to compromise on the ideals of systematically correct reasoning that made the community valuable in the first place.
He never got back to me. Three weeks later, all existing _Slate Star Codex_ posts were taken down. A [lone post on the main page](https://slatestarcodex.com/2020/06/22/nyt-is-threatening-my-safety-by-revealing-my-real-name-so-i-am-deleting-the-blog/) explained that the _New York Times_ piece was going to reveal Alexander's real last name, and that he was taking his posts down as a defensive measure. (No blog, no story?) I [wrote a script](/source?p=Ultimately_Untrue_Thought.git;a=commitdiff;h=21731ba6f1191) (`slate_starchive.py`) to replace the _Slate Star Codex_ links on this blog with links to the most recent Internet Archive copy.
------
-I continued to work on my "advanced" philosophy of categorization thesis. The disclaimer note that Scott Alexander had appended to "... Not Man for the Categories" after our Christmas 2019 discussion had said:
+I continued my philosophy of language work, looking into the academic literature on formal models of communication and deception and writing a [couple](https://www.lesswrong.com/posts/4hLcbXaqudM9wSeor/philosophy-in-the-darkest-timeline-basics-of-the-evolution) [posts](https://www.lesswrong.com/posts/YptSN8riyXJjJ8Qp8/maybe-lying-can-t-exist) encapsulating what I learned from that—and I continued work on my "advanced" philosophy of categorization thesis, the sequel to ["Where to Draw the Boundaries?"](https://www.lesswrong.com/posts/esRZaPXSHgWzyB2NL/where-to-draw-the-boundaries)
+
+The disclaimer note that Scott Alexander had appended to "... Not Man for the Categories" after our Christmas 2019 discussion had said:
> I had hoped that the Israel/Palestine example above made it clear that you have to deal with the consequences of your definitions, which can include confusion, muddling communication, and leaving openings for deceptive rhetorical strategies.
-This is certainly an _improvement_ over the original text without the note, but I took the use of the national borders metaphor here to mean that Scott still hadn't really gotten my point about there being underlying laws of thought underlying categorization: mathematical principles governing _how_ definition choices can muddle communication or be deceptive. (But that wasn't surprising; [by Scott's own admission](https://slatestarcodex.com/2013/06/30/the-lottery-of-fascinations/), [he's not a math guy](https://slatestarcodex.com/2015/01/31/the-parable-of-the-talents/).)
+This is certainly an improvement over the original text without the note, but I took the use of the national borders metaphor to mean that Scott still hadn't gotten my point about there being underlying laws of thought underlying categorization: mathematical principles governing _how_ definition choices can muddle communication or be deceptive. (But that wasn't surprising; [by Scott's own admission](https://slatestarcodex.com/2013/06/30/the-lottery-of-fascinations/), [he's not a math guy](https://slatestarcodex.com/2015/01/31/the-parable-of-the-talents/).)
Category "boundaries" are a useful _visual metaphor_ for explaining the cognitive function of categorization: you imagine a "boundary" in configuration space containing all the things that belong to the category.
-If you have the visual metaphor, but you don't have the math, you might think that there's nothing intrinsically wrong with squiggly or discontinuous category "boundaries", just as there's nothing intrinsically wrong with Alaska not being part of the contiguous U.S. states. It may be inconvenient that you can't drive from Alaska to Washington without going through Canada, and we have to deal with the consequences of that, but there's no sense in which it's _wrong_ that the borders are drawn that way: Alaska really is governed by the United States.
+If you have the visual metaphor, but you don't have the math, you might think that there's nothing intrinsically wrong with squiggly or discontinuous category "boundaries", just as there's nothing intrinsically wrong with Alaska not being part of the contiguous U.S. states. It may be inconvenient that you can't drive from Alaska to Washington without going through Canada, and we have to deal with the consequences of that, but it's not "wrong" that the borders are drawn that way: Alaska really is governed by the United States.
-But if you _do_ have the math, a moment of introspection will convince you that the analogy between category "boundaries" and national borders is shallow.
+But if you do have the math, a moment of introspection will convince you that the analogy between category "boundaries" and national borders is shallow.
-A two-dimensional political map tells you which areas of the Earth's surface are under the jurisdiction of what government. In contrast, category "boundaries" tell you which regions of very high-dimensional configuration space correspond to a word/concept, which is useful _because_ that structure is useful for making probabilistic inferences: you can use your observastions of some aspects of an entity (some of the coordinates of a point in configuration space) to infer category-membership, and then use category membership to make predictions about aspects that you haven't yet observed.
+A two-dimensional political map tells you which areas of the Earth's surface are under the jurisdiction of which government. In contrast, category "boundaries" tell you which regions of very high-dimensional configuration space correspond to a word/concept, which is useful _because_ that structure is useful for making probabilistic inferences: you can use your observastions of some aspects of an entity (some of the coordinates of a point in configuration space) to infer category-membership, and then use category membership to make predictions about aspects that you haven't yet observed.
-But the trick only works to the extent that the category is a regular, non-squiggly region of configuration space: if you know that egg-shaped objects tend to be blue, and you see a black-and-white photo of an egg-shaped object, you can get _close_ to picking out its color on a color wheel. But if egg-shaped objects tend to blue _or_ green _or_ red _or_ gray, you wouldn't know where to point to on the color wheel.
+But the trick only works to the extent that the category is a regular, non-squiggly region of configuration space: if you know that egg-shaped objects tend to be blue, and you see a black-and-white photo of an egg-shaped object, you can get close to picking out its color on a color wheel. But if egg-shaped objects tend to blue _or_ green _or_ red _or_ gray, you wouldn't know where to point to on the color wheel.
-The analogous algorithm applied to national borders on a political map would be to observe the longitude of a place, use that to guess what country the place is in, and then use the country to guess the latitude—which isn't typically what people do with maps. Category "boundaries" and national borders might both be _illustrated_ similarly in a two-dimensional diagram, but philosophically, they're different entities. The fact that Scott Alexander was appealing to national borders to explain why gerrymandered categories were allegedly okay, showed that he Didn't Get It.
+The analogous algorithm applied to national borders on a political map would be to observe the longitude of a place, use that to guess what country the place is in, and then use the country to guess the latitude—which isn't typically what people do with maps. Category "boundaries" and national borders might both be illustrated similarly in a two-dimensional diagram, but philosophically, they're different entities. The fact that Scott Alexander was appealing to national borders to explain why gerrymandered categories were allegedly okay, demonstrated that he didn't understand this.
-I still had some deeper philosophical problems to resolve, though. If squiggly categories were less useful for inference, why would someone _want_ a squiggly category boundary? Someone who said, "Ah, but I assign higher utility to doing it this way", had to be messing with you. Where would such a utility function come from? Intuitively, it had to be precisely _because_ squiggly boundaries were less useful for inference; the only reason you would realistically want to do that would be to commit fraud, to pass off pyrite as gold by redefining the word "gold."
+I still had some deeper philosophical problems to resolve, though. If squiggly categories were less useful for inference, why would someone want a squiggly category boundary? Someone who said, "Ah, but I assign higher utility to doing it this way" had to be messing with you. Where would such a utility function come from? Intuitively, it had to be precisely _because_ squiggly boundaries were less useful for inference; the only reason you would realistically want to do that would be to commit fraud, to pass off pyrite as gold by redefining the word "gold".
-That was my intuition. To formalize it, I wanted some sensible numerical quantity that would be maximized by using "nice" categories and get trashed by gerrymandering. [Mutual information](https://en.wikipedia.org/wiki/Mutual_information) was the obvious first guess, but that wasn't it, because mutual information lacks a "topology", a notion of _closeness_ that made some false predictions better than others by virtue of being "close".
+That was my intuition. To formalize it, I wanted some sensible numerical quantity that would be maximized by using "nice" categories and get trashed by gerrymandering. [Mutual information](https://en.wikipedia.org/wiki/Mutual_information) was the obvious first guess, but that wasn't it, because mutual information lacks a "topology", a notion of "closeness" that made some false predictions better than others by virtue of being "close".
-Suppose the outcome space of _X_ is `{H, T}` and the outcome space of _Y_ is `{1, 2, 3, 4, 5, 6, 7, 8}`. I wanted to say that if observing _X_=`H` concentrates _Y_'s probability mass on `{1, 2, 3}`, that's _more useful_ than if it concentrates _Y_ on `{1, 5, 8}`—but that would require the numerals in _Y_ to be _numbers_ rather than opaque labels; as far as elementary information theory was concerned, mapping eight states to three states reduced the entropy from lg<sub>2</sub> 8 = 3 to lg<sub>2</sub> 3 ≈ 1.58 no matter "which" three states they were.
+Suppose the outcome space of _X_ is `{H, T}` and the outcome space of _Y_ is `{1, 2, 3, 4, 5, 6, 7, 8}`. I wanted to say that if observing _X_=`H` concentrates _Y_'s probability mass on `{1, 2, 3}`, that's more useful than if it concentrates _Y_ on `{1, 5, 8}`. But that would require the numerals in _Y_ to be numbers rather than opaque labels; as far as elementary information theory was concerned, mapping eight states to three states reduced the entropy from lg<sub>2</sub> 8 = 3 to lg<sub>2</sub> 3 ≈ 1.58 no matter "which" three states they were.
-How could I make this rigorous? Did I want to be talking about the variance of my features conditional on category-membership? Was "connectedness" intrinsically the what I wanted, or was connectedness only important because it cut down the number of possibilities? (There are 8!/(6!2!) = 28 ways to choose two elements from `{1..8}`, but only 7 ways to choose two contiguous elements.) I thought connectedness was intrinsically important, because we didn't just want _few_ things, we wanted things that are _similar enough to make similar decisions about_.
+How could I make this rigorous? Did I want to be talking about the variance of my features conditional on category-membership? Was "connectedness" intrinsically the what I wanted, or was connectedness only important because it cut down the number of possibilities? (There are 8!/(6!2!) = 28 ways to choose two elements from `{1..8}`, but only 7 ways to choose two contiguous elements.) I thought connectedness was intrinsically important, because we didn't just want _few_ things, we wanted things that are similar enough to make similar decisions about.
I put the question to a few friends in July 2020 (Subject: "rubber duck philosophy"), and Jessica said that my identification of the variance as the key quantity sounded right: it amounted to the expected squared error of someone trying to guess the values of the features given the category. It was okay that this wasn't a purely information-theoretic criterion, because for problems involving guessing a numeric quantity, bits that get you closer to the right answer were more valuable than bits that didn't.
------
-I decided on "Unnatural Categories Are Optimized for Deception" as the title for my advanced categorization thesis. Writing it up was a large undertaking. There were a lot of nuances to address and potential objections to preëmpt, and I felt that I had to cover everything. (A reasonable person who wanted to understand the main ideas wouldn't need so much detail, but I wasn't up against reasonable people who wanted to understand.)
+I decided on "Unnatural Categories Are Optimized for Deception" as the title for my advanced categorization thesis. Writing it up was a major undertaking. There were a lot of nuances to address and potential objections to preëmpt, and I felt that I had to cover everything. (A reasonable person who wanted to understand the main ideas wouldn't need so much detail, but I wasn't up against reasonable people who wanted to understand.)
-In September 2020, Yudkowsky Tweeted [something about social media incentives prompting people to make nonsense arguments](https://twitter.com/ESYudkowsky/status/1304824253015945216). Something in me boiled over. The Tweet was fine in isolation, but I rankled at it in the context of his own incentive-driven nonsense remaining unaddressed. I left [a pleading, snarky reply](https://twitter.com/zackmdavis/status/1304838486810193921) and [vented on my own timeline](https://twitter.com/zackmdavis/status/1304838346695348224) (with preview images from the draft of "Unnatural Categories Are Optimized for Deception"):
+In September 2020, Yudkowsky Tweeted [something about social media incentives prompting people to make nonsense arguments](https://twitter.com/ESYudkowsky/status/1304824253015945216), and something in me boiled over. The Tweet was fine in isolation, but I rankled at it given the absurdly disproprotionate efforts I was undertaking to unwind his incentive-driven nonsense. I left [a pleading, snarky reply](https://twitter.com/zackmdavis/status/1304838486810193921) and [vented on my own timeline](https://twitter.com/zackmdavis/status/1304838346695348224) (with preview images from the draft of "Unnatural Categories Are Optimized for Deception"):
> Who would have thought getting @ESYudkowsky's robot cult to stop trying to trick me into cutting my dick off (independently of the empirical facts determining whether or not I should cut my dick off) would involve so much math?? OK, I guess the math part isn't surprising, but—
projects / Ultimately_Untrue_Thought.git / blobdiff