Anyone else being wrong on the internet like that wouldn't have seemed like a big deal, but Scott Alexander had [semi-jokingly](http://www.catb.org/jargon/html/H/ha-ha-only-serious.html) written that [rationalism is the belief that Eliezer Yudkowsky is the rightful caliph](https://slatestarcodex.com/2016/04/04/the-ideology-is-not-the-movement/). After extensive attempts by me and allies to get clarification from Yudkowsky amounted to nothing, we felt justified in concluding that he and his Caliphate of so-called "rationalists" was corrupt.
+[TOC]
+
### Origins of the Rationalist Civil War (April–May 2019)
Anyway, given that the "rationalists" were fake and that we needed something better, there remained the question of what to do about that, and how to relate to the old thing.
[^posse-boundary]: Although Sarah Constantin and "Riley" had also been involved in reaching out to Yudkowsky and were included in many subsequent discussions, they seemed like more marginal members of the group that was forming.
-Ben had previously worked at GiveWell and had written a lot about problems with the effective altruism (EA) movement; in particular, he argued that EA-branded institutions were making [incoherent](http://benjaminrosshoffman.com/givewell-and-partial-funding/) [decisions](http://benjaminrosshoffman.com/openai-makes-humanity-less-safe/) under the influence of incentives to [distort](http://benjaminrosshoffman.com/humility-argument-honesty/) [information](http://benjaminrosshoffman.com/honesty-and-perjury/) [in order to](http://benjaminrosshoffman.com/effective-altruism-is-self-recommending/) [seek](http://benjaminrosshoffman.com/against-neglectedness/) [power](http://benjaminrosshoffman.com/against-responsibility/).
+Ben had previously worked at GiveWell and had written a lot about problems with the Effective Altruism (EA) movement; in particular, he argued that EA-branded institutions were making [incoherent](http://benjaminrosshoffman.com/givewell-and-partial-funding/) [decisions](http://benjaminrosshoffman.com/openai-makes-humanity-less-safe/) under the influence of incentives to [distort](http://benjaminrosshoffman.com/humility-argument-honesty/) [information](http://benjaminrosshoffman.com/honesty-and-perjury/) [in order to](http://benjaminrosshoffman.com/effective-altruism-is-self-recommending/) [seek](http://benjaminrosshoffman.com/against-neglectedness/) [power](http://benjaminrosshoffman.com/against-responsibility/).
Jessica had previously worked at MIRI, where she was unnerved by what she saw as under-evidenced paranoia about information hazards and [short AI timelines](https://www.lesswrong.com/posts/KnQs55tjxWopCzKsk/the-ai-timelines-scam). (As Jack Gallagher, who was also at MIRI at the time, [put it](https://www.greaterwrong.com/posts/pQGFeKvjydztpgnsY/occupational-infohazards/comment/TcsXh44pB9xRziGgt), "A bunch of people we respected and worked with had decided the world was going to end, very soon, uncomfortably soon, and they were making it extremely difficult for us to check their work.")
I asked my boss to temporarily assign me some easier tasks that I could make steady progress on. (We had a lot of LaTeX templating of insurance policy amendments that needed to get done.) If I was going to be psychologically impaired, it was better to be up-front about how I could best serve the company given that impairment, rather than hoping the boss wouldn't notice.
-My intent of a break from the religious war didn't take. I met with Anna on the UC Berkeley campus and read her excerpts from Ben's and Jessica's emails. (She had not provided a comment on "... Boundaries?" despite my requests, including in the form of two paper postcards that I stayed up until 2 _a.m._ on 14 April 2019 writing; spamming people with hysterical and somewhat demanding postcards felt more distinctive than my usual habit of spamming people with hysterical and somewhat demanding emails.) While we (my posse) were aghast at Yudkowsky's behavior, Anna was aghast at ours: reaching out to try to have a conversation with Yudkowsky, then concluding that he was a fraud because we weren't satisfied with the outcome was like hiding soldiers in an ambulance, introducing a threat into a context where he had a right to be safe.
+My intent of a break from the religious war didn't take. I met with Anna on the UC Berkeley campus and read her excerpts from Ben's and Jessica's emails. (She had not provided a comment on ["Where to Draw the Boundaries?"](https://www.lesswrong.com/posts/esRZaPXSHgWzyB2NL/where-to-draw-the-boundaries) despite my requests, including in the form of two paper postcards that I stayed up until 2 _a.m._ on 14 April 2019 writing; spamming people with hysterical and somewhat demanding postcards felt more distinctive than my usual habit of spamming people with hysterical and somewhat demanding emails.)
I complained that I had believed our own [marketing](https://www.lesswrong.com/posts/aFEsqd6ofwnkNqaXo/go-forth-and-create-the-art) [material](https://www.lesswrong.com/posts/jP583FwKepjiWbeoQ/epistle-to-the-new-york-less-wrongians) about the "rationalists" remaking the world by wielding a hidden Bayesian structure of Science and Reason that applies [outside the laboratory](https://www.lesswrong.com/posts/N2pENnTPB75sfc9kb/outside-the-laboratory). Was that all a lie? Were we not trying to do the thing anymore? Anna was dismissive: she thought that the idea I had gotten about "the thing" was never actually part of the original vision. She kept repeating that she had tried to warn me, and I didn't listen. (Back in the late 'aughts, she had often recommended Paul Graham's essay ["What You Can't Say"](http://paulgraham.com/say.html) to people, summarizing Graham's moral that you should figure out the things you can't say in your culture and then not say them, in order to avoid getting drawn into pointless conflicts.)
MIRI researcher Scott Garrabrant wrote a post about how ["Yes Requires the Possibility of No"](https://www.lesswrong.com/posts/G5TwJ9BGxcgh5DsmQ/yes-requires-the-possibility-of-no). Information-theoretically, a signal sent with probability one transmits no information: you can only learn something from hearing a "Yes" if you believed that the answer could have been "No". I saw an analogy to my philosophy-of-language thesis, and mentioned it in a comment: if you want to believe that _x_ belongs to category _C_, you might try redefining _C_ in order to make the question "Is _x_ a _C_?" come out "Yes", but you can only do so at the expense of making _C_ less useful. Meaningful category-membership (Yes) requires the possibility of non-membership (No).
-Someone [objected that](https://www.greaterwrong.com/posts/WwTPSkNwC89g3Afnd/comment-section-from-05-19-2019/comment/FxSZwECjhgYE7p2du) she found it "unpleasant that [I] always bring [my] hobbyhorse in, but in an 'abstract' way that doesn't allow discussing the actual object level question"; it made her feel "attacked in a way that allow[ed] for no legal recourse to defend [herself]." (I thought I remembered meeting a man with the same last name at the 2016 Summer Solstice event in Berkeley; maybe it was her brother.) I [replied](https://www.greaterwrong.com/posts/WwTPSkNwC89g3Afnd/comment-section-from-05-19-2019/comment/32GPaijsSwX2NSFJi) that that was understandable, but that I hoped it was also understandable that I found it unpleasant that our standard Bayesian philosophy of language somehow got politicized, such that my attempts to do correct epistemology were perceived as attacking people.
+Someone [objected that](https://www.greaterwrong.com/posts/WwTPSkNwC89g3Afnd/comment-section-from-05-19-2019/comment/FxSZwECjhgYE7p2du) she found it "unpleasant that [I] always bring [my] hobbyhorse in, but in an 'abstract' way that doesn't allow discussing the actual object level question"; it made her feel "attacked in a way that allow[ed] for no legal recourse to defend [herself]." I [replied](https://www.greaterwrong.com/posts/WwTPSkNwC89g3Afnd/comment-section-from-05-19-2019/comment/32GPaijsSwX2NSFJi) that that was understandable, but that I hoped it was also understandable that I found it unpleasant that our standard Bayesian philosophy of language somehow got politicized, such that my attempts to do correct epistemology were perceived as attacking people.
Such a trainwreck ensued that the mods manually [moved the comments to their own post](https://www.lesswrong.com/posts/WwTPSkNwC89g3Afnd/comment-section-from-05-19-2019). Based on the karma scores and what was said,[^yes-requires-slapfight-highlights] I count it as a victory.
What I did think was that the need to keep up appearances of not being a right wing Bad Guy was a serious distortion of people's beliefs, because there are at least a few questions of fact where believing the correct answer can, in today's political environment, be used to paint one as a right-wing Bad Guy. I would have hoped for Yudkowsky to notice that this is a rationality problem and to not actively make the problem worse. I was counting "I do not welcome support from those quarters" as making the problem worse insofar as it would seem to imply that if I thought I'd learned valuable things from Moldbug, that made me less welcome in Yudkowsky's fiefdom.
-Yudkowsky certainly wouldn't endorse "Even learning things from these people makes you unwelcome" _as stated_, but "I do not welcome support from those quarters" still seemed like a pointlessly partisan silencing/shunning attempt, when one could just as easily say, "I'm not a neoreactionary, and if some people who read me are, that's obviously not my fault."
+Yudkowsky certainly wouldn't endorse "Even learning things from these people makes you unwelcome" _as stated_, but "I do not welcome support from those quarters" still seemed like a pointlessly partisan silencing/shunning attempt, when one could just as easily say, "I'm not a neoreactionary, and if some people who read me are, that's _obviously not my fault_."
Jessica asked if Yudkowsky denouncing neoreaction and the alt-right would still seem harmful, if he were to also to acknowledge, _e.g._, racial IQ differences?
> It seems to [me] that according to Zack's own account, even writing the memoir _privately_ feels like an act of war that he'd rather avoid, not just using his own territory as he sees fit to create _internal_ clarity around a thing.
>
-> I think this has to mean _either_
-> (a) that Zack isn't on the side of clarity except pragmatically where that helps him get his particular story around gender and rationalism validated
+> I think this has to mean _either_
+> (a) that Zack isn't on the side of clarity except pragmatically where that helps him get his particular story around gender and rationalism validated
> _or_
> (b) that Zack has ceded the territory of the interior of his own mind to the forces of anticlarity, not for reasons, but just because he's let the anticlaritarians dominate his frame.
Very small children who are just learning what words mean say a lot of things that aren't true (I'm a grown-up; I'm a cat; I'm a dragon), and grownups tend to play along in the moment as a fantasy game, but they don't _coordinate to make that the permanent new social reality_.
-But if the grown-ups have been trained to believe that "trans kids know who they are"—if they're emotionally eager at the prospect of having a transgender child, or fearful of the damage they might do by not affirming—they might selectively attend to confirming evidence that the child "is trans", selectively ignore contrary evidence that the kid "is cis", and end up reinforcing a cross-sex identity that would not have existed if not for their belief in it—a belief that the same people raising the same child wouldn't have held ten years ago. ([A September 2013 article in _The Atlantic_](https://archive.is/FJNII) by the father of a male child with stereotypically feminine interests was titled "My Son Wears Dresses; Get Over It", not "My Daughter Is Trans; Get Over It".)
+But if the grown-ups have been trained to believe that "trans kids know who they are"—if they're emotionally eager at the prospect of having a transgender child, or fearful of the damage they might do by not affirming—they might selectively attend to confirming evidence that the child "is trans", selectively ignore contrary evidence that the child "is cis", and end up reinforcing a cross-sex identity that would not have existed if not for their belief in it—a belief that the same people raising the same child ten years ago wouldn't have held. ([A September 2013 article in _The Atlantic_](https://archive.is/FJNII) by the father of a male child with stereotypically feminine interests was titled "My Son Wears Dresses; Get Over It", not "My Daughter Is Trans; Get Over It".)
Crucially, if gender identity isn't an innate feature of toddler psychology, _the child has no way to know anything is "wrong."_ If none of the grown-ups can say, "You're a boy because boys are the ones with penises" (because that's not what nice smart liberal people are supposed to believe in the current year), how is the child supposed to figure that out independently? [Toddlers are not very sexually dimorphic](/2019/Jan/the-dialectic/), but sex differences in play style and social behavior tend to emerge within a few years. There were no cars in the environment of evolutionary adaptedness, and yet [the effect size of the sex difference in preference for toy vehicles is a massive _d_ ≈ 2.44](/papers/davis-hines-how_large_are_gender_differences_in_toy_preferences.pdf), about one and a half times the size of the sex difference in adult height.
-(I'm going with the MtF case without too much loss of generality; I don't think the egregore is quite as eager to transition females at this age, but the dynamics are broadly similar.)
+(I'm going with the MtF case without too much loss of generality; I don't think the egregore is quite as eager to transition females at this age, but the dynamics are probably similar.)
What happens when the kid develops a self-identity as a girl, only to find out, potentially years later, that she noticeably doesn't fit in with the (cis) girls on the [many occasions that no one has explicitly spelled out in advance](/2019/Dec/more-schelling/) where people are using "gender" (perceived sex) to make a prediction or decision?
Scott Alexander has written about how [concept-shaped holes can be impossible to notice](https://slatestarcodex.com/2017/11/07/concept-shaped-holes-can-be-impossible-to-notice/). In a culture whose [civic religion](https://slatestarcodex.com/2019/07/08/gay-rites-are-civil-rites/) celebrates being trans and denies that gender has truth conditions other than the individual's say-so, there are concept-shaped holes that would make it hard for a kid to notice the hypothesis "I'm having a systematically worse childhood than I otherwise would have because all the grown-ups in my life have agreed I was a girl since I was three years old, even though all of my actual traits are sampled from the joint distribution for juvenile male humans, not juvenile female humans."
-The epistemic difficulties extend to grown-ups as well. I think people who are familiar with the relevant scientific literature or come from an older generation should find the story I've laid out above pretty compelling, but the parents are likely to be unmoved. They _know_ they didn't coach the child to claim to be a girl. On what grounds could a stranger who wasn't there (or a skeptical family friend who sees the kid maybe once a month) assert that subconscious influence must be at work?
+The epistemic difficulties extend to the grown-ups as well. I think people who are familiar with the relevant scientific literature or come from an older generation will find the story I've laid out above pretty compelling, but the parents are likely to be unmoved. They _know_ they didn't coach the child to claim to be a girl. On what grounds could a stranger who wasn't there (or a skeptical family friend who sees the kid maybe once a month) assert that subconscious influence must be at work?
In the early twentieth century, a German schoolteacher named Wilhelm von Osten claimed to have taught his horse, Clever Hans, to do arithmetic and other intellectual feats. One could ask, "How much is 2/5 plus 1/2?" and the stallion would first stomp his hoof nine times, and then ten times—representing 9/10ths, the correct answer. An investigation concluded that no deliberate trickery was involved: Hans could often give the correct answer when questioned by a stranger, demonstrating that von Osten couldn't be secretly signaling the horse when to stop stomping. But further careful experiments by Oskar Pfungst revealed that Hans was picking up on unconscious cues "leaked" by the questioner's body language as the number of stomps approached the correct answer: for instance, Hans couldn't answer questions that the questioner themself didn't know.[^pfungst]
From the skeptical family friend's perspective, there are a number of anomalies that cast serious doubt on what the family thinks is the obvious interpretation.
-(Or so I'm imagining how this might go, hypothetically. The following anecdotes are merely illustrative, and may not reflect real events.)
+(Or so I'm imagining how this might go, hypothetically. The following illustrative vignettes may not reflect real events.)
-For one thing, there may be clues that the child's information environment did not provide instruction on some of the relevant facts. Suppose that, six months before the child's social transition went down, another family friend had reportedly explained to the child that "Some people don't have penises." Apparently, grown-ups in Berkeley in the current year don't feel the need to be more specific. Growing up in such a culture, the child's initial gender statements may reflect mere confusion rather than a deep-set need—and later statements may reflect social reinforcement of earlier confusion. Suppose that after social transition, the same friend reportedly explained to the child, "When you were little, you couldn't talk, so your parents had to guess whether you were a boy or a girl based on your parts." While this claim does convey the lesson that there's a customary default relationship between gender and genitals (in case that hadn't been clear before), it also reinforces the idea that the child is transgender.
+For one thing, there may be clues that the child's information environment did not provide instruction on some of the relevant facts. Suppose that, six months before the child's social transition went down, another family friend had explained to the child that "Some people don't have penises." (Nice smart liberal grown-ups in the current year don't feel the need to be more specific.) Growing up in such a culture, the child's initial gender statements may reflect mere confusion rather than a deep-set need—and later statements may reflect social reinforcement of earlier confusion. Suppose that after social transition, the same friend explained to the child, "When you were little, you couldn't talk, so your parents had to guess whether you were a boy or a girl based on your parts." While this claim does convey the lesson that there's a customary default relationship between gender and genitals (in case that hadn't been clear before), it also reinforces the idea that the child is transgender.
For another thing, from the skeptical family friend's perspective, it's striking how the family and other grown-ups in the child's life seem to treat the child's statements about gender starkly differently than the child's statements about everything else.
-Suppose that, around the time of the social transition, the child reportedly responded to "Hey kiddo, I love you" with, "I'm a girl and I'm a vegetarian." In the skeptic's view, both halves of that sentence were probably generated by the same cognitive algorithm—something like, "practice language and be cute to caregivers, making use of themes from the local cultural environment" (where grown-ups in Berkeley talk a lot about gender and animal welfare). In the skeptic's view, if you're not going to change the kid's diet on the basis of the second part, you shouldn't social transition the kid on the basis of the first part.
+Suppose that, around the time of the social transition, the child responded to "Hey kiddo, I love you" with, "I'm a girl and I'm a vegetarian." In the skeptic's view, both halves of that sentence were probably generated by the same cognitive algorithm—something like, "practice language and be cute to caregivers, making use of themes from the local cultural environment" (of nice smart liberal grown-ups who talk a lot about gender and animal welfare). In the skeptic's view, if you're not going to change the kid's diet on the basis of the second part, you shouldn't social transition the kid on the basis of the first part.
+
+Perhaps even more striking is the way that the grown-ups seem to interpret the child's conflicting or ambiguous statements about gender. Suppose that, around the time social transition was being considered, a parent asked the child whether the child would prefer to be addressed as "my son" or "my daughter."
+
+The child replied, "My son. Or you can call me she. Everyone should call me she or her or my son."
+
+The grown-ups seem to mostly interpret exchanges like this as indicating that while the child is trans, she's confused about the gender of the words "son" and "daughter". They don't seem to pay much attention to the competing hypothesis that the child knows he's his parents "son", but is confused about the implications of she/her pronouns.
It's not hard to imagine how differential treatment by grown-ups of gender-related utterances could unintentionally shape outcomes. This may be clearer if we imagine a non-gender case. Suppose the child's father's name is John Smith, and that after a grown-up explains ["Sr."/"Jr." generational suffixes](https://en.wikipedia.org/wiki/Suffix_(name)#Generational_titles) after it happened to come up in fiction, the child declares that his name is John Smith, Jr. now. Caregivers are likely to treat this as just a cute thing that the kid said, quickly forgotten by all. But if caregivers feared causing psychological harm by denying a declared name change, one could imagine them taking the child's statement as a prompt to ask followup questions. ("Oh, would you like me to call you _John_ or _John Jr._, or just _Junior_?") With enough followup, it seems plausible that a name change to "John Jr." would meet with the child's assent and "stick" socially. The initial suggestion would have come from the child, but most of the [optimization](https://www.lesswrong.com/posts/D7EcMhL26zFNbJ3ED/optimization)—the selection that this particular statement should be taken literally and reinforced as a social identity, while others are just treated as a cute but not overly meaningful thing the kid said—would have come from the adults.
"I don't think boys like rainbows so well—oh hey! Here this ball is!"
-(When recounting this conversation, the parent helpfully adds that rainbows hadn't come up before, and that the child was looking at a rainbow-patterned item at the time of answering.)
+(When recounting this conversation, the parent adds that rainbows hadn't come up before, and that the child was looking at a rainbow-patterned item at the time of answering.)
-It would seem that the intepretation of this kind of evidence depends on one's prior convictions. If you think that transition is a radical intervention that might pass a cost–benefit analysis for treating rare cases of intractable sex dysphoria, nonsense answers like "because girls like specific things like rainbows" are disqualifying. (A twelve-year-old who could read an informed-consent form would be able to give a more compelling explanation than that, but a three-year-old just isn't ready to make this kind of decision.) Whereas if you think that some children have a gender that doesn't match their assigned sex at birth, you might expect them to express that affinity at age three, without yet having the cognitive or verbal abilities to explain it. Teasing apart where these two views make different predictions seems like it should be possible, but might be beside the point, if the real crux is over [what categories are made for](/2018/Feb/the-categories-were-made-for-man-to-make-predictions/).
+It would seem that the intepretation of this kind of evidence depends on one's prior convictions. If you think that transition is a radical intervention that might pass a cost–benefit analysis for treating rare cases of intractable sex dysphoria, answers like "because girls like specific things like rainbows" are disqualifying. (A fourteen-year-old who could read an informed-consent form would be able to give a more compelling explanation than that, but a three-year-old just isn't ready to make this kind of decision.) Whereas if you think that some children have a gender that doesn't match their assigned sex at birth, you might expect them to express that affinity at age three, without yet having the cognitive or verbal abilities to explain it. Teasing apart where these two views make different predictions seems like it should be possible, but might be beside the point, if the real crux is over [what categories are made for](/2018/Feb/the-categories-were-made-for-man-to-make-predictions/). (Is sex an objective fact that sometimes merits social recognition, or is it better to live in a Society where people are free to choose the gender that suits them?)
Anyway, that's just a hypothesis that occurred to me in early 2020, about something that _could_ happen in the culture of the current year, hypothetically, as far as I know. I'm not a parent and I'm not an expert on child development. And even if the "Clever Hans" etiological pathway I conjectured is real, the extent to which it might apply to any particular case is complex; you could imagine a kid who _was_ "actually trans" whose social transition merely happened earlier than it otherwise would have due to these dynamics.
>
> —Anonymous
-Given that I spent so many hours on this little research and writing project in May–July 2020, I think it makes sense for me to mention it at this point in my memoir, where it fits in chronologically. I have an inalienable right to talk about my own research interests, and talking about my research interests obviously doesn't violate any norm against leaking private information about someone else's family, or criticizing someone else's parenting decisions.
-
-(Only—[you two have such beautiful children](/2023/Dec/hrunkner-unnerby-and-the-shallowness-of-progress/)!)
+Given that I spent so many hours on this little research and writing project in May–July 2020, I think it makes sense for me to mention it at this point in my memoir, where it fits in chronologically. I have an inalienable right to talk about my own research interests, and talking about my own research interests obviously doesn't violate any norm against leaking private information about someone else's family, or criticizing someone else's parenting decisions.
### The _New York Times_ Pounces (June 2020)
projects / Ultimately_Untrue_Thought.git / blobdiff