1 Title: Standing Under the Same Sky
5 Tags: autogynephilia, bullet-biting, cathartic, Eliezer Yudkowsky, Scott Alexander, epistemic horror, my robot cult, personal, sex differences, two-type taxonomy, whale metaphors
8 > "The only thing standing in the way of my own progress," Sagreda said, "is that the forces that once dealt with us honestly have been buried too deep to reach. All I can touch now is the surface, which is shaped by nothing but whim."
10 > —["Bit Players"](https://subterraneanpress.com/magazine/winter_2014/bit_players_by_greg_egan) by Greg Egan
12 ... except, I would be remiss to condemn Yudkowsky without discussing—potentially mitigating factors. (I don't want to say that whether someone is a fraud should depend on whether there are mitigating factors—rather, I should discuss potential reasons why being a fraud might be the least-bad choice, when faced with a sufficiently desperate situation.)
14 So far, I've been writing from the perspective of caring (and expecting Yudkowsky to care) about human rationality as a cause in its own right—about wanting to _make sense_, and wanting to live in a Society that made sense, for its own sake, and not as a convergently instrumental subgoal of saving the world.
16 That's pretty much always where I've been at. I _never_ wanted to save the world. I got sucked in to this robot cult because Yudkowsky's philsophy-of-science blogging was just that good. I did do a little bit of work for the Singularity Institute back in the day (an informal internship in 'aught-nine, some data-entry-like work manually adding Previous/Next links to the Sequences, designing several PowerPoint presentations for Anna, writing some Python scripts to organize their donor database), but that was because it was my social tribe and I had connections. To the extent that I took at all seriously the whole save/destroy/take-over the world part (about how we needed to encode all of human morality into a recursively self-improving artificial intelligence to determine our entire future light cone until the end of time), I was scared rather than enthusiastic.
18 Okay, being scared was entirely appropriate, but what I mean is that I was scared, and concluded that shaping the Singularity was _not my problem_, as contrasted to being scared, then facing up to the responsibility anyway. After a 2013 sleep-deprivation-induced psychotic episode which [featured](http://zackmdavis.net/blog/2013/03/religious/) [futurist](http://zackmdavis.net/blog/2013/04/prodrome/)-[themed](http://zackmdavis.net/blog/2013/05/relativity/) [delusions](http://zackmdavis.net/blog/2013/05/relevance/), I wrote to Anna, Michael, and some MIRI employees who had been in my contacts for occasional contract work, that "my current plan [was] to just try to forget about _Less Wrong_/MIRI for a long while, maybe at least a year, not because it isn't technically the most important thing in the world, but because I'm not emotionally stable enough think about this stuff anymore" (Subject: "to whom it may concern"). When I got a real programming job and established an income for myself, I [donated to CfAR rather than MIRI](http://zackmdavis.net/blog/2016/12/philanthropy-scorecard-through-2016/), because public rationality was something I could be unambiguously enthusiastic about, and doing anything about AI was not.
20 At the time, it seemed fine for the altruistically-focused fraction of my efforts to focus on rationality, and to leave the save/destroy/take-over the world stuff to other, more emotionally-stable people, in accordance with the principle of comparative advantage. Yudkowsky had written his Sequences as a dependency for explaining [the need for friendly AI](https://www.lesswrong.com/posts/GNnHHmm8EzePmKzPk/value-is-fragile), ["gambl\[ing\] only upon the portion of the activism that would flow to \[his\] own cause"](https://www.lesswrong.com/posts/9jF4zbZqz6DydJ5En/the-end-of-sequences), but rationality was supposed to be the [common interest of many causes](https://www.lesswrong.com/posts/4PPE6D635iBcGPGRy/rationality-common-interest-of-many-causes). Even if I wasn't working or donating to MIRI specifically, I was still _helping_, a good citizen according to the morality of my tribe.
22 But fighting for public epistemology is a long battle; it makes more sense if you have _time_ for it to pay off. Back in the late 'aughts and early 'tens, it looked like we had time. We had these abstract philosophical arguments for worrying about AI, but no one really talked about _timelines_. I believed the Singularity was going to happen in the 21st century, but it felt like something to expect in the _second_ half of the 21st century.
24 Now it looks like we have—less time? Not just tautologically because time has passed (the 21st century is one-fifth over—closer to a quarter over), but because of new information from the visible results of the deep learning revolution.[^second-half] Yudkowsky seemed particularly [spooked by AlphaGo](https://www.lesswrong.com/posts/7MCqRnZzvszsxgtJi/christiano-cotra-and-yudkowsky-on-ai-progress?commentId=gQzA8a989ZyGvhWv2) [and AlphaZero](https://intelligence.org/2017/10/20/alphago/) in 2016–2017, not because superhuman board game players were dangerous, but because of what it implied about the universe of algorithms.
26 In part of the Sequences, Yudkowsky had been [dismissive of people who aspired to build AI without understanding how intelligence works](https://www.lesswrong.com/posts/fKofLyepu446zRgPP/artificial-mysterious-intelligence)—for example, by being overly impressed by the [surface analogy](https://www.lesswrong.com/posts/6ByPxcGDhmx74gPSm/surface-analogies-and-deep-causes) between artificial neural networks and the brain. He conceded the possibility of brute-forcing AI (if natural selection had eventually gotten there with no deeper insight, so could we) but didn't consider it a default and especially not a desirable path. (["If you don't know how your AI works, that is not good. It is bad."](https://www.lesswrong.com/posts/fKofLyepu446zRgPP/artificial-mysterious-intelligence))
28 These days, it's increasingly looking like making really large neural nets ... [actually works](https://www.gwern.net/Scaling-hypothesis)?—which seems like bad news; if it's "easy" for non-scientific-genius engineering talent to shovel large amounts of compute into the birth of powerful minds that we don't understand and don't know how to control, then it would seem that the world is soon to pass outside of our understanding and control.
30 [^second-half]: In an unfinished slice-of-life short story I started writing _circa_ 2010, my protagonist (a supermarket employee resenting his job while thinking high-minded thoughts about rationality and the universe) speculates about "a threshold of economic efficiency beyond which nothing human could survive" being a tighter bound on future history than physical limits (like the heat death of the universe), and comments that "it imposes a sense of urgency to suddenly be faced with the fabric of your existence coming apart in ninety years rather than 10<sup>90</sup>."
32 But if ninety years is urgent, what about ... nine? Looking at what deep learning can do in 2023, the idea of Singularity 2032 doesn't seem self-evidently _absurd_ in the way that Singularity 2019 seemed absurd in 2010 (correctly, as it turned out).
34 My AlphaGo moment was 5 January 2021, when OpenAI released [DALL-E](https://openai.com/blog/dall-e/) (by far the most significant news story of [that week in January 2021](https://en.wikipedia.org/wiki/January_6_United_States_Capitol_attack)). Previous AI milestones, like GANs for a _fixed_ image class, were easier to dismiss as clever statistical tricks. If you have thousands of photographs of people's faces, I didn't feel surprised that some clever algorithm could "learn the distribution" and spit out another sample; I don't know the _details_, but it doesn't seem like scary "understanding." DALL-E's ability to _combine_ concepts—responding to "an armchair in the shape of an avacado" as a novel text prompt, rather than already having thousands of examples of avacado-chairs and just spitting out another one of those—viscerally seemed more like "real" creativity to me, something qualitatively new and scary.[^qualitatively-new]
36 [^qualitatively-new]: By mid-2022, DALL-E 2 and Midjourney and Stable Diffusion were generating much better pictures, but that wasn't surprising. Seeing AI being able to do a thing at all is the model update; AI being able to do the thing much better 18 months later feels "priced in."
38 [As recently as 2020, I had been daydreaming about](/2020/Aug/memento-mori/#if-we-even-have-enough-time) working at an embryo selection company (if they needed programmers—but everyone needs programmers, these days), and having that be my altruistic[^eugenics-altruism] contribution to the great common task. Existing companies working on embryo selection [boringly](https://archive.is/tXNbU) [market](https://archive.is/HwokV) their services as being about promoting health, but [polygenic scores should work as well for maximizing IQ as they do for minimizing cancer risk](https://www.gwern.net/Embryo-selection).[^polygenic-score] Making smarter people would be a transhumanist good in its own right, and [having smarter biological humans around at the time of our civilization's AI transition](https://www.lesswrong.com/posts/2KNN9WPcyto7QH9pi/this-failing-earth) would give us a better shot at having it go well.[^ai-transition-go-well]
40 [^eugenics-altruism]: If it seems odd to frame _eugenics_ as "altruistic", translate it as a term of art referring to the component of my actions dedicating to optimizing the world at large, as contrasted to "selfishly" optimizing my own experiences.
42 [^polygenic-score]: Better, actually: [the heritability of IQ is around 0.65](https://en.wikipedia.org/wiki/Heritability_of_IQ), as contrasted to [about 0.33 for cancer risk](https://pubmed.ncbi.nlm.nih.gov/26746459/).
44 [^ai-transition-go-well]: Natural selection eventually developed intelligent creatures, but evolution didn't know what it was doing and was not foresightfully steering the outcome in any particular direction. The more humans know what we're doing, the more our will determines the fate of the cosmos; the less we know what we're doing, the more our civilization is just another primordial soup for the next evolutionary transition.
46 But pushing on embryo selection only makes sense as an intervention for optimizing the future if AI timelines are sufficiently long, and the breathtaking pace (or too-fast-to-even-take-a-breath pace) of the deep learning revolution is so much faster than the pace of human generations, that it's starting to look unlikely that we'll get that much time. If our genetically uplifted children would need at least twenty years to grow up to be productive alignment researchers, but unaligned AI is [on track to end the world in twenty years](https://www.lesswrong.com/posts/AfH2oPHCApdKicM4m/two-year-update-on-my-personal-ai-timelines), we would need to start having those children _now_ in order for them to make any difference at all.
48 [It's ironic that "longtermism" got traction as the word for the cause area of benefitting the far future](https://applieddivinitystudies.com/longtermism-irony/), because the decision-relevant beliefs of most of the people who think about the far future, end up working out to extreme _short_-termism. Common-sense longtermism—a longtermism that assumed there's still going to be a recognizable world of humans in 2123—_would_ care about eugenics, and would be willing to absorb political costs today in order to fight for a saner future. The story of humanity would not have gone _better_ if Galileo had declined to publish for pre-emptive fear of the Inquisition.
50 But if you think the only hope for there _being_ a future flows through maintaining influence over what large tech companies are doing as they build transformative AI, declining to contradict the state religion makes more sense—if you don't have _time_ to win a culture war, because you need to grab hold of the Singularity (or perform a [pivotal act](https://arbital.com/p/pivotal/) to prevent it) _now_. If the progressive machine marks you as a transphobic bigot, the machine's functionaries at OpenAI or Meta AI Research are less likely to listen to you when you explain why [their safety plan](https://openai.com/blog/our-approach-to-alignment-research/) won't work, or why they should have a safety plan at all.
52 (I remarked to "Wilhelm" in mid-2022 that DeepMind [changing its Twitter avatar to a rainbow variant of their logo for Pride month](https://web.archive.org/web/20220607123748/https://twitter.com/DeepMind) was a bad sign.)
54 So isn't there a story here where I'm the villain, willfully damaging humanity's chances of survival by picking unimportant culture-war fights in the xrisk-reduction social sphere, when _I know_ that the sphere needs to keep its nose clean in the eyes of the progressive egregore? _That's_ why Yudkowsky said the arguably-technically-misleading things he said about my Something to Protect: he _had_ to, to keep our collective nose clean. The people paying attention to contemporary politics don't know what I know, and can't usefully be told. Isn't it better for humanity if my meager talents are allocated to making AI go well? Don't I have a responsibility to fall in line and take one for the team—if the world is at stake?
56 As usual, the Yudkowsky of 2009 has me covered. In his short story ["The Sword of Good"](https://www.yudkowsky.net/other/fiction/the-sword-of-good), our protagonist Hirou wonders why the powerful wizard Dolf lets other party members risk themselves fighting, when Dolf could have protected them:
58 > _Because Dolf was more important, and if he exposed himself to all the risk every time, he might eventually be injured_, Hirou's logical mind completed the thought. _Lower risk, but higher stakes. Cold but necessary—_
60 > _But would you_, said another part of his mind, _would you, Hirou, let your friends walk before you and fight, and occasionally die, if you_ knew _that you yourself were stronger and able to protect them? Would you be able to stop yourself from stepping in front?_
62 > _Perhaps_, replied the cold logic. _If the world were at stake._
64 > _Perhaps_, echoed the other part of himself, _but that is not what was actually happening._
66 That is, there's _no story_ under which misleading people about trans issues is on Yudkowsky's critical path for shaping the intelligence explosion. _I'd_ prefer him to have free speech, but if _he_ thinks he can't afford to be honest about things he [_already_ got right in 2009](https://www.lesswrong.com/posts/QZs4vkC7cbyjL9XA9/changing-emotions), he could just—not issue pronouncements on topics where he intends to _ignore counterarguments on political grounds!_
68 In [a March 2021 Twitter discussion about why not to trust organizations that refuse to explain their reasoning, Yudkowsky wrote](https://twitter.com/esyudkowsky/status/1374161729073020937):
70 > Having some things you say "no comment" to, is not at _all_ the same phenomenon as being an organization that issues Pronouncements. There are a _lot_ of good reasons to have "no comments" about things. Anybody who tells you otherwise has no life experience, or is lying.
72 Sure. But if that's your story, I think you need to _actually not comment_. ["[A]t least 20% of the ones with penises are actually women"](https://www.facebook.com/yudkowsky/posts/10154078468809228) is _not "no comment"._ ["[Y]ou're not standing in defense of truth if you insist on a word, brought explicitly into question, being used with some particular meaning"](https://twitter.com/ESYudkowsky/status/1067198993485058048) is _not "no comment"_. We [did get a clarification on that one](https://www.facebook.com/yudkowsky/posts/10158853851009228)—but then, within a matter of months, he turned around and came back with his "simplest and best proposal" about how the "important things [...] would be all the things [he's] read [...] from human beings who are people—describing reasons someone does not like to be tossed into a Male Bucket or Female Bucket, as it would be assigned by their birth certificate", _which is also not "no comment."_
74 It's a little uncomfortable that I seem to be arguing for a duty to self-censorship here. If he has selected "pro-trans" arguments he feels safe publishing, what's the harm in publishing them? How could I object to the addition of more Speech to the discourse?
76 But I don't think it's the mere addition of the arguments to the discourse that I'm objecting to. (If some garden-variety trans ally had made the same dumb arguments, I would make the same counterarguments, but I wouldn't feel betrayed.)
78 It's the _false advertising_—the pretense that Yudkowsky is still the unchallengable world master of rationality, if he's going to behave like a garden-variety trans ally and reserve the right to _ignore counterarguments on political grounds_ (!!) when his incentives point that way.
80 In _Planecrash_, when Keltham decides he needs to destroy Golarion's universe on negative-leaning utilitarian grounds, he takes care to only deal with Evil people from then on, and not form close ties with the Lawful Neutral nation of Osirion, in order to not betray anyone who would have had thereby a reasonable expectation that their friend wouldn't try to destroy their universe: ["the stranger from dath ilan never pretended to be anyone's friend after he stopped being their friend"](https://glowfic.com/replies/1882395#reply-1882395).
82 Similarly, I think Yudkowsky should stop pretending to be our rationality teacher after he stopped being our rationality teacher and decided to be a politician instead.
84 I think it's significant that you don't see me picking fights with—say, Paul Christiano, because Paul Christiano doesn't repeatedly take a shit on my Something to Protect, because Paul Christiano _isn't trying to be a religious leader_ (in this world where religious entrepreneurs can't afford to contradict the state religion). If Paul Christiano has opinions about transgenderism, we don't know about them. If we knew about them and they were correct, I would upvote them, and if we knew about them and they were incorrect, I would criticize them, but in either case, Christiano would not try to cultivate the impression that anyone who disagrees with him is insane. That's not his bag.
88 Yudkowsky's political cowardice is arguably puzzling in light of his timeless decision theory's recommendations against giving in to extortion.
90 The "arguably" is important, because randos on the internet are notoriously bad at drawing out the consequences of the theory, to the extent that Yudkowsky has said that he wishes he hadn't published—and though I think I'm smarter than the average rando, I don't expect anyone to _take my word for it_. So let me disclaim that this is _my_ explanation of how Yudkowsky's decision theory _could be interpreted_ to recommend that he behave the way I want him to, without any pretense that I'm any sort of neutral expert witness on decision theory.
92 The idea of timeless decision theory is that you should choose the action that has the best consequences _given_ that your decision is mirrored at all the places your decision algorithm is embedded in the universe.
94 The reason this is any different from the "causal decision theory" of just choosing the action with the best consequences (locally, without any regard to this "multiple embeddings in the universe" nonsense) is because it's possible for other parts of the universe to depend on your choices. For example, in the "Parfit's Hitchhiker" scenario, someone might give you a ride out of the desert if they _predict_ you'll pay them back later. After you've already received the ride, you might think that you can get away with stiffing them—but if they'd predicted you would do that, they wouldn't have given you the ride in the first place. Your decision is mirrored _inside the world-model every other agent with a sufficiently good knowledge of you_.
96 In particular, if you're the kind of agent that gives in to extortion—if you respond to threats of the form "Do what I want, or I'll hurt you" by doing what the threatener wants—that gives other agents an incentive to spend resources trying to extort you. On the other hand, if any would-be extortionist knows you'll never give in, they have no reason to bother trying. This is where the standard ["Don't negotiate with terrorists"](/2018/Jan/dont-negotiate-with-terrorist-memeplexes/) advice comes from.
98 So, naïvely, doesn't Yudkowsky's "personally prudent to post your agreement with Stalin"[^gambit] gambit constitute giving in to an extortion threat of the form, "support the progressive position, or we'll hurt you", which Yudkowsky's own decision theory says not to do?
100 [^gambit]: In _ways that exhibit generally rationalist principles_, natch.
102 I can think of two reasons why the naïve objection might fail. (And who can say but that a neutral expert witness on decision theory wouldn't think of more?)
104 First, the true decision theory is subtler than "defy anything that you can commonsensically pattern-match as looking like 'extortion'"; the case for resisting extortion specifically rests on there existing a subjunctive dependence between your decision and the extortionist's decision: they threaten _because_ you'll give in, or don't bother _because_ you won't.
106 Okay, but then how do I compute this "subjunctive dependence" thing? Presumably it has something to do with the extortionist's decisionmaking process incuding a model of the target. How good does that model have to be for it to "count"?
108 I don't know—and if I don't know, I can't say that the relevant subjunctive dependence obviously pertains in the real-life science intellectual _vs._ social justice mob match-up. If the mob has been trained from past experience to predict that their targets will give in, should you defy them now in order to somehow make your current predicament "less real"? Depending on the correct theory of logical counterfactuals, the correct stance might be "We don't negotiate with terrorists, but [we do appease bears](/2019/Dec/political-science-epigrams/) and avoid avalanches" (because neither the bear's nor the avalanche's behavior is calculated based on our response), and the forces of political orthodoxy might be relevantly bear- or avalanche-like.
110 On the other hand, the relevant subjunctive dependence doesn't obviously _not_ pertain, either! Yudkowsky does seem to endorse commonsense pattern-matching to "extortion" in contexts [like nuclear diplomacy](https://twitter.com/ESYudkowsky/status/1580278376673120256). Or I remember back in 'aught-nine, Tyler Emerson was caught embezzling funds from the Singularity Institute, and SingInst made it a point of pride to prosecute on decision-theoretic grounds, when a lot of other nonprofits would have quietly and causal-decision-theoretically covered it up to spare themselves the embarrassment. Parsing social justice as an agentic "threat" rather than a non-agentic obstacle like an avalanche, does seem to line up with the fact that people punish heretics (who dissent from an ideological group) more than infidels (who were never part of the group to begin with), _because_ heretics are more extortable—more vulnerable to social punishment from the original group.
112 Which brings me to the second reason the naïve anti-extortion argument might fail: [what counts as "extortion" depends on the relevant "property rights", what the "default" action is](https://www.lesswrong.com/posts/Qjaaux3XnLBwomuNK/countess-and-baron-attempt-to-define-blackmail-fail). If having free speech is the default, being excluded from the dominant coalition for defying the orthodoxy could be construed as extortion. But if _being excluded from the coalition_ is the default, maybe toeing the line of orthodoxy is the price you need to pay in order to be included.
114 Yudkowsky has [a proposal for how bargaining should work between agents with different notions of "fairness"](https://www.lesswrong.com/posts/z2YwmzuT7nWx62Kfh/cooperating-with-agents-with-different-ideas-of-fairness).
116 Suppose Edgar and Fiona are splitting a pie, and if they can't initially agree on how to split it, they have to fight over it until they do, destroying some of the pie in the process. Edgar thinks the fair outcome is that they each get half the pie. Fiona claims that she contributed more ingredients to the baking process and that it's therefore fair that she gets 75% of the pie, pledging to fight if offered anything less.
118 If Edgar were a causal decision theorist, he might agree to the 75/25 split, reasoning that 25% of the pie is better than fighting until the pie is destroyed. Yudkowsky argues that this is irrational: if Edgar is willing to agree to a 75/25 split, then Fiona has no incentive not to adopt such a self-favoring definition of "fairness". (And _vice versa_ if Fiona's concept of fairness is the "correct" one.)
120 Instead, Yudkowsky argues, Edgar should behave so as to only do worse than the fair outcome if Fiona _also_ does worse: for example, by accepting a 48/32 split (after 100−(32+48) = 20% of the pie has been destroyed by the costs of fighting) or an 42/18 split (where 40% of the pie has been destroyed). This isn't Pareto-optimal (it would be possible for both Edgar and Fiona to get more pie by reaching an agreement with less fighting), but it's worth it to Edgar to burn some of Fiona's utility fighting in order to resist being exploited by her, and at least it's better than the equilibrium where the pie gets destroyed (which is Nash because neither party can unilaterally stop fighting).
122 It seemed to me that in the contest over the pie of Society's shared map, the rationalist Caliphate was letting itself get exploited by the progressive Egregore, doing worse than the fair outcome without dealing any damage to the egregore in return. Why?
124 The logic of "dump stats", presumably. Bargaining to get AI risk on the shared map—not even to get it taken seriously as we would count "taking it seriously", but just acknowledged at all—was hard enough. Trying to challenge the Egregore about an item that it actually cared about would trigger more fighting than we could afford.
126 I told the illustration about splitting a pie as a symmetrical story: if Edgar and Fiona destroy the pie fighting, than neither of them get any pie. But in more complicated scenarios (including the real world), there was no guarantee that non-Pareto Nash equilibria were equally bad for everyone.
128 I'd had a Twitter exchange with Yudkowsky in January 2020 that revealed some of his current-year thinking about Nash equilibria. I [had Tweeted](https://twitter.com/zackmdavis/status/1206718983115698176):
130 > 1940s war criminal defense: "I was only following orders!"
131 > 2020s war criminal defense: "I was only participating in a bad Nash equilibrium that no single actor can defy unilaterally!"
133 (The language of the latter being [a reference to Yudkowsky's _Inadequate Equilibria_](https://equilibriabook.com/molochs-toolbox/).)
135 Yudkowsky [quote-Tweet dunked on me](https://twitter.com/ESYudkowsky/status/1216788984367419392):
137 > Well, YES. Paying taxes to the organization that runs ICE, or voting for whichever politician runs against Trump, or trading with a doctor benefiting from an occupational licensing regime; these acts would all be great evils if you weren't trapped.
139 I pointed out the voting case as one where he seemed to be disagreeing with his past self, linking to 2008's ["Stop Voting for Nincompoops"](https://www.lesswrong.com/posts/k5qPoHFgjyxtvYsm7/stop-voting-for-nincompoops). What changed his mind?
141 "Improved model of the social climate where revolutions are much less startable or controllable by good actors," he said. "Having spent more time chewing on Nash equilibria, and realizing that the trap is _real_ and can't be defied away even if it's very unpleasant."
143 In response to Sarah Constantin mentioning that there was no personal cost to voting third-party, Yudkowsky [pointed out that](https://twitter.com/ESYudkowsky/status/1216809977144168448) the problem was the [third-party spoiler effect](https://en.wikipedia.org/wiki/Vote_splitting), not personal cost: "People who refused to vote for Hillary didn't pay the price, kids in cages did, but that still makes the action nonbest."
145 (The cages in question—technically, chain-link fence enclosures—were [actually](https://www.usatoday.com/story/news/factcheck/2020/08/26/fact-check-obama-administration-built-migrant-cages-meme-true/3413683001/) [built](https://apnews.com/article/election-2020-democratic-national-convention-ap-fact-check-immigration-politics-2663c84832a13cdd7a8233becfc7a5f3) during the Obama administration, but that doesn't seem important.)
147 I asked what was wrong with the disjunction from "Stop Voting for Nincompoops", where the earlier Yudkowsky had written that it's hard to see who should accept the argument to vote for the lesser of two evils, but refuse to accept the argument against voting because it won't make a difference. Unilaterally voting for Clinton doesn't save the kids!
149 "Vote when you're part of a decision-theoretic logical cohort large enough to change things, or when you're worried about your reputation and want to be honest about whether you voted," Yudkowsky replied.
151 "How do I compute whether I'm in a large enough decision-theoretic cohort?" I asked. Did we know that, or was that still on the open problems list?
153 Yudkowsky said that he [traded his vote for a Clinton swing state vote](https://en.wikipedia.org/wiki/Vote_pairing_in_the_2016_United_States_presidential_election), partially hoping that that would scale, "but maybe to a larger degree because [he] anticipated being asked in the future if [he'd] acted against Trump".
155 The reputational argument seems in line with Yudkowsky's [pathological obsession with not-technically-lying](https://www.lesswrong.com/posts/MN4NRkMw7ggt9587K/firming-up-not-lying-around-its-edge-cases-is-less-broadly). People asking if you acted against Trump are looking for a signal of coalitional loyalty. By telling them he traded his vote, Yudkowsky can pass their test without lying.
157 I guess that explains everything. He doesn't think he's part of a decision-theoretic logical cohort large enough to change things. He's not anticipating being asked in the future if he's acted against gender ideology. He's not worried about his reputation with people like me.
159 Curtis Yarvin [likes to compare](/2020/Aug/yarvin-on-less-wrong/) Yudkowsky to Sabbatai Zevi, the 17th-century Jewish religious leader purported to be the Messiah, who later [converted to Islam under coercion from the Ottomans](https://en.wikipedia.org/wiki/Sabbatai_Zevi#Conversion_to_Islam). "I know, without a shadow of a doubt, that in the same position, Eliezer Yudkowsky would also convert to Islam," said Yarvin.
161 I don't think this is as much of a burn as Yarvin does. Zevi was facing some very harsh coercion: a choice to convert to Islam, "prove" his divinity via deadly trial by ordeal, or just be impaled outright. Extortion-resistant decision theories aside, it's hard not to be sympathetic to someone facing this trilemma who chose to convert.
163 So to me, the more damning question is this—
165 If in the same position as Yudkowsky, would Sabbatai Zevi declare that 30% of the ones with penises are actually women?
169 I like to imagine that they have a saying out of dath ilan: once is happenstance; twice is coincidence; _three times is hostile optimization_.
171 I could forgive him for taking a shit on d4 of my chessboard (["at least 20% of the ones with penises are actually women"](https://www.facebook.com/yudkowsky/posts/10154078468809228)).
173 I could even forgive him for subsequently taking a shit on e4 of my chessboard (["you're not standing in defense of truth if you insist on a word [...]"](https://twitter.com/ESYudkowsky/status/1067198993485058048)) as long as he wiped most of the shit off afterwards (["you are being the bad guy if you try to shut down that conversation by saying that 'I can define the word "woman" any way I want'"](https://www.facebook.com/yudkowsky/posts/10158853851009228)), even though, really, I would have expected someone so smart to take a hint after the incident on d4.
175 But if he's _then_ going to take a shit on c3 of my chessboard (["important things [...] would be all the things I've read [...] from human beings who are people—describing reasons someone does not like to be tossed into a Male Bucket or Female Bucket, as it would be assigned by their birth certificate", "the simplest and best protocol is, '"He" refers to the set of people who have asked us to use "he"'"](https://www.facebook.com/yudkowsky/posts/10159421750419228)), the "playing on a different chessboard, no harm intended" excuse loses its credibility. The turd on c3 is a pretty big likelihood ratio! (That is, I'm more likely to observe a turd on c3 in worlds where Yudkowsky _is_ playing my chessboard and wants me to lose, than in world where he's playing on a different chessboard and just _happened_ to take a shit there, by coincidence.)
179 I got my COVID-19 vaccine (the one-shot Johnson & Johnson) on 3 April 2021, so I was able to visit Valinor again on 17 April, for the first time in fourteen months.
181 I had previously dropped by in January to deliver two new board books I had made, _Koios Blume Is Preternaturally Photogenic_ and _Amelia Davis Ford and the Great Plague_, but that had been a socially-distanced book delivery, not a "visit".
183 The copy of _Amelia Davis Ford and the Great Plague_ that I sent to my sister in Cambridge differed slightly from the one I brought to Valinor. There was an "Other books by the author" list on the back cover with the titles of my earlier board books. In the Cambridge edition of _Great Plague_, the previous titles were printed in full: _Merlin Blume and the Methods of Pre-Rationality_, _Merlin Blume and the Steerswoman's Oath_, _Merlin Blume and the Sibling Rivalry_. Whereas in _Preternaturally Photogenic_ and the Valinor edition of _Great Plague_, the previous titles were abbreviated: _The Methods of Pre-Rationality_, _The Steerswoman's Oath_, _The Sibling Rivalry_.
185 The visit on the seventeenth went fine. I hung out, talked, played with the kids. I had made a double-dog promise to be on my best no-politics-and-religion-at-the-dinner-table behavior.
187 At dinner, there was a moment when Koios bit into a lemon and made a funny face, to which a bunch of the grown-ups said "Awww!" A few moments later, he went for the lemon again. Alicorn speculated that Koios had noticed that the grown-ups found it cute the first time, and the grown-ups were chastened. "Aww, baby, we love you even if you don't bite the lemon."
189 It was very striking to me how, in the case of the baby biting a lemon, Alicorn _immediately_ formulated the hypothesis that what-the-grownups-thought-was-cute was affecting the baby's behavior, and everyone _immediately just got it_. I was tempted to say something caustic about how no one seemed to think a similar mechanism could have accounted for some of the older child's verbal behavior the previous year, but I kept silent; that was clearly outside the purview of my double-dog promise.
191 There was another moment when Mike made a remark about how weekends are socially constructed. I had a lot of genuinely on-topic cached witty philosophy banter about [how the social construction of concepts works](https://www.lesswrong.com/posts/edEXi4SpkXfvaX42j/schelling-categories-and-simple-membership-tests), that would have been completely innocuous if anyone _else_ had said it, but I kept silent because I wasn't sure if it was within my double-dog margin of error if _I_ said it.
195 In June 2021, MIRI Executive Director Nate Soares [wrote a Twitter thread aruging that](https://twitter.com/So8res/status/1401670792409014273) "[t]he definitional gynmastics required to believe that dolphins aren't fish are staggering", which [Yudkowsky retweeted](https://archive.is/Ecsca).[^not-endorsements]
197 [^not-endorsements]: In general, retweets are not necessarily endorsements—sometimes people just want to draw attention to some content without further comment or implied approval—but I was inclined to read this instance as implying approval, partially because this doesn't seem like the kind of thing someone would retweet for attention-without-approval, and partially because of the working relationship between Soares and Yudkowsky.
199 Soares's points seemed cribbed from part I of Scott Alexander's ["... Not Man for the Categories"](https://slatestarcodex.com/2014/11/21/the-categories-were-made-for-man-not-man-for-the-categories/), which post I had just dedicated _more than three years of my life_ to rebutting in [increasing](/2018/Feb/the-categories-were-made-for-man-to-make-predictions/) [technical](https://www.lesswrong.com/posts/esRZaPXSHgWzyB2NL/where-to-draw-the-boundaries) [detail](https://www.lesswrong.com/posts/onwgTH6n8wxRSo2BJ/unnatural-categories-are-optimized-for-deception), _specifically using dolphins as my central example_—which Soares didn't necessarily have any reason to have known about, but Yudkowsky (who retweeted Soares) definitely did. (Soares's [specific reference to the Book of Jonah](https://twitter.com/So8res/status/1401670796997660675) made it seem particularly unlikely that he had invented the argument independently from Alexander.) [One of the replies (which Soares Liked) pointed out the similar _Slate Star Codex_ article](https://twitter.com/max_sixty/status/1401688892940509185), [as did](https://twitter.com/NisanVile/status/1401684128450367489) [a couple of](https://twitter.com/roblogic_/status/1401699930293432321) quote-Tweet discussions.
201 The elephant in my brain took this as another occasion to _flip out_. I didn't _immediately_ see anything for me to overtly object to in the thread itself—[I readily conceded that](https://twitter.com/zackmdavis/status/1402073131276066821) there was nothing necessarily wrong with wanting to use the symbol "fish" to refer to the cluster of similarities induced by convergent evolution to the acquatic habitat rather than the cluster of similarities induced by phylogenetic relatedness—but in the context of our subculture's history, I read this as Soares and Yudkowsky implicitly lending more legitimacy to "... Not Man for the Categories", which was _hostile to my interests_. Was I paranoid to read this as a potential [dogwhistle](https://en.wikipedia.org/wiki/Dog_whistle_(politics))? It just seemed _implausible_ that Soares would be Tweeting that dolphins are fish in the counterfactual in which "... Not Man for the Categories" had never been published.
203 After a little more thought, I decided the thread _was_ overtly objectionable, and [quickly wrote up a reply on _Less Wrong_](https://www.lesswrong.com/posts/aJnaMv8pFQAfi9jBm/reply-to-nate-soares-on-dolphins): Soares wasn't merely advocating for a "swimmy animals" sense of the word _fish_ to become more accepted usage, but specifically deriding phylogenetic definitions as unmotivated for everyday use ("definitional gynmastics [_sic_]"!), and _that_ was wrong. It's true that most language users don't directly care about evolutionary relatedness, but [words aren't identical with their definitions](https://www.lesswrong.com/posts/i2dfY65JciebF3CAo/empty-labels). Genetics is at the root of the causal graph underlying all other features of an organism; creatures that are more closely evolutionarily related are more similar _in general_. Classifying things by evolutionary lineage isn't an arbitrary æsthetic whim by people who care about geneology for no reason. We need the natural category of "mammals (including marine mammals)" to make sense of how dolphins are warm-blooded, breathe air, and nurse their live-born young, and the natural category of "finned cold-blooded vertebrate gill-breathing swimmy animals (which excludes marine mammals)" is also something that it's reasonable to have a word for.
205 (Somehow, it felt appropriate to use a quote from Arthur Jensen's ["How Much Can We Boost IQ and Scholastic Achievement?"](https://en.wikipedia.org/wiki/How_Much_Can_We_Boost_IQ_and_Scholastic_Achievement%3F) as an epigraph.)
207 [TODO: dolphin war con'td
209 * Nate conceded all of my points (https://twitter.com/So8res/status/1402888263593959433), said the thread was in jest ("shitposting"), and said he was open to arguments that he was making a mistake (https://twitter.com/So8res/status/1402889976438611968), but still seemed to think his shitposting was based
211 * I got frustrated and lashed out; "open to arguments that he was making a mistake" felt fake to me; rats are good at paying lip service to humility, but I'd lost faith in getting them to change their behavior, like not sending PageRank to "... Not Man for the Categories"
213 * Nate wrote a longer reply on Less Wrong the next morning
215 * I pointed out that his followup thread lamented that people hadn't read "A Human's Guide to Words", but that Sequence _specifically_ used the example of dolphins. What changed?!?
217 * [Summarize Nate's account of his story], phylogeny not having the courage of its convictions
219 * Twitter exchange where he said he wasn't sure I would count his self-report as evidnece, I said it totally counts
221 * I overheated. This was an objectively dumb play. (If I had cooled down and just written up my reply, I might have gotten real engagement and a resolution, but I blew it.) I apologized a few days later.
223 * Nate's reaction to me blowing up said it looked like I was expecting deference. I deny this; I wouldn't expect people to defer to me—what I did expect was a fair hearing, and at this point, I had lost faith that I would get one. (Could you blame me, when Yudkowsky says a fair hearing is less important than agreeing with Stalin?)
225 * My theory of what's going on: I totally believe Nate's self report that he wasn't thinking about gender. (As Nate pointed out, you could give the thread an anti-trans interpretation, too.) Nevertheless, it remains the case that Nate's thinking is causally downstream of Scott's arguments in "... Not Man for the Categories." Where did Scott get it from? I think he pulled it out of his ass because it was politically convenient.
227 * This is like radiocontrast dye for dark side epistemology: we can see Scott sneezing his bad epistemology onto everyone else because he's such a popular writer. No one can think fast enough to think their own thoughts, but you would hope for an intellectual community that can do error-correction, rather than copying smart people's views including mistakes.
229 * I look up the relevant phylogenetics definitions, and write "Blood Is Thicker Than Water"
236 * depressed after talking to him at Independence Day party 2021 (I can mention that, because it was outdoors and probably lots of other people saw us, even if I can't talk about content)
238 * It wouldn't be so bad if he weren't trying to sell himself as a religious leader, and profiting from the conflation of rationalist-someone-who-cares-about-reasoning, and rationalist-member-of-robot-cult
240 * But he does, in fact, seem to actively encourage this conflation (contrast to how the Sequences had a [Litany Against Gurus](https://www.lesswrong.com/posts/t6Fe2PsEwb3HhcBEr/the-litany-against-gurus) these days, with the way he sneers as Earthlings and post-rats)
242 * a specific example that made me very angry in September 2021—
244 https://twitter.com/ESYudkowsky/status/1434906470248636419
245 > Anyways, Scott, this is just the usual division of labor in our caliphate: we're both always right, but you cater to the crowd that wants to hear it from somebody too modest to admit that, and I cater to the crowd that wants somebody out of that closet.
247 Okay, I get that it was meant as humorous exaggeration. But I think it still has the effect of discouraging people from criticizing Scott or Eliezer because they're the leaders of the Caliphate. I spent three and a half years of my life explaining in exhaustive, exhaustive detail, with math, how Scott was wrong about something, no one serious actually disagrees, and Eliezer is still using his social power to boost Scott's right-about-everything (!!) reputation. That seems really unfair, in a way that isn't dulled by "it was just a joke."
249 Or [as Yudkowsky put it](https://www.facebook.com/yudkowsky/posts/10154981483669228)—
251 > I know that it's a bad sign to worry about which jokes other people find funny. But you can laugh at jokes about Jews arguing with each other, and laugh at jokes about Jews secretly being in charge of the world, and not laugh at jokes about Jews cheating their customers. Jokes do reveal conceptual links and some conceptual links are more problematic than others.
253 It's totally understandable to not want to get involved in a political scuffle because xrisk reduction is astronomically more important! But I don't see any plausible case that metaphorically sucking Scott's dick in public reduces xrisk. It would be so easy to just not engage in this kind of cartel behavior!
255 An analogy: racist jokes are also just jokes. Alice says, "What's the difference between a black dad and a boomerang? A boomerang comes back." Bob says, "That's super racist! Tons of African-American fathers are devoted parents!!" Alice says, "Chill out, it was just a joke." In a way, Alice is right. It was just a joke; no sane person could think that Alice was literally claiming that all black men are deadbeat dads. But, the joke only makes sense in the first place in context of a culture where the black-father-abandonment stereotype is operative. If you thought the stereotype was false, or if you were worried about it being a self-fulfilling prophecy, you would find it tempting to be a humorless scold and get angry at the joke-teller.
257 Similarly, the "Caliphate" humor _only makes sense in the first place_ in the context of a celebrity culture where deferring to Yudkowsky and Alexander is expected behavior. (In a way that deferring to Julia Galef or John S. Wentworth is not expected behavior, even if Galef and Wentworth also have a track record as good thinkers.) I think this culture is bad. _Nullius in verba_.
259 * the fact that David Xu interpreted criticism of the robot cult as me going "full post-rat" suggests that Yudkowsky's framing had spilled onto others. (The framing is optimized to delegitimize dissent. Motte: someone who's critical of central rationalists; bailey: someone who's moved beyond reason.)
261 sneering at post-rats; David Xu interprets criticism of Eliezer as me going "full post-rat"?! 6 September 2021
263 > Also: speaking as someone who's read and enjoyed your LW content, I do hope this isn't a sign that you're going full post-rat. It was bad enough when QC did it (though to his credit QC still has pretty decent Twitter takes, unlike most post-rats).
265 https://twitter.com/davidxu90/status/1435106339550740482
267 https://twitter.com/zackmdavis/status/1435856644076830721
268 > The error in "Not Man for the Categories" is not subtle! After the issue had been brought to your attention, I think you should have been able to condemn it: "Scott's wrong; you can't redefine concepts in order to make people happy; that's retarded." It really is that simple! 4/6
270 I once wrote [a post whimsically suggesting that trans women should owe cis women royalties](/2019/Dec/comp/) for copying the female form (as "intellectual property"). In response to a reader who got offended, I [ended up adding](/source?p=Ultimately_Untrue_Thought.git;a=commitdiff;h=03468d274f5) an "epistemic status" line to clarify that it was not a serious proposal.
272 But if knowing it was a joke partially mollifies the offended reader who thought I might have been serious, I don't think they should be _completely_ mollified, because the joke (while a joke) reflects something about my thinking when I'm being serious: I don't think sex-based collective rights are inherently a suspect idea; I think _something of value has been lost_ when women who want female-only spaces can't have them, and the joke reflects the conceptual link between the idea that something of value has been lost, and the idea that people who have lost something of value are entitled to compensation.
274 At Valinor's 2022 [Smallpox Eradication Day](https://twitter.com/KelseyTuoc/status/1391248651167494146) party, I remember overhearing[^overhearing] Yudkowsky saying that OpenAI should have used GPT-3 to mass-promote the Moderna COVID-19 vaccine to Republicans and the Pfizer vaccine to Democrats (or vice versa), thereby harnessing the forces of tribalism in the service of public health.
276 [^overhearing]: I claim that conversations at a party with lots of people are not protected by privacy norms; if I heard it, several other people heard it; no one had a reasonable expectation that I shouldn't blog about it.
278 I assume this was not a serious proposal. Knowing it was a joke partially mollifies what offense I would have taken if I thought he might have been serious. But I don't think I should be completely mollified, because I think I think the joke (while a joke) reflects something about Yudkowsky's thinking when he's being serious: that he apparently doesn't think corupting Society's shared maps for utilitarian ends is inherently a suspect idea; he doesn't think truthseeking public discourse is a thing in our world, and the joke reflects the conceptual link between the idea that public discourse isn't a thing, and the idea that a public that can't reason needs to be manipulated by elites into doing good things rather than bad things.
280 My favorite Ben Hoffman post is ["The Humility Argument for Honesty"](http://benjaminrosshoffman.com/humility-argument-honesty/). It's sometimes argued the main reason to be honest is in order to be trusted by others. (As it is written, ["[o]nce someone is known to be a liar, you might as well listen to the whistling of the wind."](https://www.lesswrong.com/posts/K2c3dkKErsqFd28Dh/prices-or-bindings).) Hoffman points out another reason: we should be honest because others will make better decisions if we give them the best information available, rather than worse information that we chose to present in order to manipulate their behavior. If you want your doctor to prescribe you a particular medication, you might be able to arrange that by looking up the symptoms of an appropriate ailment on WebMD, and reporting those to the doctor. But if you report your _actual_ symptoms, the doctor can combine that information with their own expertise to recommend a better treatment.
282 If you _just_ want the public to get vaccinated, I can believe that the Pfizer/Democrats _vs._ Moderna/Republicans propaganda gambit would work. You could even do it without telling any explicit lies, by selectively citing the either the protection or side-effect statistics for each vaccine depending on whom you were talking to. One might ask: if you're not _lying_, what's the problem?
284 The _problem_ is that manipulating people into doing what you want subject to the genre constraint of not telling any explicit lies, isn't the same thing as informing people so that they can make sensible decisions. In reality, both mRNA vaccines are very similar! It would be surprising if the one associated with my political faction happened to be good, whereas the one associated with the other faction happened to be bad. Someone who tried to convince me that Pfizer was good and Moderna was bad would be misinforming me—trying to trap me in a false reality, a world that doesn't quite make sense—with [unforseeable consequences](https://www.lesswrong.com/posts/wyyfFfaRar2jEdeQK/entangled-truths-contagious-lies) for the rest of my decisionmaking. As someone with an interest in living in a world that makes sense, I have reason to regard this as _hostile action_, even if the false reality and the true reality both recommend the isolated point decision of getting vaccinated.
286 (The authors of the [HEXACO personality model](https://en.wikipedia.org/wiki/HEXACO_model_of_personality_structure) may have gotten something importantly right in [grouping "honesty" and "humility" as a single factor](https://en.wikipedia.org/wiki/Honesty-humility_factor_of_the_HEXACO_model_of_personality).)
288 I'm not, overall, satisfied with the political impact of my writing on this blog. One could imagine someone who shared Yudkowsky's apparent disbelief in public reason advising me that my practice of carefully explaining at length what I believe and why, has been an ineffective strategy—that I should instead clarify to myself what policy goal I'm trying to acheive, and try to figure out some clever gambit to play trans activists and gender-critical feminists against each other in a way that advances my agenda.
290 From my perspective, such advice would be missing the point. [I'm not trying to force though some particular policy.](/2021/Sep/i-dont-do-policy/) Rather, I think I _know some things_ about the world, things I wish I had someone had told me earlier. So I'm trying to tell others, to help them live in _a world that makes sense_.
295 [David Xu writes](https://twitter.com/davidxu90/status/1436007025545125896) (with Yudkowsky ["endors[ing] everything [Xu] just said"](https://twitter.com/ESYudkowsky/status/1436025983522381827)):
297 > I'm curious what might count for you as a crux about this; candidate cruxes I could imagine include: whether some categories facilitate inferences that _do_, on the whole, cause more harm than benefit, and if so, whether it is "rational" to rule that such inferences should be avoided when possible, and if so, whether the best way to disallow a large set of potential inferences is [to] proscribe the use of the categories that facilitate them—and if _not_, whether proscribing the use of a category in _public communication_ constitutes "proscribing" it more generally, in a way that interferes with one's ability to perform "rational" thinking in the privacy of one's own mind.
299 > That's four possible (serial) cruxes I listed, one corresponding to each "whether".
301 I reply: on the first and second cruxes, concerning whether some categories facilitate inferences that cause more harm than benefit on the whole and whether they should be avoided when possible, I ask: harm _to whom?_ Not all agents have the same utility function! If some people are harmed by other people making certain probabilistic inferences, then it would seem that there's a _conflict_ between the people harmed (who prefer that such inferences be avoided if possible), and people who want to make and share probabilistic inferences about reality (who think that that which can be destroyed by the truth, should be).
303 On the third crux, whether the best way to disallow a large set of potential inferences is to proscribe the use of the categories that facilitate them: well, it's hard to be sure whether it's the _best_ way: no doubt a more powerful intelligence could search over a larger space of possible strategies than me. But yeah, if your goal is to _prevent people from noticing facts about reality_, then preventing them from using words that refer those facts seems like a pretty effective way to do it!
305 On the fourth crux, whether proscribing the use of a category in public communication constitutes "proscribing" in a way that interferes with one's ability to think in the privacy of one's own mind: I think this is mostly true for humans. We're social animals. To the extent that we can do higher-grade cognition at all, we do it using our language faculties that are designed for communicating with others. How are you supposed to think about things that you don't have words for?
309 > I could have included a fifth and final crux about whether, even _if_ The Thing In Question interfered with rational thinking, that might be worth it; but this I suspect you would not concede, and (being a rationalist) it's not something I'm willing to concede myself, so it's not a crux in a meaningful sense between us (or any two self-proclaimed "rationalists").
311 > My sense is that you have (thus far, in the parts of the public discussion I've had the opportunity to witness) been behaving as though the _one and only crux in play_—that is, the True Source of Disagreement—has been the fifth crux, the thing I refused to include with the others of its kind. Your accusations against the caliphate _only make sense_ if you believe the dividing line between your behavior and theirs is caused by a disagreement as to whether "rational" thinking is "worth it"; as opposed to, say, what kind of prescriptions "rational" thinking entails, and which (if any) of those prescriptions are violated by using a notion of gender (in public, where you do not know in advance who will receive your communications) that does not cause massive psychological damage to some subset of people.
313 > Perhaps it is your argument that all four of the initial cruxes I listed are false; but even if you believe that, it should be within your set of ponderable hypotheses that people might disagree with you about that, and that they might perceive the disagreement to be _about_ that, rather than (say) about whether subscribing to the Blue Tribe view of gender makes them a Bad Rationalist, but That's Okay because it's Politically Convenient.
315 > This is the sense in which I suspect you are coming across as failing to properly Other-model.
317 After everything I've been through over the past six years, I'm inclined to think it's not a "disagreement" at all.
319 It's a _conflict_. I think what's actually at issue is that, at least in this domain, I want people to tell the truth, and the Caliphate wants people to not tell the truth. This isn't a disagreement about rationality, because telling the truth _isn't_ rational _if you don't want people to know things_.
321 At this point, I imagine defenders of the Caliphate are shaking their heads in disappointment at how I'm doubling down on refusing to Other-model. But—_am_ I? Isn't this just a re-statement of Xu's first proposed crux, except reframed as a "values difference" rather than a "disagreement"?
323 Is the problem that my use of the phrase "tell the truth" (which has positive valence in our culture) functions to sneak in normative connotations favoring "my side"?
325 Fine. Objection sustained. I'm happy to use to Xu's language: I think what's actually at issue is that, at least in this domain, I want to facilitate people making inferences (full stop), and the Caliphate wants to _not_ facilitate people making inferences that, on the whole, cause more harm than benefit. This isn't a disagreement about rationality, because facilitating inferences _isn't_ rational _if you don't want people to make inferences_ (for example, because they cause more harm than benefit).
327 Better? Perhaps, to some 2022-era rats and EAs, this formulation makes my position look obviously in the wrong: I'm saying that I'm fine with my inferences _causing more harm than benefit_ (!). Isn't that monstrous of me? Why would someone do that?
329 One of the better explanations of this that I know of was (again, as usual) authored by Yudkowsky in 2007, in a post titled ["Doublethink (Choosing to be Biased)"](https://www.lesswrong.com/posts/Hs3ymqypvhgFMkgLb/doublethink-choosing-to-be-biased).
331 The Yudkowsky of 2007 starts by quoting a passage from George Orwell's _1984_, in which O'Brien (a loyal member of the ruling Party in the totalitarian state depicted in the novel) burns a photograph of Jones, Aaronson, and Rutherford (former Party leaders whose existence has been censored from the historical record). Immediately after burning the photograph, O'Brien denies that it ever existed.
333 The Yudkowsky of 2007 continues—it's again worth quoting at length—
335 > What if self-deception helps us be happy? What if just running out and overcoming bias will make us—gasp!—_unhappy?_ Surely, _true_ wisdom would be _second-order_ rationality, choosing when to be rational. That way you can decide which cognitive biases should govern you, to maximize your happiness.
337 > Leaving the morality aside, I doubt such a lunatic dislocation in the mind could really happen.
341 > For second-order rationality to be genuinely _rational_, you would first need a good model of reality, to extrapolate the consequences of rationality and irrationality. If you then chose to be first-order irrational, you would need to forget this accurate view. And then forget the act of forgetting. I don't mean to commit the logical fallacy of generalizing from fictional evidence, but I think Orwell did a good job of extrapolating where this path leads.
343 > You can't know the consequences of being biased, until you have already debiased yourself. And then it is too late for self-deception.
345 > The other alternative is to choose blindly to remain biased, without any clear idea of the consequences. This is not second-order rationality. It is willful stupidity.
349 > One of chief pieces of advice I give to aspiring rationalists is "Don't try to be clever." And, "Listen to those quiet, nagging doubts." If you don't know, you don't know _what_ you don't know, you don't know how _much_ you don't know, and you don't know how much you _needed_ to know.
351 > There is no second-order rationality. There is only a blind leap into what may or may not be a flaming lava pit. Once you _know_, it will be too late for blindness.
353 Looking back on this from 2022, the only criticism I have is that Yudkowsky was too optimistic to "doubt such a lunatic dislocation in the mind could really happen." In some ways, people's actual behavior is _worse_ than what Orwell depicted. The Party of Orwell's _1984_ covers its tracks: O'Brien takes care to burn the photograph _before_ denying memory of it, because it would be _too_ absurd for him to act like the photo had never existed while it was still right there in front of him.
355 In contrast, Yudkowsky's Caliphate of the current year _doesn't even bother covering its tracks_. Turns out, it doesn't need to! People just don't remember things!
357 The [flexibility of natural language is a _huge_ help here](https://www.lesswrong.com/posts/MN4NRkMw7ggt9587K/firming-up-not-lying-around-its-edge-cases-is-less-broadly). If the caliph were to _directly_ contradict himself in simple, unambiguous language—to go from "Oceania is not at war with Eastasia" to "Oceania is at war with Eastasia" without any acknowledgement that anything had changed—_then_ too many people might notice that those two sentences are the same except that one has the word _not_ in it. What's a caliph to do, if he wants to declare war on Eastasia without acknowledging or taking responsibility for the decision to do so?
359 The solution is simple: just—use more words! Then if someone tries to argue that you've _effectively_ contradicted yourself, accuse them of being uncharitable and failing to model the Other. You can't lose! Anything can be consistent with anything if you apply a sufficiently charitable reading; whether Oceania is at war with Eastasia depends on how you choose to draw the category boundaries of "at war."
361 Thus, O'Brien should envy Yudkowsky: burning the photograph turns out to be unnecessary! ["Changing Emotions"](https://www.lesswrong.com/posts/QZs4vkC7cbyjL9XA9/changing-emotions) is _still up_ and not retracted, but that didn't stop the Yudkowsky of 2016 from pivoting to ["at least 20% of the ones with penises are actually women"](https://www.facebook.com/yudkowsky/posts/10154078468809228) when that became a politically favorable thing to say. I claim that these posts _effectively_ contradict each other. The former explains why men who fantasize about being women are _not only_ out of luck given forseeable technology, but _also_ that their desires may not even be coherent (!), whereas the latter claims that men who wish they were women may, in fact, _already_ be women in some unspecified psychological sense.
363 _Technically_, these don't _strictly_ contradict each other: I can't point to a sentence from each that are the same except one includes the word _not_. (And even if there were such sentences, I wouldn't be able to prove that the other words were being used in the same sense in both sentences.) One _could_ try to argue that "Changing Emotions" is addressing cis men with a weird sex-change fantasy, whereas the "ones with penises are actually women" claim was about trans women, which are a different thing.
365 _Realistically_ ... no. These two posts _can't_ both be right. In itself, this isn't a problem: people change their minds sometimes, which is great! But when people _actually_ change their minds (as opposed to merely changing what they say in public for political reasons), you expect them to be able to _acknowledge_ the change, and hopefully explain what new evidence or reasoning brought them around. If they can't even _acknowledge the change_, that's pretty Orwellian, like O'Brien trying to claim that the photograph is of different men who just coincidentally happen to look like Jones, Aaronson, and Rutherford.
367 And if a little bit of Orwellianism on specific, narrow, highly-charged topics might be forgiven—because everyone else in your Society is doing it, and you would be punished for not playing along, an [inadequate equilibrium](https://equilibriabook.com/) that no one actor has the power to defy—might we not expect the father of the "rationalists" to stand his ground on the core theses of his ideology, like whether telling the truth is good?
369 I guess not! ["Doublethink (Choosing to be Biased)"](https://www.lesswrong.com/posts/Hs3ymqypvhgFMkgLb/doublethink-choosing-to-be-biased) is _still up_ and not retracted, but that didn't stop Yudkowsky from [endorsing everything Xu said](https://twitter.com/ESYudkowsky/status/1436025983522381827) about "whether some categories facilitate inferences that _do_, on the whole, cause more harm than benefit, and if so, whether it is 'rational' to rule that such inferences should be avoided when possible" being different cruxes than "whether 'rational' thinking is 'worth it'".
371 I don't doubt Yudkowsky could come up with some clever casuistry why, _technically_, the text he wrote in 2007 and the text he endorsed in 2021 don't contradict each other. But _realistically_ ... again, no.
373 [TODO: elaborate on how 2007!Yudkowsky and 2021!Xu are saying the opposite things if you just take a plain-language reading and consider, not whether individual sentences can be interpreted as "true", but what kind of _optimization_ the text is doing to the behavior of receptive readers]
375 I don't, actually, expect people to spontaneously blurt out everything they believe to be true, that Stalin would find offensive. "No comment" would be fine. Even selective argumentation that's _clearly labeled as such_ would be fine. (There's no shame in being an honest specialist who says, "I've mostly thought about these issues though the lens of ideology _X_, and therefore can't claim to be comprehensive; if you want other perspectives, you'll have to read other authors and think it through for yourself.")
377 What's _not_ fine is selective argumentation while claiming "confidence in [your] own ability to independently invent everything important that would be on the other side of the filter and check it [yourself] before speaking" when you _very obviously have done no such thing_.
381 In September 2021, I took a trip to the east coast to see my sister and new baby niece in Cambridge and some friends in New York, and otherwise to work on blog posts from my hotel room. (You've seen one American city, you've seen them all, I reasoned, _modulo_ a different ratio of Dunkin' Donuts to Starbucks.)
383 The thought occured to me that while I was on vacation anyway, it might be nice to see an escort (which I had only done once before, in March 2016). I browsed around the "Boston" section on _eros.com_. Under the "ebony" subsection[^ebony], I found a profile that I liked on the basis of photographs and a professional-seeming website. I'll call her "Crystal" (which was not the name she used, and the name she used was also surely not her real name). The "consideration" page on the website listed three hours at $1500, and four hours for $2000. ("All rates are for my time and companionship only.") I filled out the booking form requesting a three-hour engagement.
385 [^ebony]: I don't know why they say "ebony" instead of "black". Porn sites do this, too. Seems problematic to have an ethnic term that only gets used in sexualized contexts?
387 "Crystal" didn't get back to me within several days, and I resigned myself to the implied rejection. (I already felt morally ambiguous about filling out the form; I certainly wasn't about to _shop around_.) But then on 4:37 _p.m._ on the day before my flight left, she got back to me asking if I was still interested in connecting, explaining that there had been a problem with her spam filter (Subject: "Hello! My apologies For The Late Reply"). I rushed to the bank just before it closed to withdraw $2000 cash that would have been harder to get at on my trip, between ATM withdrawal limits and the lack of Wells Fargo branches in Massachusetts.
389 She suggested switching to texts to get around the spam issue, and I texted her a photo of my ID and a link to my LinkedIn profile to confirm my identity (or gentlemanliness). She asked, "Would you like to go on a dinner date, four hours, 2,000."[^no-dollar-sign] (Already feeling morally ambiguous, I certainly wasn't going to complain about getting _upsold_.)[^upsold] I eagerly agreed, and suggested an Indian restaurant a half-mile walk from my hotel.
391 [^no-dollar-sign]: I appreciated the absence of a dollar sign in front of the figure. Feels less crass.
393 [^upsold]: But I had mentioned the $2000 bank withdrawal in my email, so it made sense that that figure was more salient to her than the fact that my form submission from weeks earlier had said three hours.
395 I didn't get any writing done the day of our date, although I had at least [gotten in a couple hundred words the previous day](/source?p=Ultimately_Untrue_Thought.git;a=commitdiff;h=470bbc84ba) on my draft review of Charles Murray's _Facing Reality: Two Truths About Race in America_. I texted my sister and my mother (who was also in town to see my sister) that I had other plans that night, requested housekeeping service for my hotel room (which had been suspended by default due to COVID), and mostly just walked around Cambridge for the late morning and afternoon. I bought a shirt with a collar at Target.
397 My mother called to inquire more about my plans, which I preferred to be at least somewhat vague about. I said that the safety concerns about meeting someone from the internet were mostly for women. She asked that I at least tell my sister where I was going, to which I agreed. I texted my sister: "Mom thinks there's a safety rationale for telling someone my plans, which I think is retarded, but specifically, it's a casual dinner date at"—I gave the name and address of the restaurant—"with a woman I met online (no reply expected; there's nothing useful for you to do with this information and this message is just to appease Mom)".
399 "Crystal" was late (date was originally scheduled for 4 _p.m._, then she texted in the early afternoon to move it to 4:30, then finally texted "I'm here"/"Just valeting my car" at 5:04), and then it turned out that she was expecting to meet me at the hotel while I was waiting at the restaurant, but we met in the middle and walked back to the restaurant.
401 It was nice—an opportunity to talk to someone who I otherwise wouldn't get to meet. Messaging someone like her on OKCupid would have felt fake, and cold-approaching someone like her in public was unthinkable. It felt more honest to just pay for her time.
403 What do I mean by "someone like her"? Definitely not race _per se_. Rather ... non-nerds?—normies. I know how to talk to _the kinds of women I meet in "rationalist"/EA circles_, and even (very rarely) ask them on a date.[^romantic-poem] That doesn't feel fake, because they're just peers who happen to be female. (I may have renounced [the ideological psychological sex difference denialism of my youth](/2021/May/sexual-dimorphism-in-the-sequences-in-relation-to-my-gender-problems/#antisexism), but I'm not _sexist_.)
405 What I don't know how to do without the moral [indulgence](https://en.wikipedia.org/wiki/Indulgence) of money changing hands is to ask out a beautiful woman _because she's a beautiful woman_. I won't say it's morally wrong, exactly; it's just not how I was raised. (I mean, I wasn't raised to hire escorts, either, but somehow the transactionality of it puts it outside some of the ethical constraints of ordinary courtship.)
407 [^romantic-poem]: Though the meter is occasionally a little bit bizarre, I'm very proud of [the poem I wrote in 2016 depicting a woman I was interested in eradicating malaria by wiping out all the mosquitos using CRISPR gene drive](/ancillary/megan-and-the-anopheles-gambiae/), although our one date didn't amount to anything. She later married Scott Alexander.
409 Dinner was nice. We talked. She had a dayjob in real estate. There were some conversational topics I wanted to steer away from until after dinner. When we got to my hotel room, I felt freer to speak. I explained autogynephilia to her, and (briefly) the Category War with my robot cult.
411 I didn't tell her about the book review I was writing.
413 We got undressed. I put on a condom, but I didn't let (have?) her touch my penis. That would have seemed unethical to me. I just wanted to admire her naked body, to touch, to cuddle. My own pleasure I could—handle myself. I kept telling her how beautiful she was—and that I was jealous. That I wanted to know what it would be like, if [by some miracle of technology or magic](/2021/May/sexual-dimorphism-in-the-sequences-in-relation-to-my-gender-problems/#secret-fantasy-frame-stories), I could look like her, be shaped like her. To have that face and that voice and those breasts (my gynecomastia from [that HRT experiment four years before](/2017/Sep/hormones-day-156-developments-doubts-and-pulling-the-plug-or-putting-the-cis-in-decision/) simply didn't compare) and that—waist-to-hip ratio. (She said I could have her breasts; they were heavy.)
415 I'm not actually ultra-confident that what I call "ethics" didn't make things weirder for her. She had said that she had been treating her night job was an alternative to conventional dating, that she wasn't seeing anyone nonprofessionally. It's possible that to a woman in that position, a normal man who wants to use a woman in the normal way is actually preferable to the kind of creep I am?—in contrast to my hopes that satisfying the kind of creep I am would be gratifyingly easy job for the money.
417 (At any rate, I think it would have been _more_ creepy if I tried to convince her that I was "actually" a woman in some unspecified metaphysical sense.)
419 I wasn't coming. She said that for $2000, I definitely deserved to get off. That didn't matter to me. I ended up finishing that after she left.
421 Before she left, as she was using the bathroom and washing up, I tried to say something about how I wished I could have a photo to remember her by, but I knew that it would be impolite to ask. (I had done my reading. "Do not ask for additional pictures, selfies or services they have not already agreed upon.") No photos, she said.
423 I wanted to clarify that I wasn't _asking_, I was making a meta-comment about how I _wasn't_ asking—and then I realized that normie culture must not support that level of indirection. To her, there was no semantic difference between apophasis and actually saying the thing, even if my "rationalist" friends would have understood what I meant. Considering my frustration at Yudkowsky's not-technically-lying games, I wondered if the normie way wasn't superior.
427 The New York leg of my trip went fine. I visited Sarah. I met another _Less Wrong_-er whose comments I had liked. I made $60 babysitting Zvi Mowshowitz's kids.
429 I visited Ben and met his new girlfiend. Jessica wasn't around. We hadn't talked much since the "Lenore" disaster nine months before. He said that my behavior then _had_ been a betrayal, but a finite one. It happens. (Not "it happens" meaning that it's _okay_, but that it _happens_; moral agents keep accounts; [you forgive, but don't forget](/2020/Dec/liability/).)
431 I got to meet my neoreactionary Twitter mutual, who held the distinction of having been banned from the _Slate Star Codex_ comment section ["for reasons of total personal caprice"](https://archive.md/sRfBj#selection-1633.27-1633.64). I wore my _Quillette_ T-shirt. He offered to buy me a drink. I said I didn't drink, but he insisted that getting drunk was the ritual by which men established trust. I couldn't argue with that, and ended up having a glass and a half of wine while we talked for a couple hours.
433 So much of my intellectual life for the past five years had been shaped by the fight to keep mere heresies on the shared map, that it was a nice change to talk to an out-and-out _apostate_, with whom none of none of my ingrained defensive motions were necessary. (I just want to restore the moral spirit of 2008 liberalism but with better epistemology; he wants to bring back [_coverture_](https://en.wikipedia.org/wiki/Coverture) on the grounds that women of the Eurasian subspecies of humanity haven't exercised mate choice in 10,000 years and aren't being helped by starting now.) There was one moment when I referred to the rationalists as my guys, and instinctively disclaimed, "and of course, we're mostly guys." He pointed out that I didn't need to tell _him_ that.
437 In October 2021, Jessica Taylor [published a post about her experiences at MIRI](https://www.lesswrong.com/posts/MnFqyPLqbiKL8nSR7/my-experience-at-and-around-miri-and-cfar-inspired-by-zoe), making analogies between sketchy social pressures she had experienced in the core rationalist community (around short AI timelines, secrecy, deference to community leaders, _&c._) and those reported in [Zoe Cramer's recent account of her time at Leverage Research](https://medium.com/@zoecurzi/my-experience-with-leverage-research-17e96a8e540b).
439 Scott Alexander posted [a comment claiming to add important context](https://www.lesswrong.com/posts/MnFqyPLqbiKL8nSR7/my-experience-at-and-around-miri-and-cfar-inspired-by-zoe?commentId=4j2GS4yWu6stGvZWs), essentially blaming Jessica's problems on her association with Michael Vassar, to the point of describing her psychotic episode as a "Vassar-related phenomenon" (!). Alexander accused Vassar of trying "'jailbreak'" people from normal social reality, which "involve[d] making them paranoid about MIRI/CFAR and convincing them to take lots of drugs". Yudkowsky posted [a comment that uncritically validated Scott's reliability as a narrator](https://www.lesswrong.com/posts/MnFqyPLqbiKL8nSR7/my-experience-at-and-around-miri-and-cfar-inspired-by-zoe?commentId=x5ajGhggHky9Moyr8).
441 To me, this looked like raw factional conflict: Jessica had some negative-valence things to say about the Caliphate, so Caliphate leaders moved in to discredit her by association. Quite effectively, as it turned out: the karma score on Jessica's post dropped by more than half, while Alexander's comment got voted up to more than 380 karma. (The fact that Scott said ["it's fair for the community to try to defend itself"](https://www.lesswrong.com/posts/MnFqyPLqbiKL8nSR7/my-experience-at-and-around-miri-and-cfar-inspired-by-zoe?commentId=qsEMmdo6DKscvBvDr) in ensuing back-and-forth suggests that he also saw the conversation as an adversarial one, even if he thought Jessica shot first.)
443 I explained [why I thought Scott was being unfair](https://www.lesswrong.com/posts/MnFqyPLqbiKL8nSR7/my-experience-at-and-around-miri-and-cfar-inspired-by-zoe?commentId=GzqsWxEp8uLcZinTy) (and [offered textual evidence](https://www.lesswrong.com/posts/MnFqyPLqbiKL8nSR7/my-experience-at-and-around-miri-and-cfar-inspired-by-zoe?commentId=yKo2uuCcwJxbwwyBw) against the silly claim that Michael was _trying_ to drive Jessica crazy).
445 Scott [disagreed](https://www.lesswrong.com/posts/MnFqyPLqbiKL8nSR7/my-experience-at-and-around-miri-and-cfar-inspired-by-zoe?commentId=XpEpzvHPLkCH7W7jS) that joining the "Vassarites"[^vassarite-scare-quotes] wasn't harmful to me. He revealed that during my March 2019 problems, he had emailed my posse:
447 > accusing them of making your situation worse and asking them to maybe lay off you until you were maybe feeling slightly better, and obviously they just responded with their "it's correct to be freaking about learning your entire society is corrupt and gaslighting" shtick.
449 [^vassarite-scare-quotes]: Scare quotes because "Vassarite" seems likely to be Alexander's coinage; we didn't call ourselves that.
451 But I will _absolutely_ bite the bullet on it being correct to freak out about learning your entire Society is corrupt and gaslighting (as I explained to Scott in an asynchronous 22–27 October 2021 conversation on Discord).
453 Imagine living in the Society of Alexander's ["Kolmogorov Complicity and the Parable of Lightning"](https://slatestarcodex.com/2017/10/23/kolmogorov-complicity-and-the-parable-of-lightning/) (which I keep linking) in the brief period when the lightening taboo is being established, trying to make sense of everyone you know suddenly deciding, seemingly in lockstep, that thunder comes before lightning. (When you try to point out that this isn't true and no one believed it five years ago, they point out that it depends on what you mean by the word 'before'.)
455 Eventually, you would get used to it, but at first, I think this would be legitimately pretty upsetting! If you were already an emotionally fragile person, it might even escalate to a psychiatric emergency through the specific mechanism "everyone I trust is inexplicably lying about lightning → stress → sleep deprivation → temporary psychosis". That is, it's not that Society being corrupt directly causes mental ilness—that would be silly—but confronting a corrupt Society is very stressful, and that can [snowball into](https://lorienpsych.com/2020/11/11/ontology-of-psychiatric-conditions-dynamic-systems/) things like lost sleep, and sleep is [really](https://www.jneurosci.org/content/34/27/9134.short) [biologically important](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6048360/).
457 This is a pretty bad situation to be in—to be faced with the question, "Am _I_ crazy, or is _everyone else_ crazy?" But one thing that would make it slightly less bad is if you had a few allies, or even just _an_ ally—someone to confirm that the obvious answer, "It's not you," is, in fact, obvious.
459 But in a world where [everyone who's anyone](https://thezvi.wordpress.com/2019/07/02/everybody-knows/) agrees that thunder comes before lightning—including all the savvy consequentialists who realize that being someone who's anyone is an instrumentally convergent strategy for acquiring influence—anyone who would be so imprudent to take your everyone-is-lying-about-lightning concerns seriously, would have to be someone with ... a nonstandard relationship to social reality. Someone meta-savvy to the process of people wanting to be someone who's anyone. Someone who, honestly, is probably some kind of _major asshole_. Someone like—Michael Vassar!
461 From the perspective of an outside observer playing a Kolmogorov-complicity strategy, your plight might look like "innocent person suffering from mental illness in need of treatment/management", and your ally as "bad influence who is egging the innocent person on for their own unknown but probably nefarious reasons". If that outside observer chooses to draw the category boundaries of "mental illness" appropriately, that story might even be true. So why not quit making such a fuss, and accept treatment? Why fight, if fighting comes at a personal cost? Why not submit?
463 I had my answer. But I wasn't sure that Scott would understand.
465 To assess whether joining the "Vassarites" had been harmful to me, one would need to answer: as compared to what? In the counterfactual where Michael vanished from the world in 2016, I think I would have been just as upset about the same things for the same reasons, but with fewer allies and fewer ideas to make sense of what was going on in my social environment.
467 Additionally, it was really obnoxious when people had tried to use my association with Michael to try to discredit the content of what I was saying—interpreting me as Michael's pawn. Gwen, one of the "Zizians", in a blog post about her grievances against CfAR, has [a section on "Attempting to erase the agency of everyone who agrees with our position"](https://everythingtosaveit.how/case-study-cfar/#attempting-to-erase-the-agency-of-everyone-who-agrees-with-our-position), complaining about how people try to cast her and Somni and Emma as Ziz's minions, rather than acknowledging that they're separate people with their own ideas who had good reasons to work together. I empathized a lot with this. My thing, and separately Ben Hoffman's [thing about Effective Altruism](http://benjaminrosshoffman.com/drowning-children-rare/), and separately Jessica's thing in the OP, didn't really have a whole lot to do with each other, except as symptoms of "the so-called 'rationalist' community is not doing what it says on the tin" (which itself wasn't a very specific diagnosis). But insofar as our separate problems did have a hypothesized common root cause, it made sense for us to talk to each other and to Michael about them.
469 Was Michael using me, at various times? I mean, probably. But just as much, _I was using him_. Particularly with the November 2018–April 2019 thing (where I and the "Vassarite" posse kept repeatedly pestering Scott and Eliezer to clarify that categories aren't arbitrary): that was the "Vassarites" doing an _enormous_ favor for _me_ and _my_ agenda. (If Michael and crew hadn't had my back, I wouldn't have been anti-social enough to keep escalating.) And here Scott was trying to get away with claiming that _they_ were making my situation worse? That's _absurd_. Had he no shame?
471 I _did_, I admitted, have some specific, nuanced concerns—especially since the December 2020 psychiatric disaster, with some nagging doubts beforehand—about ways in which being an inner-circle "Vassarite" might be bad for someone, but at the moment, I was focused on rebutting Scott's story, which was _silly_. A defense lawyer has an easier job than a rationalist—if the prosecution makes a terrible case, you can just destroy it, without it being your job to worry about whether your client is separately guilty of vaguely similar crimes that the incompetent prosecution can't prove.
473 When Scott expressed concern about the group-yelling behavior that [Ziz had described in a blog comment](https://sinceriously.fyi/punching-evil/#comment-2345) ("They spent 8 hours shouting at me, gaslighting me") and [Yudkowsky had described on Twitter](https://twitter.com/ESYudkowsky/status/1356494768960798720) ("When MichaelV and co. try to run a 'multiple people yelling at you' operation on me, I experience that as 'lol, look at all that pressure' instead _feeling pressured_"), I clarified that that thing was very different from what it was like to actually be friends with them. The everyone-yelling operation seemed like a new innovation (that I didn't like) that they wield as a psychological weapon only against people who they think are operating in bad faith? In the present conversation with Scott, I had been focusing on rebutting the claim that my February–April 2017 (major) and March 2019 (minor) psych problems were caused by the "Vassarites", because with regard to those _specific_ incidents, the charge was absurd and false. But, well ... my January 2021 (minor) psych problems actually _were_ the result of being on the receiving end of the everyone-yelling thing. I briefly described the December 2020 "Lenore" disaster, and in particular the part where Michael/Jessica/Jack yelled at me.
475 Scott said that based on my and others' testimony, he was updating away from Vassar being as involved in psychotic breaks than he thought, but towards thinking Vassar was worse in other ways than he thought. He felt sorry for my bad December 2020/January 2021 experience—so much that he could feel it through the triumphant vindication at getting conifrmation that the Vassarites were behaving badly in ways he couldn't previously prove.
477 Great, I said, I was happy to provide information to help hold people (including Michael as a particular instance of "people") accountable for the specific bad things that they're actually guilty of, rather than scapegoated as a Bad Man with mysterious witch powers.
479 Scott supposed that he should also be investigating "Lenore", who he sarcastically remarked was liable to be yet another case of someone having a psychotic break just as she was getting close to the Vassarites, but that somehow there's no plausible connection between those two things.
481 I pointed out that that's exactly what one would expect if the Vassar/breakdown correlation was mostly a selection effect rather than causal—that is, if the causal graph was the fork "prone-to-psychosis ← underlying-bipolar-ish-condition → gets-along-with-Michael".
483 I had also had a sleep-deprivation-induced-psychotic-break-with-hospitalization in February 2013, and shortly thereafter, I remember Anna remarking that I was sounding a lot like Michael. But I hadn't been talking to Michael at all beforehand! (My previous email conversation with him had been in 2010.) So what could Anna's brain have been picking up on, when she said that? My guess: there was some underlying dimension of psychological variation (psychoticism? bipolar?—you tell me; this is supposed to be Scott's professional specialty) where Michael and I were already weird/crazy in similar ways, and sufficiently bad stressors could push me further along that dimension (enough for Anna to notice). Was Scott also going to blame Yudkowsky for making people [autistic](https://twitter.com/ESYudkowsky/status/1633396201427984384)?
485 Concerning the lightning parable, Scott said that from his perspective, the point of "Kolmogorov Complicity" was that, yes, people can be crazy, but that we have to live in Society without spending all our time freaking out about it. If, back in the days of my ideological anti-sexism, the first ten Yudkowsky posts I had read had said that men and women are psychologically different for biological reasons and that anyone who denies this is a mind-killed idiot—which Scott assumed Yudkowsky did think—he could imagine me being turned off. It was probably good for me and the world that that wasn't my first ten experiences of the rationalist community.
487 I agreed that this was a real concern. (I had been so enamored with Yudkowsky's philosophy-of-science writing that there was no chance of _me_ bouncing on account of the sexism that I perceived, but I wasn't the marginal case.) There are definitely good reasons to tread carefully when trying to add sensitive-in-our-culture content to Society's shared map. But I didn't think treading carefully should take precedence over _getting the goddamned right answer_.
489 As an example of what I thought treading carefully but getting the goddamned right answer looked like, I was really proud of [my April 2020 review of Charles Murray's _Human Diversity_](/2020/Apr/book-review-human-diversity/). I definitely wasn't saying, Emil Kirkegaard-style, "the black/white IQ gap is genetic, anyone who denies this is a mind-killed idiot." Rather, _first_ I reviewed the Science in the book, and _then_ I talked about the politics surrounding Murray's reputation and the technical reasons for believing that the gap is real and partly genetic, and _then_ I went meta on the problem and explained why it makes sense that political forces make this hard to talk about. I thought this was how one goes about mapping the territory without being a moral monster with respect to one's pre-Dark Enlightenment morality. (And [Emil was satisfied, too](https://twitter.com/KirkegaardEmil/status/1425334398484983813).)
493 At the end of the September 2021 Twitter altercation, I [said that I was upgrading my "mute" of @ESYudkowsky to a "block"](https://twitter.com/zackmdavis/status/1435468183268331525). Better to just leave, rather than continue to hang around in his mentions trying (consciously [or otherwise](https://www.lesswrong.com/posts/sXHQ9R5tahiaXEZhR/algorithmic-intent-a-hansonian-generalized-anti-zombie)) to pick fights, like a crazy ex-girlfriend. (["I have no underlying issues to address; I'm certifiably cute, and adorably obsessed"](https://www.youtube.com/watch?v=UMHz6FiRzS8) ...)
495 I did end up impulsively writing one more comment on one of his Facebook posts (with an aside at the top about whether that was OK), and Yudkowsky [said that Twitter looked worse for me than Facebook](/images/yudkowsky-twitter_is_worse_for_you.png)—the implication being that I _did_ still have commenting privileges as far as he was concerned. Good. I'm proud to be a crazy ex-girlfriend who knows she's crazy and _voluntarily_ deletes your number from her phone, rather than the crazy ex-girlfriend you need to block.
497 I still had more things to say—a reply to the February 2021 post on pronoun reform, and the present memoir telling this Whole Dumb Story—but those could be written and published unilaterally. Given that we clearly weren't going to get to clarity and resolution, I didn't want to bid for any more of my ex-hero's attention and waste more of his time (valuable time, _limited_ time); I still owed him for creating me.
499 Leaving a personality cult is hard. As I struggled to write, I noticed that I was wasting a lot of cycles worrying about what he'd think of me, rather than saying the things I needed to say. I knew it was pathetic that my religion was so bottlenecked on _one guy_—particularly since the holy texts themselves (written by that one guy) [explicitly said not to do that](https://www.lesswrong.com/posts/t6Fe2PsEwb3HhcBEr/the-litany-against-gurus)—but unwinding those psychological patterns was still a challenge.
501 An illustration of the psychological dynamics at play: on an August 2021 EA Forum post about demandingness objections to longtermism, Yudkowsky [commented that](https://forum.effectivealtruism.org/posts/fStCX6RXmgxkTBe73/towards-a-weaker-longtermism?commentId=Kga3KGx6WAhkNM3qY) he was "broadly fine with people devoting 50%, 25% or 75% of themselves to longtermism [...] as opposed to tearing themselves apart with guilt and ending up doing nothing much, which seem[ed] to be the main alternative."
503 I found the comment reassuring regarding the extent or lack thereof of my own contributions to the great common task—and that's the problem: I found the _comment_ reassuring, not the _argument_. It would make sense to be reassured by the claim (if true) that human psychology is such that I don't realistically have the option of devoting more than 25% of myself to the great common task. It does _not_ make sense to be reassured that _Eliezer Yudkowsky said he's broadly fine with it_. That's just being a personality-cultist.
505 In January 2022, in an attempt to deal with my personality-cultist writing block, I sent him one last email asking if he particularly _cared_ if I published a couple blog posts that said some negative things about him. If he actually _cared_ about potential reputational damage to him from my writing things that I thought I had a legitimate interest in writing about, I would be _willing_ to let him pre-read the drafts before publishing and give him the chance to object to anything he thought was unfair ... but I'd rather agree that that wasn't necessary. I explained the privacy norms that I intended to follow—that I could explain _my_ actions, but had to Glomarize about the content of any private conversations that may or may not have occurred.
507 It had taken me a while (with apologies for my atrocious [sample efficiency](https://ai.stackexchange.com/a/5247)), but I was finally ready to give up on him; I thought the efficient outcome was that I should just tell my Whole Dumb Story on my blog and never bother him again. Since he probably _didn't_ particularly care (because it's not AGI alignment and therefore unimportant) and it would be psychologically easier on me if I knew he diidn't hold it against me, could I please have his advance blessing to just write and publish what I was thinking so I can get it all out of my system and move on with my life?
509 If it helped—as far as _I_ could tell, I was only doing what _he_ taught me to do in 2007–2009: [carve reality at the joints](https://www.lesswrong.com/posts/esRZaPXSHgWzyB2NL/where-to-draw-the-boundaries), [speak the truth even if your voice trembles](https://www.lesswrong.com/posts/pZSpbxPrftSndTdSf/honesty-beyond-internal-truth), and [make an extraordinary effort](https://www.lesswrong.com/posts/GuEsfTpSDSbXFiseH/make-an-extraordinary-effort) when you've got [Something to Protect](https://www.lesswrong.com/posts/SGR4GxFK7KmW7ckCB/something-to-protect) (Subject: "blessing to speak freely, and privacy norms?").
511 I can't say whether he replied (because if he did, that would be covered by the privacy norm), but I think sending the email helped me. Although maybe I was wrong to ask if he wouldn't hold it against me. If you read the text of this memoir, I'm clearly holding things against _him_. If he's not my caliph anymore (with the asymmetrical duties between ruler and subject, the higher to protect and the lower to serve), and I'm entitled to my feelings, isn't he entitled to his?
513 In February 2022, I finally managed to finish a draft of ["Challenges to Yudkowsky's Pronoun Reform Proposal"](/2022/Mar/challenges-to-yudkowskys-pronoun-reform-proposal/) (A year after the post it replies to! I did other things that year, probably.) It's long (12,000 words), because I wanted to be thorough and cover all the angles. (To paraphrase Ralph Waldo Emerson, when you strike at Eliezer Yudkowsky, _you must kill him._)
515 If I had to compress it by a factor of 200 (down to 60 words), I'd say my main point was that, given a conflict over pronoun conventions, there's no "right answer", but we can at least be objective in _describing what the conflict is about_, and Yudkowsky wasn't doing that; his "simplest and best proposal" favored the interests of some parties to the dispute (as was seemingly inevitable), _without admitting he was doing so_ (which was not inevitable).[^describing-the-conflict]
517 [^describing-the-conflict]: I had been making this point for four years. [As I wrote in February 2018's "The Categories Were Made for Man to Make Predictions"](/2018/Feb/the-categories-were-made-for-man-to-make-predictions/#describing-the-conflict), "If different political factions are engaged in conflict over how to define the extension of some common word [...] rationalists may not be able to say that one side is simply right and the other is simply wrong, but we can at least strive for objectivity in _describing the conflict_."
519 In addition to prosecuting the object level (about pronouns) and the meta level (about acknowleding the conflict) for 12,000 words, I had also written _another_ several thousand words at the meta-meta level, about the political context of the argument and Yudkowsky's comments about what is "sometimes personally prudent and not community-harmful", but I wasn't sure whether to include it in the post itself, or post it as a separate comment on the _Less Wrong_ linkpost mirror, or save it for the memoir. I was worried about it being too "aggressive", attacking Yudkowsky too much, disregarding our usual norms about only attacking arguments and not people. I wasn't sure how to be aggressive and explain _why_ I wanted to disregard the usual norms in this case (why it was _right_ to disregard the usual norms in this case) without the Whole Dumb Story of the previous six years leaking in (which would take even longer to write).
521 I asked secret posse member for political advice. I thought my argumens were very strong, but that the object-level argument about pronoun conventions just wasn't very interesting; what I _actually_ wanted people to see was the thing where the Big Yud of the current year _just can't stop lying for political convenience_. How could I possibly pull that off in a way that the median _Less Wrong_-er would hear? Was it a good idea to "go for the throat" with the "I'm better off because I don't trust Eliezer Yudkowsky to tell the truth in this domain" line?
523 Secret posse member said the post was super long and boring. ("Yes. I'm bored, too," I replied.) They said that I was optimizing for my having said the thing, rather than for the reader being able to hear it. In the post, I had complained that you can't have it both ways: either pronouns convey sex-category information (in which case, people who want to use natal-sex categories have an interest in defending their right to misgender), or they don't (in which case, there would be no reason for trans people to care about what pronouns people use for them). But by burying the thing I actually wanted people to see in thousands of words of boring argumentation, I was evading the fact that _I_ couldn't have it both ways: either I was calling out Yudkowsky as betraying his principles and being dishonest, or I wasn't.
525 "[I]f you want to say the thing, say it," concluded secret posse member. "I don't know what you're afraid of."
527 I was afraid of taking irrevocable war actions against the person who taught me everything I know. (And his apparent conviction that the world was ending _soon_, made it worse. Wouldn't it feel petty, if the last thing you ever said to your grandfather was calling him a liar in front of the whole family, even if he had in fact lied?)
529 I wanted to believe that if I wrote all the words dotting every possible _i_ and crossing every possible _t_ at all three levels of meta, then that would make it [a description and not an attack](http://benjaminrosshoffman.com/can-crimes-be-discussed-literally/)—that I could have it both ways if I explained the lower level of organization beneath the high-level abstractions of "betraying his principles and being dishonest." If that didn't work because [I only had five words](https://www.lesswrong.com/posts/4ZvJab25tDebB8FGE/you-have-about-five-words), then—I didn't know what I'd do. I'd think about it.
531 After a month of dawdling, I eventually decided to pull the trigger on publishing "Challenges", without the extended political coda.[^coda] The post was a little bit mean to Yudkowsky, but not so mean that I was scared of the social consequences of pulling the trigger. (Yudkowsky had been mean to Christiano and Richard Ngo and Rohin Shah in [the recent MIRI dialogues](https://www.lesswrong.com/s/n945eovrA3oDueqtq); I didn't think this was worse than that.)
533 [^coda]: The text from the draft coda would later be incorporated into the present memoir.
535 I cut the words "in this domain" from the go-for-the-throat concluding sentence that I had been worried about. "I'm better off because I don't trust Eliezer Yudkowsky to tell the truth," full stop.
537 The post was a _critical success_ by my accounting, due to eliciting a [a highly-upvoted (110 karma at press time) comment by _Less Wrong_ administrator Oliver Habryka](https://www.lesswrong.com/posts/juZ8ugdNqMrbX7x2J/challenges-to-yudkowsky-s-pronoun-reform-proposal?commentId=he8dztSuBBuxNRMSY) on the _Less Wrong_ mirror. Habryka wrote:
539 > [...] basically everything in this post strikes me as "obviously true" and I had a very similar reaction to what the OP says now, when I first encountered the Eliezer Facebook post that this post is responding to.
541 > And I do think that response mattered for my relationship to the rationality community. I did really feel like at the time that Eliezer was trying to make my map of the world worse, and it shifted my epistemic risk assessment of being part of the community from "I feel pretty confident in trusting my community leadership to maintain epistemic coherence in the presence of adversarial epistemic forces" to "well, I sure have to at least do a lot of straussian reading if I want to understand what people actually believe, and should expect that depending on the circumstances community leaders might make up sophisticated stories for why pretty obviously true things are false in order to not have to deal with complicated political issues".
543 > I do think that was the right update to make, and was overdetermined for many different reasons, though it still deeply saddens me.
545 Brutal! Recall that Yudkowsky's justification for his behavior had been that "it is sometimes personally prudent and _not community-harmful_ to post your agreement with Stalin" (emphasis mine), and here we had the administrator of Yudkowsky's _own website_ saying that he's deeply saddened that he now expects Yudkowsky to _make up sophisticated stories for why pretty obviously true things are false_ (!!).
547 Is that ... _not_ evidence of harm to the community? If that's not community-harmful in Yudkowsky's view, then what would be example of something that _would_ be? _Reply, motherfucker!_
549 ... or rather, "Reply, motherfucker", is what I fantasized about being able to say, if I hadn't already expressed an intention not to bother him anymore.
553 On 1 April 2022, Yudkowsky published ["MIRI Announces New 'Death With Dignity' Strategy"](https://www.lesswrong.com/posts/j9Q8bRmwCgXRYAgcJ/miri-announces-new-death-with-dignity-strategy), a cry of despair in the guise of an April Fool's Day post. MIRI didn't know how to align a superintelligence, no one else did either, but AI capabilities work was continuing apace. With no credible plan to avert almost-certain doom, the most we could do now was to strive to give the human race a more dignified death, as measured in log-odds of survival: an alignment effort that doubled the probability of a valuable future from 0.0001 to 0.0002 was worth one information-theoretic bit of dignity.
555 In a way, "Death With Dignity" isn't really an update. Yudkowsky had always refused to name a "win" probability, while maintaining that Friendly AI was ["impossible"](https://www.lesswrong.com/posts/nCvvhFBaayaXyuBiD/shut-up-and-do-the-impossible). Now, he says the probability is approximately zero.
557 Paul Christiano, who has a much more optimistic picture of humanity's chances, nevertheless said that he liked the "dignity" heuristic. I like it, too. It—takes some of the pressure off. I [made an analogy](https://www.lesswrong.com/posts/j9Q8bRmwCgXRYAgcJ/miri-announces-new-death-with-dignity-strategy?commentId=R59aLxyj3rvjBLbHg): your plane crashed in the ocean. To survive, you must swim to shore. You know that the shore is west, but you don't know how far. The optimist thinks the shore is just over the horizon; we only need to swim a few miles and we'll probably make it. The pessimist thinks the shore is a thousand miles away and we will surely die. But the optimist and pessimist can both agree on how far we've swum up to this point, and that the most dignified course of action is "Swim west as far as you can."
561 Perhaps for lack of any world-saving research to do, Yudkowsky started writing fiction again, largely in the form of Glowfic (a genre of collaborative storytelling pioneered by Alicorn) featuring the world of dath ilan (capitalization _sic_). Dath ilan had originally been introduced in a [2014 April Fool's Day post](https://yudkowsky.tumblr.com/post/81447230971/my-april-fools-day-confession), in which Yudkowsky "confessed" that the explanation for his seemingly implausible genius is that he's "actually" an ordinary person from a smarter, saner alternate version of Earth where the ideas he presented to this world as his own were common knowledge.
563 The bulk of the dath ilan Glowfic canon was an epic titled [_Planecrash_](https://www.glowfic.com/boards/215)[^planecrash-title] coauthored with Lintamande, in which Keltham, an unusually selfish teenage boy from dath ilan, apparently dies in a freak aviation accident, and [wakes up in the world of](https://en.wikipedia.org/wiki/Isekai) Golarion, setting of the _Dungeons-&-Dragons_–alike _Pathfinder_ role-playing game. A [couple](https://www.glowfic.com/posts/4508) of [other](https://glowfic.com/posts/6263) Glowfic stories with different coauthors further flesh out the setting of dath ilan, which inspired a new worldbuilding trope, the [_medianworld_](https://www.glowfic.com/replies/1619639#reply-1619639), a setting where the average person is like the author along important dimensions.[^medianworlds]
565 [^planecrash-title]: The title is a pun, referring to both the airplane crash leading to Keltham's death in dath ilan, and how his resurrection in Golarion collides dath ilan with [the "planes" of existence of the _Pathfinder_ universe](https://pathfinderwiki.com/wiki/Great_Beyond).
567 [^medianworlds]: You might think that the thought experiment of imagining what someone's medianworld is like would only be interesting for people who are "weird" in our own world, thinking that our world is a medianworld for people who are normal in our world. But [in high-dimensional spaces, _most_ of the probability-mass is concentrated in a "shell" some distance around the mode](/2021/May/sexual-dimorphism-in-the-sequences-in-relation-to-my-gender-problems/#typical-point), because even though the per-unit-hypervolume probability _density_ is greatest at the mode, there's vastly _more_ hypervolume in the hyperspace around it. The upshot is that typical people are atypical along _some_ dimensions, so normies can play the medianworld game, too.
569 (I asked Anna how Yudkowsky could stand the Glowfic people. She said she thought Eliezer could barely stand anyone. That makes sense, I said.)
571 Everyone in dath ilan receives rationality training from childhood,[^category-vindication] but knowledge and training deemed psychologically hazardous to the general population is safeguarded by an order of [Keepers of Highly Unpleasant Things it is Sometimes Necessary to Know](https://www.glowfic.com/replies/1612937#reply-1612937). AGI research takes place in a secret underground city; some unspecified form of social engineering steers the _hoi polloi_ away from thinking about the possibility of AI.
573 [^category-vindication]: On the topic of dath ilan's rationality training, I did appreciate [this passage about the cognitive function of categorization](https://www.glowfic.com/replies/1779051#reply-1779051):
575 > Dath ilani kids get told to not get fascinated with the fact that, in principle, 'bounded-agents' with finite memories and finite thinking speeds, have any considerations about mapping that depend on what they want. It doesn't mean that you get to draw in whatever you like on your map, because it's what you want. It doesn't make reality be what you want.
577 Vindication! (This showed that Yudkowsky _does_ understand what was at issue in the "... Not Man for the Categories" dispute, even if I can't be credited with winning the argument for political reasons.)
579 Something that annoyed me about the portrayal of dath ilan was their incredibly casual attitude towards hiding information for some alleged greater good, seemingly without considering that [there are benefits and not just costs to people knowing things](http://benjaminrosshoffman.com/humility-argument-honesty/).
581 You can, of course, make up a sensible [Watsonian](https://tvtropes.org/pmwiki/pmwiki.php/Main/WatsonianVersusDoylist) rationale for this. A world with much smarter people is more "volatile"; with more ways for criminals and terrorists to convert knowledge into danger, maybe you _need_ more censorship just to prevent Society from blowing itself up.
583 I'm more preoccupied by a [Doylistic](https://tvtropes.org/pmwiki/pmwiki.php/Main/WatsonianVersusDoylist) interpretation—that dath ilan's obsessive secret-Keeping reflects something deep about how the Yudkowsky of the current year relates to speech and information, in contrast to the Yudkowsky who wrote the Sequences. The Sequences had encouraged you—yes, _you_, the reader—to be as rational as possible. In contrast, the dath ilan mythos seems to portray advanced rationality as dangerous knowledge that people need to be protected from. ["The universe is not so dark a place that everyone needs to become a Keeper to ensure the species's survival,"](https://glowfic.com/replies/1861879#reply-1861879) we're told. "Just dark enough that some people ought to."
585 Someone at the 2021 Event Horizon Independence Day party had told me that I had been misinterpreting the "Speak the truth, even if your voice trembles" slogan from the Sequences. I had interpreted the slogan as suggesting the importance of speaking the truth _to other people_ (which I think is what "speaking" is usually about), but my interlocutor said it was about, for example, being able to speak the truth aloud in your own bedroom, to yourself. I think some textual evidence for my interpretation can be found in Daria's ending to ["A Fable of Science and Politics"](https://www.lesswrong.com/posts/6hfGNLf4Hg5DXqJCF/a-fable-of-science-and-politics), a multiple-parallel-endings story about an underground Society divided into factions over the color of the unseen sky, and one person's reaction when they find a passageway leading aboveground to a view of the sky:
587 > Daria, once Green, tried to breathe amid the ashes of her world. _I will not flinch_, Daria told herself, _I will not look away_. She had been Green all her life, and now she must be Blue. Her friends, her family, would turn from her. _Speak the truth, even if your voice trembles_, her father had told her; but her father was dead now, and her mother would never understand. Daria stared down the calm blue gaze of the sky, trying to accept it, and finally her breathing quietened. _I was wrong_, she said to herself mournfully; _it's not so complicated, after all_. She would find new friends, and perhaps her family would forgive her ... or, she wondered with a tinge of hope, rise to this same test, standing underneath this same sky? "The sky is blue," Daria said experimentally, and nothing dire happened to her; but she couldn't bring herself to smile. Daria the Blue exhaled sadly, and went back into the world, wondering what she would say.
589 Daria takes it as a given that she needs to be open about her new blue-sky belief, even though it's socially costly to herself and to her loved ones; the rationalist wisdom from her late father did _not_ say to go consult a priest or a Keeper to check whether telling everyone about the blue sky is a good idea.[^other-endings] I think this reflects the culture of the _Overcoming Bias_ in 2006 valuing the existence of a shared social reality that reflects actual reality: the conviction that it's both possible and desirable for people to rise to the same test, standing underneath the same sky.
591 [^other-endings]: Even Eddin's ending, which portrays Eddin as more concerned with consequences than honesty, has him "trying to think of a way to prevent this information from blowing up the world", rather than trying to think of a way to suppress the information, in contrast to how Charles, in his ending, _immediately_ comes up with the idea to block off the passageway leading to the aboveground. Daria and Eddin are clearly written as "rationalists"; the deceptive strategy only comes naturally to the non-rationalist Charles. (Although you could Watsonianly argue that Eddin is just thinking longer-term than Charles: blocking off _this_ passageway and never speaking a word of it to another soul, won't prevent someone from finding some other passage to the aboveground, eventually.)
593 In contrast, the culture of dath ilan does not seem to particularly value people _standing under the same sky_.
595 For example, we are told of an Ordinary Merrin Conspiracy centered around a famous medical technician with a psychological need to feel unimportant, of whom ["everybody in Civilization is coordinating to pretend around her"](https://www.glowfic.com/replies/1764946#reply-1764946) that her achievements are nothing special, which is deemed to be kindness to her. It's like a reverse [Emperor Norton](https://en.wikipedia.org/wiki/Emperor_Norton) situation. (Norton was ordinary, but everyone around him colluded to make him think he was special; Merrin is special, but everyone around her colludes to make her think she's ordinary.)
597 But _as_ a rationalist, I condemn the Ordinary Merrin Conspiracy as _morally wrong_, for the same [reasons I condemn the Emperor Norton Conspiracy](/2018/Feb/the-categories-were-made-for-man-to-make-predictions/#emperor-norton). As [it was taught to me on _Overcoming Bias_ back in the 'aughts](https://www.lesswrong.com/posts/HYWhKXRsMAyvRKRYz/you-can-face-reality): what's true is already so. Denying it won't make it better. Acknowledging it won't make it worse. And _because_ it is true, it is what is there to be interacted with. Anything untrue isn't there to be lived. People can stand what is true, _because they are already doing so_.
599 In ["For No Laid Course Prepare"](https://glowfic.com/posts/6263), the story about how Merrin came to the attention of dath ilan's bureau of Exception Handling, we see the thoughts of a Keeper, Rittaen, who talks to Merrin. We're told that the discipline of modeling people mechanistically rather than [through empathy](https://www.lesswrong.com/posts/NLMo5FZWFFq652MNe/sympathetic-minds) is restricted to Keepers to prevent the risk of ["turning into an exceptionally dangerous psychopath"](https://glowfic.com/replies/1862201#reply-1862201). Rittaen [uses his person-as-machine Sight](https://glowfic.com/replies/1862204#reply-1862204) to infer that Merrin was biologically predisposed to learn to be afraid of having too much status.
601 Notwithstanding that Rittaen can be Watsonianly assumed to have detailed neuroscience skills that the author Doylistically doesn't know how to write, I am entirely unimpressed by the assertion that this idea is somehow _dangerous_, a secret that only Keepers can bear, rather than something _Merrin herself should be clued into_. "It's not [Rittaen's] place to meddle just because he knows Merrin better than Merrin does," we're told.
603 In the same story, Merrin is dressed up as a member of a fictional alien species, the Sparashki, due to having been summoned to the hospital from a fan convention with no time to change outfits. An agent from Exception Handling [tells Merrin that the bureau's Fake Conspiracy section is running an operation to plant evidence that Sparashki are real](https://glowfic.com/replies/1860952#reply-1860952), and asks Merrin not to contradict this, and Merrin just ... goes along with it.
605 It's in-character for Merrin to go along with it, because she's a pushover. My question is, why is it okay that Exception Handling explicitly has a Fake Conspiracies section (!), any more than it would have been if FTX or Enron explicitly had a Fake Accounting department?
607 Is it okay because the idea of humanoid aliens walking around in real life is sufficiently implausible that no one is "really fooled"? But this seems to be contradicted by the text, in which [the narrator tells us that Exception Handling deliberately does things that wouldn't otherwise make sense in order to make it harder to understand what's actually going on in Civilization](https://glowfic.com/replies/1860955#reply-1860955) (presumably, in order to protect the secret underground AGI alignment project). Even if very few dath ilani take the "Sparashki are real" ruse literally, it's still portrayed as successfully _kicking up epistemic dust_. If it were really the case that no one was being fooled about anything, then Exception Handling wouldn't have a strategic reason to do it.
609 Is it okay because dath ilan are the [designated good guys](https://tvtropes.org/pmwiki/pmwiki.php/Main/DesignatedHero)? Well, [so was FTX](https://forum.effectivealtruism.org/posts/sdjcH7KAxgB328RAb/ftx-ea-fellowships).
611 As another notable example of dath ilan hiding information for the alleged greater good, in Golarion, Keltham discovers that he's a sexual sadist, and deduces that Civilization has deliberately prevented him from realizing this, because there aren't enough corresponding masochists to go around in dath ilan. Having concepts for "sadism" and "masochism" as variations in human psychology would make sadists like Keltham sad about the desirable sexual experiences they'll never get to have, so Civilization arranges for them to _not be exposed to knowledge that would make them sad, because it would make them sad_ (!!).
613 It did not escape my notice that when "rationalist" authorities _in real life_ considered public knowledge of some paraphilia to be an infohazard (ostensibly for the benefit of people with that paraphilia), I _didn't take it lying down_.
615 This parallel between dath ilan's sadism/masochism coverup and the autogynephilia coverup I had fought in real life, was something I was only intending to comment on in passing in the present memoir, rather than devoting any more detailed analysis to, but as I was having trouble focusing on my own writing in September 2022, I ended up posting some critical messages about dath ilan's censorship regime in the "Eliezerfic" Discord server for reader discussion of _Planecrash_, using the sadism/masochism coverup as my central example.
617 What happens, I asked, to the occasional dath ilani free speech activists, with their eloquent manifestos arguing that Civilization would be better off coordinating on maps that reflect the territory, rather than coordinating to be a Keeper-managed zoo? (They _had_ to exist: in a medianworld centered on Yudkowsky, there are going to be a few weirdos who are +2.5 standard deviations on "speak the truth, even if your voice trembles" and −2.5 standard deivations on love of clever plots; this seems less weird than negative utilitarians, who were [established to exist](https://www.glowfic.com/replies/1789623#reply-1789623).) I _assumed_ they get dealt with somehow in the end (exiled from most cities? ... involuntarily cryopreserved?), but there had to be an interesting story about someone who starts out whistleblowing small lies (which Exception Handling allows; they think it's cute, and it's "priced in" to the game they're playing), and then just keeps _escalating and escalating and escalating_ until Governance decides to unperson him.
619 Although Yudkowsky participated in the server, I had reasoned that my participation didn't violate my previous intent not to bother him anymore, because it was a publicly-linked Discord server with hundreds of members. Me commenting on the story for the benefit of the _other_ 499 people in the chat room wouldn't generate a notification _for him_, the way it would if I sent him an email or replied to him on Twitter.
621 In the #dath-ilan channel of the server, Yudkowsky elaborated on the reasoning for the masochism coverup:
623 > altruistic sadists would if-counterfactually-fully-informed prefer not to know, because Civilization is capped on the number of happy sadists. even if you can afford a masochist, which requires being very rich, you're buying them away from the next sadist to whom masochists were previously just barely affordable
625 In response to a question about how frequent sadism is among Keepers, Yudkowsky wrote:
627 > I think they're unusually likely to be aware, nonpracticing potential sexual sadists. Noticing that sort of thing about yourself, and then not bidding against the next sadist over for the limited masochist supply, and instead just operating your brain so that it doesn't hurt much to know what you can't have, is exactly the kind of cost you're volunteering to take on when you say you wanna be a Keeper.
628 > that's archetypally exactly The Sort Of Thing Keepers Do And Are
630 > They choose not to, not just out of consideration for the next person in line, but because not harming the next person in line is part of the explicit bargain of becoming a Keeper.
631 > Like, this sort of thing is exactly what you're signing up for when you throw yourself on the bounded rationality grenade.
632 > Let the truth destroy what it can—but in you, not in other people.
634 I objected (to the room, I told myself, not technically violating my prior intent to not bother Yudkowsky himself anymore) that "Let the truth destroy what it can—in yourself, not in other people" is such an _incredibly_ infantilizing philosophy. It's a meme that optimizes for shaping people (I know, _other_ people) into becoming weak, stupid, and unreflective, like Thellim's impression of Jane Austen characters. I expect people on Earth—not even "rationalists", just ordinary adults—to be able to cope with ... learning facts about psychology that imply that there are desirable sexual experiences they won't get to have.
636 A user called Numendil insightfully pointed out that dath ilani might be skeptical of an Earthling saying that an unpleasant aspect our of existence is actually fine, for the same reason we would be skeptical of a resident of Golarion saying that; it makes sense for people from richer civilizations to look "spoiled" to people from poorer ones.
638 Other replies were more disturbing. One participant wrote:
640 > I think of "not in other people" not as "infantilizing", but as recognizing independent agency. You don't get to do harm to other people without their consent, whether that is physical or pychological.
642 I pointed out that this obviously applies to, say, religion. Was it wrong to advocate for atheism in a religious Society, where robbing someone of their belief in God might be harming them?
644 "Every society strikes a balance between protectionism and liberty," someone said. "This isn't news."
646 It's not news about _humans_, I conceded. It was just—I thought people who were fans of Yudkowsky's writing in 2008 had a reasonable expectation that the dominant messaging in the local subculture would continue in 2022 to be _in favor_ of telling the truth and _against_ benevolently intended Noble Lies. It ... would be interesting to know why that changed.
650 > dath ilan is essentially a paradise world. In a paradise world, people have the slack to make microoptimisations like that, to allow themselves Noble Lies and not fear for what could be hiding in the gaps. Telling the truth is a heuristic for this world where Noble Lies are often less Noble than expected and trust is harder to come by.
652 I said that I thought people were missing this idea that the reason "truth is better than lies; knowledge is better than ignorance" is such a well-performing [injunction](https://www.lesswrong.com/posts/dWTEtgBfFaz6vjwQf/ethical-injunctions) in the real world (despite the fact that there's no law of physics preventing lies and ignorance from having beneficial consequences), is because [it protects against unknown unknowns](https://www.lesswrong.com/posts/E7CKXxtGKPmdM9ZRc/of-lies-and-black-swan-blowups). Of course an author who wants to portray an ignorance-maintaining conspiracy as being for the greater good, can assert by authorial fiat whatever details are needed to make it all turn out for the greater good, but _that's not how anything works in real life_.
654 I started a new thread to complain about the attitude I was seeing (Subject: "Noble Secrets; Or, Conflict Theory of Optimization on Shared Maps"). When fiction in this world, _where I live_, glorifies Noble Lies, that's a cultural force optimizing for making shared maps less accurate, I explained. As someone trying to make shared maps _more_ accurate, this force was hostile to me and mine. I understood that "secrets" and "lies" are not the same thing, but if you're a consequentialist thinking in terms of what kinds of optimization pressures are being applied to shared maps, [it's the same issue](https://www.lesswrong.com/posts/YptSN8riyXJjJ8Qp8/maybe-lying-can-t-exist): I'm trying to steer _towards_ states of the world where people know things, and the Keepers of Noble Secrets are trying to steer _away_ from states of the world where people know things. That's a conflict. I was happy to accept Pareto-improving deals to make the conflict less destructive, but I wasn't going to pretend the pro-ignorance forces were my friends just because they self-identified as "rationalists" or "EA"s. I was willing to accept secrets around nuclear or biological weapons, or AGI, on "better ignorant than dead" grounds, but the "protect sadists from being sad" thing wasn't a threat to anyone's life; it was _just_ coddling people who can't handle reality, which made _my_ life worse.
656 I wasn't buying the excuse that secret-Keeping practices that wouldn't be okay on Earth were somehow okay on dath ilan, which was asserted by authorial fiat to be sane and smart and benevolent enough to make it work. Alternatively, if I couldn't argue with authorial fiat: the reasons why it would be bad on Earth (even if it wouldn't be bad in the author-assertion paradise of dath ilan) are reasons why _fiction about dath ilan is bad for Earth_.
658 And just—back in the 'aughts, I said, Robin Hanson had this really great blog called _Overcoming Bias_. (You probably haven't heard of it.) I wanted that _vibe_ back, of Robin Hanson's blog in 2008—the will to _just get the right answer_, without all this galaxy-brained hand-wringing about who the right answer might hurt.
660 (_Overcoming Bias_ had actually been a group blog then, but I was enjoying the æsthetic of saying "Robin Hanson's blog" (when what I had actually loved about _Overcoming Bias_ was Yudkowsky's Sequences) as a way of signaling contempt for the Yudkowsky of the current year.)
662 I would have expected a subculture descended from the memetic legacy of Robin Hanson's blog in 2008 to respond to that tripe about protecting people from the truth being a form of "recognizing independent agency" with something like—
664 "Hi! You must be new here! Regarding your concern about truth doing harm to people, a standard reply is articulated in the post ["Doublethink (Choosing to be Biased)"](https://www.lesswrong.com/posts/Hs3ymqypvhgFMkgLb/doublethink-choosing-to-be-biased). Regarding your concern about recognizing independent agency, a standard reply is articulated in the post ["Your Rationality Is My Business"](https://www.lesswrong.com/posts/anCubLdggTWjnEvBS/your-rationality-is-my-business)."
666 —or _something like that_. Not that the reply needed to use those particular Sequences links, or _any_ Sequences links; what's important is that someone needed to counter to this very obvious [anti-epistemology](https://www.lesswrong.com/posts/XTWkjCJScy2GFAgDt/dark-side-epistemology).
668 And what we actually saw in response to the "You don't get to do harm to other people" message was ... it got 5 "+1" emoji-reactions.
670 Yudkowsky [chimed in to point out that](/images/yudkowsky-it_doesnt_say_tell_other_people.png) "Doublethink" was about _oneself_ not reasonably being in the epistemic position of knowing that one should lie to oneself. It wasn't about telling the truth to _other_ people.
672 On the one hand, fair enough. My generalization from "you shouldn't want to have false beliefs for your own benefit" to "you shouldn't want other people to have false beliefs for their own benefit" (and the further generalization to it being okay to intervene) was not in the text of the post itself. It made sense for Yudkowsky to refute my misinterpretation of the text he wrote.
674 On the other hand—given that he was paying attention to this #overflow thread anyway, I might have naïvely hoped that he would appreciate what I was trying to do?—that, after the issue had been pointed out, he would decided that he _wanted_ his chatroom to be a place where we don't want other people to have false beliefs for their own benefit?—a place that approves of "meddling" in the form of _telling people things_.
676 The other chatroom participants mostly weren't buying what I was selling.
678 A user called April wrote that "the standard dath ilani has internalized almost everything in the sequences": "it's not that the standards are being dropped[;] it's that there's an even higher standard far beyond what anyone on earth has accomplished". (This received a checkmark emoji-react from Yudkowsky, an indication of his agreement/endorsement.)
680 Someone else said he was "pretty leery of 'ignore whether models are painful' as a principle, for Earth humans to try to adopt," and went on to offer some thoughts for Earth. I continued to maintain that it was ridiculous that we were talking of "Earth humans" as if there were any other kind—as if rationality in the Yudkowskian tradition wasn't something to aspire to in real life.
682 Dath ilan [is _fiction_](https://www.lesswrong.com/posts/rHBdcHGLJ7KvLJQPk/the-logical-fallacy-of-generalization-from-fictional), I pointed out. Dath ilan _does not exist_. I thought it was a horrible distraction to try to see our world through Thellim's eyes and feel contempt over how much better things must be on dath ilan (which, to be clear, again, _does not exist_), when one could be looking through the eyes of an ordinary reader of Robin Hanson's blog in 2008 (the _real_ 2008, which _actually happened_), and seeing everything we've lost.
684 [As it was taught to me then](https://www.lesswrong.com/posts/iiWiHgtQekWNnmE6Q/if-you-demand-magic-magic-won-t-help): if you demand Keepers, _Keepers won't help_. If I'm going to be happy anywhere, or achieve greatness anywhere, or learn true secrets anywhere, or save the world anywhere, or feel strongly anywhere, or help people anywhere—I may as well do it _on Earth_.
686 The thread died out soon enough. I had some more thoughts about dath ilan's predilection for deception, of which I typed up some notes for maybe adapting into a blog post later, but there was no point in wasting any more time on Discord.
688 On 29 November 2022 (four years and a day after the "hill of meaning in defense of validity" Twitter performance that had ignited my rationalist civil war), Yudkowsky remarked about the sadism coverup again:
690 > Keltham is a romantically obligate sadist. This is information that could've made him much happier if masochists had existed in sufficient supply; Civilization has no other obvious-to-me-or-Keltham reason to conceal it from him.
692 Despite the fact that there was no point in wasting any more time on Discord, I decided not to resist the temptation to open up the thread again and dump some paragraphs from my notes on the conspiracies of dath ilan.
694 If we believe that [IQ research validates the "Jews are clever" stereotype](https://web.mit.edu/fustflum/documents/papers/AshkenaziIQ.jbiosocsci.pdf), I wondered if there's a distinct (albeit probably correlated) "enjoying deception" trait that validates the "Jews are sneaky" stereotype? If dath ilan is very high in this "sneakiness" trait (relative to Earth Jews), that would help explain all the conspiracies![^edgy-anti-semitism]
696 [^edgy-anti-semitism]: It probably would have been possible to bring up the sneakiness-trait hypothesis in a less edgy way, but I didn't care to.
698 Not-actually-plausible conspiracies that everyone is in on (like "Sparashki are real") are a [superstimulus](https://www.lesswrong.com/posts/Jq73GozjsuhdwMLEG/superstimuli-and-the-collapse-of-western-civilization) like zero-calorie sweetener: engineered to let everyone enjoy the thrill of lying, without doing any real damage to shared maps.
700 In "For No Laid Course Prepare", Merrin's coworkers falsely maintain to outsiders that Merrin always cosplays as a Sparashki while on duty. ["This is not considered a lie, in that it would be universally understood and expected that no one in this social circumstance would tell the truth,"](https://www.glowfic.com/replies/1857346#reply-1857346) the narrator tells us. The language used here is strikingly similar to that of one of the corrupt executives in [_Moral Mazes_](https://thezvi.wordpress.com/2019/05/30/quotes-from-moral-mazes/): "We lie all the time, but if everyone knows that we're lying, is a lie really a lie?"
702 But if [everyone really knew](https://thezvi.wordpress.com/2019/07/02/everybody-knows/), what would be _function_ of saying the false thing? On dath ilan (if not in Earth boardrooms), one supposes the answer is "Because it's fun"? But that just prompts the followup question: but what is the function of the brain giving out a "fun" reward in this context? It seems like at _some_ point, there has to be the expectation of _some_ cognitive system (although possibly not an entire "person") taking the signals literally.[^funny-or-powerful-falsehood]
704 [^funny-or-powerful-falsehood]: This is why, when I notice myself misrepresenting my actual beliefs or motivations because I think it's funny or rhetorically powerful, I often take care to disclaim it immediately, precisely because I _don't_ think that "everybody knows"; I'm not going to give up on humor or powerful rhetoric, but I'm also not going to delude myself into thinking it's "zero-calorie": people who don't "get the joke" _are_ going to be misled, and I don't think it's unambigously "their fault" for not being able to read my "intent" to arbitrary precision. But maybe dath ilan is (by authorial fiat) sufficiently good at achieving common knowledge in large groups that they _can_ pull off a zero-calorie "everyone knows" conspiracy without damaging shared maps?
706 The existence of such a widespread sneakiness/"taste for deception" trait among the eliezera, in conjunction with their culture just not particularly valuing public knowledge (because they assume everything important is being handled by the Keepers), explains the recurring conspiracies and coverups, like the Ordinary Merrin Conspiracy, Exception Handling's fabrication of evidence for Sparashki being real, the sadism/masochism coverup, and [the village that deliberately teaches anti-redhead bigotry to children in order to test the robustness of dath ilan's general humanism indoctrination](https://www.lesswrong.com/posts/uyBeAN5jPEATMqKkX/lies-told-to-children-1).
708 I stress (at least now, even if I didn't do a good job of explaining it at the time) that this hypothesis _doesn't_ require dath ilani to be cartoon villains who hate knowledge and want people to be ignorant. Just that, as a result of the widespread sneakiness trait and their outsourcing information-process to the Keepers, in the course of trying to accomplish other things, plans-that-involve-conspiracies are often higher in their search ordering than plans-that-involve-keeping-people-informed.
710 I claimed that there was a hidden-core-of-rationality thing about a culture that values living in truth, that the dath ilani didn't have. In previous discussion of the Sparashki example, a user called lc had written, "If you see someone wearing an elf costume at work and conclude elves are real and make disastrous decisions based on that conclusion you are mentally deranged". And indeed, you would be mentally deranged if you did that _on Earth_, because we don't have an elves-are-real conspiracy on Earth.
712 In elves-are-real conspiracy-world, you (Whistleblower) see someone (Conspirator) wearing an elf costume at work and say, "Nice costume." They say, "What costume?" You say, "I see that you're dressed like an elf, but elves aren't real." They say, "What do you mean? Of course elves are real. I'm right here." You say, "You know exactly what I mean."
714 It would appear that there's a conflict between Conspirator (who wants to maintain a social reality in which they're an elf, because it's fun, and the conspiracy is sufficiently outlandish that it's assumed that no one is "really" being deceived) and Whistleblower (who wants default social reality to map to actual reality; make-believe is fine at a designated fandom convention which has designated boundaries, but let's be serious at work, where your coworkers are trying to make a living and haven't opted-in to this false social reality).
716 I was skeptical that a culture where people collude to maintain a fake social reality at their job in a hospital, and everyone else is expected to play along because it's fun, really has this living-in-truth thing. People play those social-reality games on Earth, too, and when _they_ say no one is being deceived, they're _definitely_ lying about that, and I doubted that the eliezera were actually built that differently.
718 A user called ajvermillion asked why I was being so aggressively negative about dath ilan. He compared it to Keltham's remark about how [people who grew up under a Lawful Evil government were disposed to take a more negative view of paternalism](https://www.glowfic.com/replies/1874754#reply-1874754) than they do in dath ilan, where paternalism basically works fine because dath ilan is benevolent.
720 This question put me in a somewhat awkward position: it was a legitimate question that I felt I had to answer, that I had no way of answering honestly without at least _alluding_ to my prior greviances against Yudkowsky ... which were off-topic for the server. (Again, I had told myself that I was here to comment on the story, not to prosecute my greviances.)
722 I tried to explain, briefly. Someone who might be even _more_ paranoid about abuses of power than someone who grew up with a Lawful Evil government, is someone who grew up under a power structure that put on a _good show_ of being clean and nice, but was actually corrupt and mean.
724 Yudkowsky had this whole marketing image of him being uniquely sane and therefore uniquely benevolent, and because his Sequences were so life-changingly good, I _actually fell for it_. There was a long Dumb Story (this Story) that was off-topic and I hadn't then finished writing up, but basically, I had what I claimed were very strong reasons not to trust the guy anymore; I think he cares a lot about not explicitly _lying_, but what made the Sequences special is that they articulated a vastly higher standard than that, that he had no intention of living up to.
726 And so, yeah, insofar as fiction about dath ilan functioned as marketing material for Yudkowsky's personality cult that I thought was damaging people like me (in some ways, while simultaneously helping us in other ways), I had an incentive to come up with literary criticism that paints dath ilan negatively?
728 It was great for ajvermillion to notice this! It _would_ be bad if my brain were configured to come up with dath-ilan-negative literary criticism, and for me to _simultaneously_ present myself as an authority on dath ilan whom you should trust. But if dath-ilan-negative literary criticism was undersupplied for structural reasons (because people who like a story are selected for not seeing things the story is doing that are Actually Bad), and my brain was configured to generate it anyway (because I disliked the person Yudkowsky had become, in contrast to the person he was in 2008), it seemed pro-social for me to post it, for other people to take or leave according to their own judgement?
730 Yudkowsky soon entered the thread again, initially replying to someone else. (I remarked parenthetically that his appearance made me think I should stop wasting time snarking in his fiction server and just finish my memoir already.) We had a brief back-and-forth:
732 > **Eliezer** — 11/29/2022 10:33 PM
733 > the main thing I'd observe contrary to Zack's take here, is that Keltham thought that not learning about masochists he can never have, was obviously in retrospect what he'd have wanted Civilization to do, or do unless and until Keltham became rich enough to afford a masochist and then he could be told
734 > in other words, Keltham thought he was obviously being treated the way that counterfactual fully-informed Keltham would have paid Governance to treat not-yet-informed Keltham
735 > that this obeys the social contract that Keltham thought he had, is part of why Keltham is confident that the logic of this particular explanation holds together
736 > **zackmdavis** — 11/29/2022 10:35 PM
737 > the level of service that Keltham is expecting is _not the thing I learned from Robin Hanson's blog in 2008_
738 > **Eliezer** — 11/29/2022 10:36 PM
739 > I am sorry that some of the insane people I attracted got together and made each other more insane and then extensively meta-gaslit you into believing that everyone generally and me personally was engaging in some kind of weird out-in-the-open gaslighting that you could believe in if you attached least-charitable explanations to everything we were doing
741 It was pretty annoying that Yudkowsky was still attributing my greviances to Michael's malign influence—as if the gender identity revolution was something I would otherwise have just _taken lying down_. In the counterfactual where Michael had died in 2015, I think something like my February 2017 breakdown would have likely happened anyway. (Between August 2016 and January 2017, I sent Michael 14 emails, met with him once, and watched 60% of South Park season 19 at his suggestion, so he was _an_ influence on my thinking during that period, but not a disproportionately large one compared to everything else I was doing at the time.) How would I have later reacted to the November 2018 "hill of meaning" Tweets (assuming they weren't butterfly-effected away in this counterfactual)? It's hard to say. Maybe, if that world's analogue of my February 2017 breakdown had gone sufficiently badly (with no Michael to visit me in the psych ward or help me make sense of things afterwards), I would have already been a broken man, and not even sent Yudkowsky an email. In any case, I feel very confident that my understanding of the behavior of "everyone generally and [Yudkowsky] personally" would not have been _better_ without Michael _et al._'s influence.
744 > you may recall that this blog included something called the "Bayesian Conspiracy"
745 > they won't tell you about it, because it interferes with the story they were trying to drive you insaner with, but it's so
746 > **zackmdavis** — 11/29/2022 10:37 PM
747 > it's true that the things I don't like about modern Yudkowsky were still there in Sequences-era Yudkowsky, but I think they've gotten _worse_
748 > **Eliezer** — 11/29/2022 10:39 PM
749 > well, if your story is that I was always a complicated person, and you selected some of my posts and liked the simpler message you extracted from those, and over time I've shifted in my emphases in a way you don't like, while still having posts like Meta-Honesty and so on... then that's a pretty different story than the one you were telling in this Discord channel, like, just now. today.
751 Is it, though? The "always a complicated person [who has] shifted in [his] emphases in a way [I] don't like" story was true, of course, but it elided the substantive reasons _why_ I didn't like the new emphases, which could presumably be evaluated on their own merits.
753 It's interesting that Yudkowsky listed "still having posts like Meta-Honesty" as an exculpatory factor here. The thing is, I [wrote a _critique_ of Meta-Honesty](https://www.lesswrong.com/posts/MN4NRkMw7ggt9587K/firming-up-not-lying-around-its-edge-cases-is-less-broadly). It was well-received (being [cited as a good example in the introductory post for the 2019 Less Wrong Review](https://www.lesswrong.com/posts/QFBEjjAvT6KbaA3dY/the-lesswrong-2019-review), for instance). I don't think I could have written a similarly impassioned critique of anything from the Sequences era, because the stuff from the Sequences era still looked _correct_ to me. To me, "Meta-Honesty" was evidence _for_ Yudkowsky having relinquished his Art and lost his powers, not evidence that his powers were still intact.
755 I didn't have that response thought through in real time. At the time, I just agreed:
757 > **zackmdavis** — 11/29/2022 10:39 PM
759 > I'm not done working through the hate-warp
761 (This being a reference to part of _Planecrash_ in which [Keltham tells Carissa to be aware of her un-dath ilani tendency to feel "hatred that warps reality to be more hateable"](https://www.glowfic.com/replies/1882822#reply-1882822).)
763 > **Eliezer** — 11/29/2022 10:40 PM
764 > so one thing hasn't changed: the message that you, yourself, should always be trying to infer the true truth, off the information you already have.
765 > if you know you've got a hate-warp I don't know why you're running it and not trying to correct for it
766 > are you in fact also explicitly aware that the people who talk to you a lot about "gaslighting" are, like, insane?
767 > **zackmdavis** — 11/29/2022 10:42 PM
768 > I'm not really part of Vassar's clique anymore, if that's what you mean
769 > **Eliezer** — 11/29/2022 10:44 PM
770 > it looks from outside here like they stomped really heavy footprints all over your brain that have not healed or been filled in
771 > **zackmdavis** — 11/29/2022 10:49 PM
772 > it looks from inside here that the thing I'm not healed from is the thing where, as Oliver Habryka put it, I "should expect that depending on the circumstances community leaders might make up sophisticated stories for why pretty obviously true things are false" ([https://www.lesswrong.com/posts/juZ8ugdNqMrbX7x2J/challenges-to-yudkowsky-s-pronoun-reform-proposal?commentId=he8dztSuBBuxNRMSY](https://www.lesswrong.com/posts/juZ8ugdNqMrbX7x2J/challenges-to-yudkowsky-s-pronoun-reform-proposal?commentId=he8dztSuBBuxNRMSY)), and Michael and Ben and Jessica were _really_ helpful for orienting me to that particular problem, even if I disagree with them about a lot of other things and they seem crazy in other ways
773 > (rule thinkers in, not out)
775 (I was pleased to get the link to Habryka's comment in front of Yudkowsky, if he hadn't already seen it.)
777 > **Eliezer** — 11/29/2022 10:55 PM
778 > the most harm they did you was to teach you to see malice where you should have seen mortality
779 > noninnocent error is meaningfully different from innocent error; and noninnocent error is meaningfully different from malice
780 > Keltham deduced the lack of masochists in dath ilan by asking the question, "Why would Civilization have kept this information from me?", _ruling out_ or actually not even thinking of such ridiculous hypotheses as "Because it was fun", and settling on the obvious explanation that explained _why Keltham would have wanted Civilization to do that for him_—masochists not existing or being incredibly rare and unaffordable to him. You looked at this and saw malice everywhere; you couldn't even see _the fictional world_ the author was trying to give you _direct description about_. You didn't say that you disbelieved in the world; you could not see what was being _described_.
782 (When a literary critic proposes a "dark" interpretation of an author's world, I think it's implied that they're expressing disbelief in the "intended" world; the fact that I was impudently refusing to buy the benevolent interpretation wasn't because I didn't understand it.)
784 > Hate-warp like this is bad for truth-perception; my understanding of the situation is that it's harm done to you by the group you say you left. I would read this as being a noninnocent error of that group; that they couldn't get what they wanted from people who still had friends outside their own small microculture, and noninnocently then decided that this outer culture was bad and people needed to be pried loose from it. They tried telling some people that this outer culture was gaslighting them and maliciously lying to them and had to be understood in wholly adversarial terms to break free of the gaslighting; that worked on somebody, and made a new friend for them; so their brain noninnocently learned that it ought to use arguments like that again, so they must be true.
785 > This is a sort of thing I super did not do because I _understood_ it as a failure mode and Laid My Go Stones Against Ever Actually Being A Cult; I armed people with weapons against it, or tried to, but I was optimistic in my hopes about how much could actually be taught.
786 > **zackmdavis** — 11/29/2022 11:20 PM
787 > Without particularly defending Vassar _et al._ or my bad literary criticism (sorry), _modeling the adversarial component of non-innocent errors_ (as contrasted to "had to be understood in wholly adversarial terms") seems very important. (Maybe lying is "worse" than rationalizing, but if you can't hold people culpable for rationalization, you end up with a world that's bad for broadly the same reasons that a world full of liars is bad: we can't steer the world to good states if everyone's map is full of falsehoods that locally benefitted someone.)
788 > **Eliezer** — 11/29/2022 11:22 PM
789 > Rationalization sure is a huge thing! That's why I considered important to discourse upon the science of it, as was then known; and to warn people that there were more complicated tangles than that, which no simple experiment had shown yet.
790 > **zackmdavis** — 11/29/2022 11:22 PM
792 > **Eliezer** — 11/29/2022 11:23 PM
793 > It remains something that mortals do, and if you cut off anybody who's ever done that, you'll be left with nobody. And also importantly, people making noninnocent errors, if you accuse them of malice, will look inside themselves and correctly see that this is not how they work, and they'll stop listening to the (motivated) lies you're telling them about themselves.
794 > This also holds true if you make up overly simplistic stories about 'ah yes well you're doing that because you're part of $woke-concept-of-society' etc.
795 > **zackmdavis** — 11/29/2022 11:24 PM
796 > I think there's _also_ a frequent problem where you try to accuse people of non-innocent errors, and they motivatedly interpret _you_ as accusing malice
797 > **Eliezer** — 11/29/2022 11:25 PM
798 > Then invent new terminology. I do that all the time when existing terminology fails me.
799 > Like I literally invented the term 'noninnocent error' right in this conversation.
800 > **zackmdavis** — 11/29/2022 11:27 PM
801 > I've tried this, but maybe it wasn't good enough, or I haven't been using it consistently enough: [https://www.lesswrong.com/posts/sXHQ9R5tahiaXEZhR/algorithmic-intent-a-hansonian-generalized-anti-zombie](https://www.lesswrong.com/posts/sXHQ9R5tahiaXEZhR/algorithmic-intent-a-hansonian-generalized-anti-zombie)
802 > I should get ready for bed
803 > I will endeavor to edit out the hate-warp from my memoir before publishing, and _probably_ not talk in this server
804 > **Eliezer** — 11/29/2022 11:31 PM
805 > I think you should worry first about editing the hate-warp out of yourself, but editing the memoir might be useful practice for it. Good night.
807 It turned out that I was lying about probably not talking in the server anymore. (Hedging with the word "probably" didn't make the claim true, and of course I wasn't _consciously_ lying, but that hardly seems exculpatory.)
809 The next day, I belatedly pointed out that "Keltham thought that not learning about masochists he can never have, was obviously in retrospect what he'd have wanted Civilization to do" seemed to contradict "one thing hasn't changed: the message that you, yourself, should always be trying to infer the true truth". In the first statement, it didn't sound like Keltham thinks it's good that Civilization didn't tell him so that he could figure it out for himself (in accordance with the discipline of "you, yourself, always trying to infer the truth"). It sounded like he was better off not knowing—better off having a _less accurate self-model_ (not having the concept of "obligate romantic sadism"), better off having a _less accurate world-model_ (thinking that masochism isn't real).
811 In response to someone positing that dath ilani were choosing to be happier but less accurate predictors, I said that I read a blog post once about why you actually didn't want to do that, linking to [an Internet Archive copy of "Doublethink (Choosing to Be Biased)"](https://web.archive.org/web/20080216204229/https://www.overcomingbias.com/2007/09/doublethink-cho.html) from 2008[^hanson-conceit]—at least, that was _my_ attempted paraphrase; it was possible that I'd extracted a simpler message from it than the author intended.
813 [^hanson-conceit]: I was really enjoying the "Robin Hanson's blog in 2008" conceit.
815 A user called Harmless explained the loophole. "Doublethink" was pointing out that decisions that optimize the world for your preferences can't come from nowhere: if you avoid painful thoughts in your map, you damage your ability to steer away from painful outcomes in the territory. However, there was no rule that all the information-processing going into decisions that optimize the world for your preferences had to take place in _your brain_ ...
817 I saw where they were going and completed the thought: you could build a Friendly AI or a Civilization to see all the dirty things for you, that would make you unhappy to have to see yourself.
819 Yudkowsky clarified his position:
821 > My exact word choices often do matter: I said that you should always be trying to _infer_ the truth. With the info you already have. In dath ilan if not in Earth, you might decline to open a box labeled "this info will make you permanently dissatisfied with sex" if the box was labeled by a prediction market.
822 > Trying to avoid inferences seems to me much more internally costly than declining to click on a spoiler box.
824 I understood the theory, but I was still extremely skpetical of the practice, assuming the eliezera were even remotely human. Yudkowsky described the practice of "keeping BDSM secret and trying to prevent most sadists from discovering what they are—informing them only when and if they become rich enough or famous enough that they'd have a high probability of successfully obtaining a very rare masochist" as a "basically reasonable policy option that [he] might vote for, not to help the poor dear other people, but to help [his] own counterfactual self."
826 The problem I saw with this is that becoming rich and famous isn't a purely random exogenous event. In order to make an informed decision about whether or not to put in the effort to try to _become_ rich and famous (as contrasted to choosing a lower-risk or more laid-back lifestyle), you need accurate beliefs about the perks of being rich and famous.
828 The dilemma of whether to make more ambitious economic choices in pusuit of sexual goals was something that _already_ happens to people on Earth, rather than being hypothetical. I once met a trans woman who spent a lot of her twenties and thirties working very hard to get money for various medical procedures. I think she would be worse off under a censorship regime run by self-styled Keepers who thought it was kinder to prevent _poor people_ from learning about the concept of "transsexualism".
830 Further discussion established that Yudkowsky was (supposedly) already taking into account that class of distortion on individuals' decisions, but that the empirical setting of probabilities and utilities happened to be such that ignorance came out on top.
832 I wasn't sure what my wordcount and "diplomacy" "budget limits" for the server were, but I couldn't let go; I kept the thread going on subsequent days. There was something I felt I should be able to convey, if I could just find the right words.
834 When [Word of God](https://tvtropes.org/pmwiki/pmwiki.php/Main/WordOfGod) says, "trying to prevent most [_X_] from discovering what they are [...] continues to strike me as a basically reasonable policy option", then, separately from the particular value of _X_, I expected people to jump out of their chairs and say, "No! This is wrong! Morally wrong! People can stand what is true about themselves, because they are already doing so!"
836 And to the extent that I was the only person jumping out of my chair, and there was a party-line response of the form, "Ah, but if it's been decreed by authorial fiat that these-and-such probabilities and utilities take such-and-these values, then in this case, self-knowledge is actually bad under the utilitarian calculus," I wasn't disputing the utilitarian calculus. I was wondering—here I used the "🐛" bug emoji customarily used in Glowfic culture to indicate uncertainty about the right words to use—_who destroyed your souls?_
840 > it feels powerfully relevant to me that the people of whom I am saying this _are eliezera_. I get to decide what they'd want because, unlike with Earth humans, I get to put myself in their shoes. it's plausible to me that the prediction markets say that I'd be sadder if I was exposed to the concept of sadism in a world with no masochists. if so, while I wouldn't relinquish my Art and lose my powers by trying to delude myself about that once I'd been told, I'd consider it a friendly act to keep the info from me—_because_ I have less self-delusional defenses than a standard Earthling, really—and a hostile act to tell me; and if you are telling me I don't get to make that decision for myself because it's evil, and if you go around shouting it from the street corners in dath ilan, then yeah I think most cities don't let you in.
842 I wish I had thought to ask if he'd have felt the same way in 2008.
844 Ajvermillion was still baffled at my skepticism: if the author specifies that the world of the story is simple in this-and-such direction, on what grounds could I _disagree_?
846 I admitted, again, that there was a sense in which I couldn't argue with authorial fiat. But I thought that an author's choice of assumptions reveals something about what they think is true in our world, and commenting on that should be fair game for literary critics. Suppose someone wrote a story and said, "in the world portrayed in this story, everyone is super-great at _kung fu_, and they could beat up everyone from our Earth, but they never have to practice at all."
848 (Yudkowsky retorted, "...you realize you're describing like half the alien planets in comic books? when did Superman ever get depicted as studying kung fu?" I wish I had thought to admit that, yes, I _did_ hold Eliezer Yudkowsky to a higher standard of consilient worldbuilding than DC Comics. Would he rather I _didn't_?)
850 Something about innate _kung fu_ world seems fake in a way that seems like a literary flaw. It's not just about plausibility. Fiction often incorporates unrealistic elements in order to tell a story that has relevance to real human lives. Innate _kung fu_ skills are scientifically plausible[^instinct] in a way that faster-than-light travel is not, but throwing faster-than-light travel into the universe so that you can do a [space opera](https://tvtropes.org/pmwiki/pmwiki.php/Main/SpaceOpera) doesn't make the _people_ fake in the way that Superman's fighting skills are fake.
852 [^instinct]: All sorts of other instinctual behaviors exist in animals; I don't se why skills humans have to study for years as a "martial art" couldn't be coded into the genome.
854 Maybe it was okay for Superman's fighting skills to be fake from a literary perspective (because realism along that dimension is not what Superman is _about_), but if the Yudkowskian ethos exulted intelligence as ["the power that cannot be removed without removing you"](https://www.lesswrong.com/posts/SXK87NgEPszhWkvQm/mundane-magic), readers had grounds to demand that the dath ilani's thinking skills be real, and a world that's claimed by authorial fiat to be super-great at epistemic rationality, but where the people don't have a will-to-truth stronger than their will-to-happiness, felt fake to me. I couldn't _prove_ that it was fake. I agreed with Harmless's case that, _technically_, as far as the Law went, you could build a Civilization or a Friendly AI to see all the ugly things that you preferred not to see.
856 But if you could—would you? And more importantly, if you would—could you?
858 It was possible that the attitude I was evincing here was just a difference between the eliezera out of dath ilan and the Zackistani from my medianworld, and that there was nothing more to be said about it. But I didn't think the thing was a _genetic_ trait of the Zackistani! _I_ got it from spending my early twenties obsessively re-reading blog posts that said things like, ["I believe that it is right and proper for me, as a human being, to have an interest in the future [...] One of those interests is the human pursuit of truth [...] I wish to strengthen that pursuit further, in this generation."](https://www.lesswrong.com/posts/anCubLdggTWjnEvBS/your-rationality-is-my-business)
860 There were definitely communities on Earth where I wasn't allowed in because of my tendency to shout things from street corners, and I respected those people's right to have a safe space for themselves.
862 But those communities ... didn't call themselves _rationalists_, weren't _pretending_ be to be inheritors of the great tradition of E. T. Jaynes and Robin Dawes and Richard Feynman. And if they _did_, I think I would have a false advertising complaint against them.
864 "[The eleventh virtue is scholarship. Study many sciences and absorb their power as your own](https://www.yudkowsky.net/rational/virtues) ... unless a prediction market says that would make you less happy," just didn't have the same ring to it. Neither did "The first virtue is curiosity. A burning itch to know is higher than a solemn vow to pursue truth. But higher than both of those, is trusting your Society's institutions to tell you which kinds of knowledge will make you happy"—even if you stipulated by authorial fiat that your Society's institutions are super-competent, such that they're probably right about the happiness thing.
866 Attempting to illustrate [the mood I thought dath ilan was missing](https://www.econlib.org/archives/2016/01/the_invisible_t.html), I quoted (with Discord's click-to-reveal spoiler blocks around the more plot-relevant sentences) the scene from _Atlas Shrugged_ where our heroine Dagny expresses a wish to be kept ignorant for the sake of her own happiness, and gets shut down by John Galt—and Dagny _thanks_ him.[^atlas-shrugged-ref]
868 > "[...] Oh, if only I didn't have to hear about it! If only I could stay here and never know what they're doing to the railroad, and never learn when it goes!"
870 > "You'll have to hear about it," said Galt; it was that ruthless tone, peculiarly his, which sounded implacable by being simple, devoid of any emotional value, save the quality of respect for facts. "You'll hear the whole course of the last agony of Taggart Transcontinental. You'll hear about every wreck. You'll hear about every discontinued train. You'll hear about every abandoned line. You'll hear about the collapse of the Taggart Bridge. Nobody stays in this valley except by a full, conscious choice based on a full, conscious knowledge of every fact involved in his decision. Nobody stays here by faking reality in any manner whatever."
872 > She looked at him, her head lifted, knowing what chance he was rejecting. She thought that no man of the outer world would have said this to her at this moment—she thought of the world's code that worshipped white lies as an act of mercy—she felt a stab of revulsion against that code, suddenly seeing its full ugliness for the first time [...] she answered quietly, "Thank you. You're right."
874 [^atlas-shrugged-ref]: In Part Three, Chapter II, "The Utopia of Greed".
876 This (probably predictably) failed to resonate with other server participants, who were baffled why I seemed to be appealing to Ayn Rand's authority.
878 I was actually going for a _reverse_ appeal-to-authority: if _Ayn Rand_ understood that facing reality is virtuous, why didn't the 2020s "rationalists"? Wasn't that undignified? I didn't think the disdain for "Earth people" (again, as if there were any other kind) was justified, when Earth's philosophy of rationality (as exemplified by Ayn Rand or Robert ["Get the Facts"](https://www.goodreads.com/quotes/38764-what-are-the-facts-again-and-again-and-again) Heinlein) was doing better than dath ilan's on this critical dimension.
880 But if people's souls had been damaged such that they didn't have the "facing reality is virtuous" gear, it wasn't easy to install the gear by talking at them.
882 Why was I so sure _my_ gear was correct?
884 I wondered if the issue had to do with what Yudkowsky had [identified as the problem of non-absolute rules](https://www.lesswrong.com/posts/xdwbX9pFEr7Pomaxv/meta-honesty-firming-up-honesty-around-its-edge-cases#5__Counterargument__The_problem_of_non_absolute_rules_), where not-literally-absolute rules like "Don't kill" or "Don't lie" have to be stated _as if_ they were absolutes in order to register to the human motivational system with sufficient force.
886 Technically, as a matter of decision theory, "sacred values" are crazy. It's easy to say—and feel with the passion of religious conviction—that it's always right to choose Truth and Life, and that no one could choose otherwise except wrongly, in the vile service of Falsehood and Death. But reality presents us with quantitative choices over uncertain outcomes, in which everything trades off against everything else under the [von Neumann–Morgenstern axioms](https://en.wikipedia.org/wiki/Von_Neumann%E2%80%93Morgenstern_utility_theorem); if you had to choose between a small, unimportant Truth and the Life of millions, you'd probably choose Life—but more importantly, the very fact that you might have to choose, means that Truth and Life can't both be infinitely sacred to you, and must be measured on a common scale with lesser goods like mere Happiness.
888 I knew that. The other people in the chatroom knew that. So to the extent that the argument amounted to me saying "Don't lie" (about the existence of masochism), and them saying "Don't lie unless the badness of lying is outweighed by the goodness of increased happiness", why was I so confident that I was in the right, when they were wisely acknowledging the trade-offs under the Law, and I was sticking to my (incoherent) sacred value of Truth? Didn't they obviously have the more sophisticated side of the argument?
890 The problem was that, in my view, the people who weren't talking about Truth as if it were a sacred value were being _wildly recklessly casual_ about harms from covering things up, as if they didn't see the non-first-order harms _at all_. I felt I had to appeal to the lessons for children about how Lying Is Bad, because if I tried to make a more sophisticated argument about it being _quantitatively_ crazy to cover up psychology facts that make people sad, I would face a brick wall of "authorial fiat declares that the probabilities and utilities are specifically fine-tuned such that ignorance is good".
892 Even if you specified by authorial fiat that "latent sadists could use the information to decide whether or not to try to become rich and famous" didn't tip the utility calculus in itself, [facts are connected to each other](https://www.lesswrong.com/posts/wyyfFfaRar2jEdeQK/entangled-truths-contagious-lies); there were _more consequences_ to the coverup, more ways in which better-informed people could make better decisions than worse-informed people.
894 What about the costs of all the other recursive censorship you'd have to do to keep the secret? (If a biography mentioned masochism in passing along with many other traits of the subject, you'd need to either censor the paragraphs with that detail, or censor the whole book. Those are real costs, even under a soft-censorship regime where people can give special consent to access "Ill Advised" products.) Maybe latent sadists could console themselves with porn if they knew, or devote their careers to making better sex robots, just as people on Earth with non-satisfiable sexual desires manage to get by. (I _knew some things_ about this topic.) What about dath ilan's heritage optimization (read: eugenics) program? Are they going to try to breed more masochists, or fewer sadists, and who's authorized to know that? And so on.
896 Or imagine a world where male homosexuality couldn't be safely practiced due to super-AIDS. (I know very little about BDSM.) I still think men with that underlying predisposition would be better off _having a concept_ of "homosexuality" (even if they couldn't practice it), rather than the concept itself being censored. There are also other systematic differences that go along with sexual orientation (the "feminine gays, masculine lesbians" thing); if you censor the _concept_, you're throwing away that knowledge.
898 (When I had brought up the super-AIDS hypothetical in the chat, Ajvermillion complained that I was trying to bait people into self-cancelling by biting the bullet on suppressing homosexuality. I agreed that the choice of example was engineered to activate people's progressive moral intuitions about gay rights—it was great for him to notice that—but I thought that colliding philosophical intuitions like that was intellectually productive; it wasn't an attempt to gather blackmail material.)
900 A user called RationalMoron asked if I was appealing to a terminal value. Did I think people should have accurate self-models even if they didn't want to?
902 Obviously I wasn't going to use a universal quantifier over all possible worlds and all possible minds, but in human practice, yes: people who prefer to believe lies about themselves are doing the wrong thing; people who lie to their friends to keep them happy are doing the wrong thing. People can stand what is true, because they are already doing so. I realized that this was a children's lesson without very advanced math, but I thought it was a better lesson than, "Ah, but what if a _prediction market_ says they can't???"
904 I maintained that the fact that the eliezera prefer not to know that there are desirable sexual experiences that they can't have, contradicted April's earlier claim (which had received a Word of God checkmark-emoji) that "it's not that the standards are being dropped[;] it's that there's an even higher standard far beyond what anyone on earth has accomplished".
906 Apparently I struck a nerve. Yudkowsky started "punching back":
908 > **Eliezer** — 12/08/2022 12:45 PM
909 > Do zacki have no concept of movie spoilers, such that all movies are just designed not to rely on uncertainty for dramatic tension? Do children have to be locked in individual test rooms because they can't comprehend the concept of refusing to look at other children's answer sheets because it's evidence and you should observe it? Do adults refuse to isolate the children so they can have practice problems, because you can't stop them from learning the answer to skill-building problems, only the legendary evil alien eliezera would do that? Obviously they don't have surprise parties.
910 > It's noticeably more extreme than the _Invention of Lying_ aliens, who can still have nudity taboos
911 > I'd also note that I think in retrospect (only after having typed it) that Zack could not have generated these examples of other places where society refrains from observation, and that I think this means I am tracking the thing Zack fears in a way that Zack cannot because his thinking is distorted and he is arguing rather than seeing; and this, not verbally advocating for "truth", is more what respect for truth really is.
913 I thought the "you could not have generated the answer I just told you" gambit was a pretty dirty argumentative trick on Yudkowsky's part. (Given that I could, how would I be able to prove it?—this was itself a good use-case for concealing spoilers.)
915 As it happened, however, I _had_ already considered the case of spoilers as a class of legitimate infohazards, and was prepared to testify that I had already thought of it, and explain why I thought hiding spoilers were relevantly morally different from the coverups I was objecting to. The previous night, 7 December 2022, I had had a phone call with Anna Salamon,[^evidence-of-independent-generation] in which I (remembered that I) had cited dath ilan's [practice of letting children figure out heliocentrism for themselves](https://www.glowfic.com/replies/1777588#reply-1777588) as not being objectionable in the way the sadism/masochism coverup was.
917 [^evidence-of-independent-generation]: I was lucky to be able to point to Anna as a potential witness to defend myself against the "could not have generated" trick—as a matter of principle, not because I seriously expected anyone to care enough to go ask Anna if she remembered the conversation the same way.
919 I also mentioned that when I had used spoiler blocks on the _Atlas Shrugged_ quote I had posted upthread, I had briefly considered making some kind of side-remark noting that the spoiler blocks were also a form of information-hiding, but couldn't think of anything funny or relevant enough (which, if my self-report could be trusted, showed that I had independently generated the idea of spoilers being an example of hiding information—but I didn't expect other people to uncritically believe my self-reports).
921 It seemed like the rationale for avoiding spoilers of movie plots or homework exercises had to do with the outcome being different if you got spoiled: you have a different æsthetic experience if you experience the plot twist in the 90th minute of the movie rather than the fourth paragraph of the _Wikipedia_ article. Dath ilan's sadism/masochism coverup didn't seem to have the same structure: when I try to prove a theorem myself before looking at how the textbook says to do it, it's not because I would be _sad about the state of the world_ if I looked at the textbook; it's because the temporary ignorance of working it out myself results in a stronger state of final knowledge.
923 That is, the difference between "spoilers" (sometimes useful) and "coverups" (bad) had to do with whether the ignorant person is expected to eventually uncover the hidden information, and whether the ignorant person knows that there's hidden information that they're expected to uncover. In the case of the sadism/masochism coverup (in contrast to the cases of movie spoilers or homework exercises), it seemed like neither of these conditions pertained. (Keltham knows that the Keepers are keeping secrets, but he seems to actively have beliefs about human psychology that imply masochism is implausible; it seems more like he has a false map, rather than a blank spot on his map for the answer to the homework exercise to be filled in.) I thought that was morally relevant.
925 (Additionally, I would have hoped that my two previous mentions in the thread of supporting keeping nuclear, bioweapon, and AI secrets should have already made it clear that I wasn't against _all_ cases of Society hiding information, but to further demonstrate my ability to generate counterexamples, I mentioned that I would also admit _threats_ as a class of legitimate infohazard: if I'm not a perfect decision theorist, I'm better off if Tony Soprano just doesn't have my email address to begin with, if I don't trust myself to calculate when I "should" ignore his demands.)
927 As for the claim that my thinking was distorted and I was arguing instead of seeing, it was definitely true that I was _motivated to look for_ criticisms of Yudkowsky and dath ilan, for personal reasons outside the scope of the server, and I thought it was great for people to notice this and take it into account. I hoped to nevertheless be competent to only report real criticisms and not fake criticisms. (Whether I succeeded, of course, was up to the reader to decide.)
931 > only half the battle even if you could do it. you're also not reporting any facts/arguments on the other side, which is a much larger and visible gap to me, and has a lot to do with why I'm not presently considering this criticism from a peer despite your spoken adherence to virtues I value. **QUESTION FOR ZACK ONLY, NOBODY ELSE ANSWER OR SAY ANYTHING ABOUT IT IN THIS MAIN CHANNEL:** What are some of the ways that Planecrash valorizes truth, as you, yourself, see that virtue?
933 I didn't ask why it was relevant whether or not I was a "peer." If we're measuring IQ (143 _vs._ [131](/images/wisc-iii_result.jpg)), or fiction-writing ability (several [highly-acclaimed](https://www.lesswrong.com/posts/HawFh7RvDM4RyoJ2d/three-worlds-collide-0-8) [stories](https://www.yudkowsky.net/other/fiction/the-sword-of-good) [including the world's most popular _Harry Potter_ fanfiction](https://www.hpmor.com/) _vs._ a [_My Life as a Teenage Robot_ fanfiction](https://archive.ph/WdydM) with double-digit favorites and a [few](/2018/Jan/blame-me-for-trying/) [blog](http://zackmdavis.net/blog/2016/05/living-well-is-the-best-revenge/) [vignettes](https://www.lesswrong.com/posts/dYspinGtiba5oDCcv/feature-selection) here and there), or contributions to AI alignment (founder of the field _vs._ author of some dubiously relevant blog comments), I'm obviously _not_ his peer. It didn't seem like that was necessary when one could just [evaluate my arguments about dath ilan on their own merits](https://www.lesswrong.com/posts/5yFRd3cjLpm3Nd6Di/argument-screens-off-authority). But I wasn't going to be so impertinent to point that out when the master was testing me (!) and I was eager to pass the test.
935 I said that I'd like to take an hour to compose a _good_ answer. (It was 10:26 _p.m._) If I tried to type something off-the-cuff on the timescale of five minutes, it wasn't going to be of similar quality as my criticisms, because, as I had just admitted, I had _totally_ been running a biased search for criticisms—or did the fact that I had to ask that mean I had already failed the test?
939 > I mean, yeah, in fact the greater test is already having that info queued, but conversely it's even worse if you think back or reread and people are not impressed with the examples you find. I cannot for politeness lie and deny that if you did it in five minutes it would be _more_ impressive, but I think that it is yet the correct procedure to take your time.
941 (As an aside—this isn't something I thought or said at the time—I _do_ think it makes sense to run an asymmetric search for flaws in some contexts, even though it would be disastrous to only look on one side of the argument when considering a belief you're uncertain about. Code reviewers often only comment in detail on flaws or bugs that they find, and say only "LGTM" (looks good to me) when they don't find any. Why? Because the reviewers aren't necessarily trying to evaluate "This code is good" as an abstract belief[^low-stakes]; they're trying to improve the code, and there's an asymmetry in payoffs where eliminating a flaw is an improvement, whereas identifying something the code does right just means the author was doing their job. If you didn't trust a reviewer's competence and thought they were making spurious negative reviews, you might legitimately test them by asking them to argue what's _good_ about a pull request that they just negatively reviewed, but I don't think it should be concerning if they asked for some extra time.)
943 [^low-stakes]: For typical low-stakes business software in the "move fast and break things" regime. In applications where bugs are more costly, you do want to affirmatively verify "the code is good" as a belief.
945 I said that I also wanted to propose a reframing: the thing that the present thread was complaining about was a lack of valorization of truth-_telling_, honesty, wanting _other_ people to have accurate maps. Or maybe that was covered by "as you, yourself, see that virtue"?
947 Yudkowsky said that he would accept that characterization of what the thread was about if my only objection was that dath ilan didn't tell Keltham about BSDM, and that I had no objection to Keltham's judgement that in dath ilan, he would have preferred not to know.
949 I expounded for some more paragraphs about why I _did_ object to Keltham's judgement, and then started on my essay exam—running with my "truth-telling" reframing.
951 I wanted to nominate the part where the Conspiracy is unveiled—I thought I remembered Keltham saying something about how Carissa's deception was the worst thing anyone could have done to him—that is, the fact that someone he trusted was putting him in a fake reality was _itself_ considered a harm, separately from the fact that Cheliax is evil. I re-read pages 74 onwards of the ["What the Truth Can Destroy"](https://www.glowfic.com/posts/5930) thread, and didn't see Keltham saying the thing I thought he said (maybe it happened in the next thread, or I had misremembered), but found two more things to submit as answers to my lit exam, which I posted at 12:30 _a.m._ (so I had actually taken two hours rather than the one I had asked for).
953 First, I liked how [Snack Service intervenes to stage](https://www.glowfic.com/replies/1811461#reply-1811461) a "truth and reconciliation commission" for Keltham and his paramours, on the grounds that it's necessary for Asmodeus and Cayden Caliean and Adabar and Keltham to make their best decisions. People testifying in public (with the Chelaxians and Oririons present, as one would at a trial) reflects a moral about the importance of common knowledge, _shared_ maps. The testimony being public ensured that not just that Keltham got to know what's been done to him, but that his paramours and counterparties _know that he knows_. There was something honorable about getting things on the public record like that, in the end, even while Snack Service was willing to participate in the conspiracy _before_ the jig was up.
955 Second, I liked Korva's speech about why she hates Keltham, and how Keltham not only takes it in stride, but also asks to buy the right to take Korva with him to Osirion. When Abrogail expresses surprise that Keltham would want Korva, Keltham cites a dath ilani proverb about advice that's easier to get from people who aren't friends with you. This reflects an understanding that your friends wanting to be nice to you can be a source of distortions; Keltham specifically values Korva _as a critic_.
957 The next day, I added that I realized that I had missed a huge opportunity to successfully reply on a five-minute time scale (to pass "the greater test [of] already having that info queued"): the "in _Planecrash_" part of the prompt made me think I had to find something in Keltham's story (which is why I took another two hours to hand in my essay), but other threads within the dath ilan Glowfic continuity should obviously count for the purpose of the test, and I did in fact already have cached thoughts about how Thellim's contempt for Jane Austen characters beautifully mirrored my contempt for protecting people from psychology facts that would hurt their feelings. I could _prove_ that I already had it cached (if not queued, as evidenced by my remembering it the next day), because I had mentioned it both in the conversation leading to the present thread, and in my memoir draft.
961 > so I think that you're looking an awful lot at what _characters say_ and nearly not at all at what the universe does. this plausibly reflects a deep flaw in your art, because it sure does seem to me that you are a lot better at noticing what people say about truth in words, detecting whose monkey-side they seem to be on, than you are imo at carefully weighing up both sides of things as is the art of finding-truth-in-reality. it plausibly also reflects some people who ill-shaped you, pointing you at the fictional characters and angering you at their spoken words and verbal thoughts, as was advantageous to them, and not pointing you towards, like, looking at the messages in the fiction itself rather than the words spoken by characters, because that would not have served their ill purpose of alienating you and turning you into an angry thing more useful for their purposes. (I would not ordinarily use language like this but I regret that it is the language you have now seemingly been ill-shaped to speak, for another's usefulness.)
962 > if I ask you, not what any _character says_, not even what any _societies say_, but _what happens in Planecrash_ and what the _causal process_ there seems to think about matters important to you, what do you see?
964 As a _quick_ reply to the followup question (posted within 19 minutes of it being asked), I said that Cheliax was at a structural disadvantage in its conflict with the forces of Good, because learning how to think inevitably turns mortals away from Asmodeus's will.
966 But I was _more_ interested in replying to the part about me being ill-shaped to another's purpose. (I said that I wouldn't have considered that on-topic for the fiction server, but if _he_ thought it was on-topic, then it made sense for me to reply—and I did so at 12:26 _p.m._ the next day, after some time to think. Discord lends itself quite well to a mix of synchronous and asynchronous communication, depending on when people happen to be at their computers.)
968 I said that he seemed _really_ stuck on this hypothesis that it was Michael Vassar's fault that I'd been shaped into an alienated and angry thing.
970 To be clear, I totally agreed that I had been shaped into an alienated and an alienated and angry thing. Obviously. But speaking of people "look[ing] inside themselves and correctly see[ing] that this is not how they work" (as Yudkowsky had said earlier), I thought he was getting the causality all wrong.
972 It seemed to _me_ that the reason I had become an alienated and angry thing is because I had been shaped by [making an extraordinary effort](https://www.lesswrong.com/posts/GuEsfTpSDSbXFiseH/make-an-extraordinary-effort) since 2016 to respond to a class of things that included Yudkowsky "mak[ing] up sophisticated stories for why pretty obviously true things are false"—again referencing Oliver Habryka's comment on "Challenges to Yudkowsky's Pronoun Reform Proposal."
974 That's the context in which it wasn't surprising that my Art had involved some amount of specialization in "detecting whose monkey-side they seem to be on." In a world where monkeys are trying to cover up otherwise-obvious truths, successfully blowing the whistle on them involves being sensitive to their monkey games; figuring out the truth they're trying to cover up is the easy part. The whistleblowing-skill of of promoting otherwise-obvious things to _common_ knowledge in opposition to a Power trying to prevent common knowledge, is different from the science-skill of figuring out organically-nonobvious things from scratch. It _makes sense_ for Alexander Solzhenitsyn and Andrey Kolmogorov—or for that matter, John Galt and Robert Stadler—to have developed different crystalized skills.
976 (Indeed, it even makes sense for Kolmogorov and Stadler to _not_ develop some skills, because the skills would show up under Detect Thoughts.)
978 If it was all Michael's fault for "extensively meta-gas[lighting me] into believing that everyone generally and [him] personally [were] engaging in some kind of weird out-in-the-open gaslighting", I asked, echoing Yudkowsky's language from earlier (with appropriate quotation marks and brackets),[^gaslighting] then _how come Oli could see it, too?_
980 [^gaslighting]: In particular, Yudkowsky was the one who introduced the word _gaslighting_ into the conversation; I hadn't previously used the word myself.
984 > I think if you asked Oli about the state of reality with respect to this whole affair, he'd have a very different take from your take, _if you're still able to hear differences instead of only those similarities you demand._
986 That sounded like an easy enough experimental test! I wrote Habryka an email explaining the context, and asking him what "very different take" he might have, if any. (I resisted the temptation to start a [Manifold market](https://manifold.markets/) first.) As I mentioned in the email, I didn't expect to have a very different take from Habryka _about the state of reality_. ("Zack is (still?!) very upset about this, but Oli mostly doesn't care" isn't a disagreement about the state of reality.) I didn't think I disagreed with _Yudkowsky_ much about the state of reality.
988 He didn't reply. (I might have guessed the wrong email address, out of the two I had on file for him?) I don't blame him. It might have been timelessly ungrateful of me to ask. (The reason people are reluctant to make on-the-record statements in politically charged contexts is because they're afraid the statements will be _used_ to drag them into more political fights later. He had already done me a huge favor by being brave enough to state the obvious in March; I had no right to demand anything more of him.)
990 Regarding my quick reply about Cheliax's structural disadvantage, Yudkowsky said it was "okay as one element", but complained that the characters had already observed it out loud, and that I "didn't name any concrete sequence of events that bore it out or falsified it." He continued:
992 > I think you could find a lot more than this if your brain were still able to see incongruent facts as well as only congruent facts. what does the underlying reality of Planecrash think about your Most Important Issues? what is shown, but maybe never even told at all? you gave the reply of somebody who can _only_ see social realities and _only_ what people say and not what _just happens, at all_, even inside a story, you didn't say _a thing that happened._
994 At this point, I was a bit suspicious that _any_ answer that wasn't exactly whatever he was thinking of would be dismissed as too social or too inferentially close to something one of the characters had said. What did it mean for the _universe_ to say something about valorizing truth?
996 The original prompt ("What are some of the ways _Planecrash_ valorizes truth") had put me into 11th-grade English class mode; the revision "if I ask you, not what any _character_ says [...]" made me think the 11th-grade English teacher expected a different answer. The revised–revised prompt "what does the underlying reality of _Planecrash_ think about your Most Important Issues?", with the previous rebukes in my context window, made me think I should be reaching for an act of philosophical [Original Seeing](https://www.lesswrong.com/posts/SA79JMXKWke32A3hG/original-seeing), rather than trying to be a diligent schoolstudent playing the 11th-grade English class game. I pondered the matter ... and I _thought of something_.
998 _Thesis_: the universe of _Planecrash_ is saying that virtue ethics—including, as a special case, my virtue ethics about it being good to tell the truth and reveal information—are somewhat unnatural.
1000 In the story, the god Adabar values trading fairly, even with those who can't verify their partners are keeping up their end of the deal, and also wants to promote fair trading _elsewhere_ in Reality (as contrasted to just being fair Himself).
1002 Adabar is kind of a weirdo. He's not a vanishly rare freak (whose specification would require lots of uncompressible information); there _is_ a basin of attraction in the space of pre-gods, where creatures who develop a computationally efficient "fairness" heuristic in their ancestral environment, reify that into their utilityfunction when they ascend, but it's not a _huge_ basin of attraction; most gods aren't like Adabar.
1004 It's the same thing with honesty. Generic consequentialists have no reason to "tell the truth" to agents with different utility functions when they're not under compact and being compensated for the service. Why _would_ you emit signals that other agents can interpret as a map that reflects the territory? [You can't get more paperclips that way!](https://arbital.com/p/not_more_paperclips/)
1006 I had previously written about this in ["Commucation Requires Common Interests or Differential Signal Costs"](https://www.lesswrong.com/posts/ybG3WWLdxeTTL3Gpd/communication-requires-common-interests-or-differential); you needed some common interests in order for flexible, "digital" language to exist at all. ("Digital" language where the relationship between signals and meaning can be arbitrary, in contrast to costly signaling, where me expending resources at least tell you that I could afford those resources.)
1008 It's _possible_ for imperfectly deceptive social organisms to develop a taste for "honesty" as a computationally efficient heuristic for navigating to Pareto improvements in the ancestral environment, which _might_ get reified into the utilityfunction as they ascend—but that's an Adabar-class weird outcome, not the default outcome.
1010 So—insofar as my Most Important Issues revolved around an obsession with motivational transparency, wanting to live in a world that wasn't lying to me, wanting to _reveal information_ as an end in itself, unilaterally rather than only as part of a coordinated negotiation scheme, without necessarily being _paid_ for it, but just because it is _right_ ...
1012 It seems like my answer to the question of, "What does the underlying causal process of _Planecrash_ think about your Most Important Issues; what are some the ways that _Planecrash_ valorizes truth-telling as you, yourself, see that virtue?" is, "It doesn't" (!). Truth-telling is a virtue ethic, and _Planecrash_ depicts a universe ruled by consequentialist gods who only implement virtue ethics insofar as that made it into their utilityfunction.
1014 I realized, of course, that this certainly wasn't the answer Yudkowsky was looking for. But it seemed like a _better_ answer than me trying to play the schoolstudent. He asked what I saw when I looked at what the fictional universe was saying about _my_ problems, and I looked, and _I saw something_. (Something philosophically substantive, not just a social reality.) It seemed more honest to just report that, rather than try to [guess the teacher's password](https://www.lesswrong.com/posts/NMoLJuDJEms7Ku9XS/guessing-the-teacher-s-password). So, after sleeping on it first, I posted the explanation of what I saw to the channel (including the parts about how the original prompts steered me, and that I realized that this wasn't the answer he was looking for).
1017 * third bluebook didn't get a response
1018 * I was feeling kind of glum about my poor performance; I had reasons that made sense to me for why I said the things I said, but that _wasn't what I was being tested on_; I definitely _didn't_ say "Nothing, because Big Yud and Linta are lying liars who hate truth"; I moped in Alicorner #drama
1019 * Big Yud saw it (I know this because he left an emoji react in the relevant section of messages) and took pity on me
1022 [TODO: derail with Lintamande]
1024 [TODO: knives, and showing myself out]
1028 Anyway, that—briefly (I mean it)—is the Whole Dumb Story about how I wasted the last seven years of my life. It's probably not that interesting? Life goes on—for now. My dayjob contract expired at the end of 2022. In 2023, I've been finishing up this memoir, and posting some other ideas to _Less Wrong_. (I got into another slapfight about me being un-collaborative, which is not interesting enough to summarize.)
1030 After this, the AI situation is looking worrying enough, that I'm thinking I should try to do some more direct xrisk-reduction work, although I haven't definitely selected any particular job or project. (It probably won't matter, but it will be dignified.) Now that the shape of the threat is on the horizon, I think I'm less afraid of being directly involved. Something about having large language models to study in the 'twenties is—grounding, compared to the superstitious fears of the paperclip boogeyman of my nightmares in the 'teens.
1032 Like all intellectuals, as a teenager I imagined that I would write a book. It was always going to be about gender, but I was vaguely imagining a novel, which never got beyond vague imaginings. That was before the Sequences. I'm 35 years old now. I think my intellectual life has succeeded in ways I didn't know how to imagine, before. I think my past self would be proud of this blog—140,000 words of blog posts stapled together is _morally_ a book—once he got over the shock of heresy.
1034 [TODO conclusion, cont'd—
1035 * Do I have regrets about this Whole Dumb Story? A lot, surely—it's been a lot of wasted time. But it's also hard to say what I should have done differently; I could have listened to Ben more and lost faith Yudkowsky earlier, but he had earned a lot of benefit of the doubt?
1036 * even young smart AGPs who can appreciate my work have still gotten pinkpilled
1037 * Jonah had told me that my planning horizon was too short—like the future past a year wasn't real to me. (This plausibly also explains my impatience with college.) My horizon is starting to broaden as AI timelines shorten
1038 * less drama (in my youth, I would have been proud that at least this vice was a feminine trait; now, I prefer to be good even if that means being a good man)
projects / Ultimately_Untrue_Thought.git / blob