1 Title: Standing Under the Same Sky
4 Tags: autogynephilia, bullet-biting, cathartic, Eliezer Yudkowsky, Scott Alexander, epistemic horror, my robot cult, personal, sex differences, two-type taxonomy, whale metaphors
7 > "The only thing standing in the way of my own progress," Sagreda said, "is that the forces that once dealt with us honestly have been buried too deep to reach. All I can touch now is the surface, which is shaped by nothing but whim."
9 > —["Bit Players"](https://subterraneanpress.com/magazine/winter_2014/bit_players_by_greg_egan) by Greg Egan
11 Perhaps for [lack of any world-saving research to do](https://www.lesswrong.com/posts/j9Q8bRmwCgXRYAgcJ/miri-announces-new-death-with-dignity-strategy), Yudkowsky started writing fiction again, largely in the form of Glowfic (a genre of collaborative storytelling pioneered by Alicorn) featuring the world of dath ilan.
13 The bulk of the dath ilan Glowfic canon was an epic titled [_Planecrash_](https://www.glowfic.com/boards/215)[^planecrash-title] coauthored with Lintamande, in which Keltham, an unusually selfish teenage boy from dath ilan, apparently dies in a freak aviation accident, and [wakes up in the world of](https://en.wikipedia.org/wiki/Isekai) Golarion, setting of the _Dungeons-&-Dragons_–alike _Pathfinder_ role-playing game. A [couple](https://www.glowfic.com/posts/4508) of [other](https://glowfic.com/posts/6263) Glowfic stories with different coauthors further flesh out the setting of dath ilan, a smarter, better-coordinated alternate version of Earth steered by an order of [Keepers of Highly Unpleasant Things it is Sometimes Necessary to Know](https://www.glowfic.com/replies/1612937#reply-1612937) who safeguard advanced rationality techniques from a population allegedly too psychologically fragile to handle them.
15 [^planecrash-title]: The title is a triple pun, referring to the airplane crash leading to Keltham's death in dath ilan, and how his resurrection in Golarion collides dath ilan with [the "planes" of existence of the _Pathfinder_ universe](https://pathfinderwiki.com/wiki/Great_Beyond), and Keltham's threat to destroy (crash) the _Pathfinder_ reality if mortals aren't given better afterlife conditions. (I use the word "threat" colloquially here; the work itself goes into some detail distinguishing between mere bargaining and decision-theoretic threats that should be defied.)
17 On the topic of dath ilan's rationality training, I appreciated [this passage about the cognitive function of categorization](https://www.glowfic.com/replies/1779051#reply-1779051):
19 > Dath ilani kids get told to not get fascinated with the fact that, in principle, 'bounded-agents' with finite memories and finite thinking speeds, have any considerations about mapping that depend on what they want. It doesn't mean that you get to draw in whatever you like on your map, because it's what you want. It doesn't make reality be what you want.
21 Vindication! This showed that Yudkowsky _did_ understand what was at issue in the dispute over "... Not Man for the Categories", even if he couldn't say "Zack is right and Scott is wrong" for political reasons.
23 Despite that tidbit, however, the dath ilan mythos overall seemed visibly defective to me compared to the Sequences—particularly regarding how humans should relate to unpleasant truths.
25 Someone at the 2021 Event Horizon Independence Day party had told me that I had been misinterpreting the "Speak the truth, even if your voice trembles" slogan from the Sequences. I had interpreted the slogan as suggesting the importance of speaking the truth _to other people_ (which I think is what "speaking" is usually about), but my interlocutor said it was about, for example, being able to speak the truth aloud in your own bedroom, to yourself. I think some textual evidence for my interpretation can be found in Daria's ending to ["A Fable of Science and Politics"](https://www.lesswrong.com/posts/6hfGNLf4Hg5DXqJCF/a-fable-of-science-and-politics), a multiple-parallel-endings story about an underground Society divided into factions over the color of the unseen sky, and one person's reaction when they find a passageway leading aboveground to a view of the sky:
27 > Daria, once Green, tried to breathe amid the ashes of her world. _I will not flinch_, Daria told herself, _I will not look away_. She had been Green all her life, and now she must be Blue. Her friends, her family, would turn from her. _Speak the truth, even if your voice trembles_, her father had told her; but her father was dead now, and her mother would never understand. Daria stared down the calm blue gaze of the sky, trying to accept it, and finally her breathing quietened. _I was wrong_, she said to herself mournfully; _it's not so complicated, after all_. She would find new friends, and perhaps her family would forgive her ... or, she wondered with a tinge of hope, rise to this same test, standing underneath this same sky? "The sky is blue," Daria said experimentally, and nothing dire happened to her; but she couldn't bring herself to smile. Daria the Blue exhaled sadly, and went back into the world, wondering what she would say.
29 Daria takes it as a given that she needs to be open about her new blue-sky belief, even though it's socially costly to herself and to her loved ones; the rationalist wisdom from her late father did _not_ say to go consult a priest or a Keeper to check whether telling everyone about the blue sky is a good idea.[^other-endings] I think this reflects the culture of the _Overcoming Bias_ in 2006 valuing the existence of a shared social reality that reflects actual reality: the conviction that it's both possible and desirable for people to rise to the same test, standing underneath the same sky.
31 [^other-endings]: Even Eddin's ending, which portrays Eddin as more concerned with consequences than honesty, has him "trying to think of a way to prevent this information from blowing up the world", rather than trying to think of a way to suppress the information, in contrast to how Charles, in his ending, _immediately_ comes up with the idea to block off the passageway leading to the aboveground. Daria and Eddin are clearly written as "rationalists"; the deceptive strategy only comes naturally to the non-rationalist Charles. (Although you could Watsonianly argue that Eddin is just thinking longer-term than Charles: blocking off _this_ passageway and never speaking a word of it to another soul, won't prevent someone from finding some other passage to the aboveground, eventually.)
33 In contrast, the culture of dath ilan does not seem to particularly value people _standing under the same sky_. It's not just the Keepers; we also see many other cases of the dath ilani covering things up for some alleged greater good with seemingly no regard to the costs of people have less accurate world-models.
35 In one notable example, Keltham, the protagonist of _Planecrash_, is an obligate sexual sadist, but never discovered this fact about himself during his first life in dath ilan, because dath ilan has arranged to cover up the existence of sadism and masochism, because sadists like Keltham would be sad if they discovered that there weren't enough masochists to go around.
37 It did not escape my notice that when "rationalist" authorities in real life considered public knowledge of some paraphilia to be an infohazard (ostensibly for the benefit of people with that paraphilia), I _didn't take it lying down_.
39 The parallel between dath ilan's sadism/masochism coverup and the autogynephilia coverup I had fought in real life, was something I was only intending to comment on in passing in the present memoir, rather than devoting any more detailed analysis to, but as I was having trouble focusing on my own writing in September 2022, I ended up posting some critical messages about dath ilan's censorship regime in the "Eliezerfic" Discord server for reader discussion of _Planecrash_, using the sadism/masochism coverup as my central example.
41 (I would later adapt my complaints into a standalone post, "On the Public Anti-Epistemology of dath ilan".)
43 Although Yudkowsky participated in the server, I had reasoned that my participation didn't violate my previous intent not to bother him anymore, because it was a publicly-linked Discord server with hundreds of members. Me commenting on the story for the benefit of the _other_ 499 people in the chatroom wouldn't generate a notification _for him_, the way it would if I sent him an email or replied to him on Twitter.
45 The other chatroom participants mostly weren't buying what I was selling.
47 When I objected to [Word of God](https://tvtropes.org/pmwiki/pmwiki.php/Main/WordOfGod)'s identification of the Keeper's credo as "Let the truth destroy what it can—in yourself, not in other people" as an incredibly infantalizing philosophy, someone replied, "I think of 'not in other people' not as 'infantilizing', but as recognizing independent agency. You don't get to do harm to other people without their consent, whether that is physical or pychological."
49 I pointed out that this obviously applies to, say, religion. Was it wrong to advocate for atheism in a religious Society, where robbing someone of their belief in God might be harming them?
51 "Every society strikes a balance between protectionism and liberty," someone said. "This isn't news."
53 It's not news about _humans_, I conceded. But I thought people who were fans of Yudkowsky's writing in 2008 had a reasonable expectation that the dominant messaging in the local subculture would continue in 2022 to be _in favor_ of telling the truth and _against_ benevolently intended noble lies. It would be interesting to know why that had changed.
55 I started a new thread for my topic (Subject: "Noble Secrets; Or, Conflict Theory of Optimization on Shared Maps"). It died out after a couple days, but I reopened it later in response to more discussion of the masochism coverup.
57 Yudkowsky made an appearance. (After he replied to someone else, I remarked parenthetically that his appearance made me think I should stop wasting time snarking in his fiction server and just finish my memoir already.) We had a brief back-and-forth:
59 > **Eliezer** — 11/29/2022 10:33 PM
60 > the main thing I'd observe contrary to Zack's take here, is that Keltham thought that not learning about masochists he can never have, was obviously in retrospect what he'd have wanted Civilization to do, or do unless and until Keltham became rich enough to afford a masochist and then he could be told
61 > in other words, Keltham thought he was obviously being treated the way that counterfactual fully-informed Keltham would have paid Governance to treat not-yet-informed Keltham
62 > that this obeys the social contract that Keltham thought he had, is part of why Keltham is confident that the logic of this particular explanation holds together
63 > **zackmdavis** — 11/29/2022 10:35 PM
64 > the level of service that Keltham is expecting is _not the thing I learned from Robin Hanson's blog in 2008_
65 > **Eliezer** — 11/29/2022 10:36 PM
66 > I am sorry that some of the insane people I attracted got together and made each other more insane and then extensively meta-gaslit you into believing that everyone generally and me personally was engaging in some kind of weird out-in-the-open gaslighting that you could believe in if you attached least-charitable explanations to everything we were doing
68 It was pretty annoying that Yudkowsky was still attributing my greviances to Michael's malign influence—as if [the rationalists' wholesale embrace of gender identity revolution](/2023/Jul/blanchards-dangerous-idea-and-the-plight-of-the-lucid-crossdreamer/) was something I would otherwise have just taken lying down. In the counterfactual where Michael had died in 2015, I think something like [my February 2017 mental breakdown](/2017/Mar/fresh-princess/) would have likely happened anyway.[^vassar-influence]
70 [^vassar-influence]: Between August 2016 and January 2017, I sent Michael 14 emails, met with him once, and watched 60% of South Park season 19 at his suggestion, so he was _an_ influence on my thinking during that period, but not a disproportionately large one compared to everything else I was doing at the time.
72 How would I have later reacted to [Yudkowsky's November 2018 "hill of meaning in defense of validity" Tweets](/2023/Jul/a-hill-of-validity-in-defense-of-meaning/) (assuming they weren't butterfly-effected away in this counterfactual)? It's hard to say. Maybe, if that world's analogue of my February 2017 breakdown had gone sufficiently badly (with no Michael to visit me in the psych ward or help me make sense of things afterwards), I would have already been a broken man, and not even sent Yudkowsky an email. In any case, I feel very confident that my understanding of the behavior of "everyone generally and [Yudkowsky] personally" would not have been _better_ without Michael _et al._'s influence.
74 Also, while "weird out-in-the-open gaslighting" was clearly a hostile paraphrase optimized to make my position look silly, I'm tempted to bite the bullet on it being an apt description of repeatedly publishing text that's optimized to delegitimize thinking about biological sex (which is gaslighting, manipulating people such that they doubt their own perceptions of reality) and justifying it on the grounds that ["people do _know_ they're living in a half-Stalinist environment"](/images/yudkowsky-personally_prudent_and_not_community-harmful.png) (which makes it weird and out in the open).
77 > you may recall that this blog included something called the "Bayesian Conspiracy"
78 > they won't tell you about it, because it interferes with the story they were trying to drive you insaner with, but it's so
79 > **zackmdavis** — 11/29/2022 10:37 PM
80 > it's true that the things I don't like about modern Yudkowsky were still there in Sequences-era Yudkowsky, but I think they've gotten _worse_
81 > **Eliezer** — 11/29/2022 10:39 PM
82 > well, if your story is that I was always a complicated person, and you selected some of my posts and liked the simpler message you extracted from those, and over time I've shifted in my emphases in a way you don't like, while still having posts like Meta-Honesty and so on... then that's a pretty different story than the one you were telling in this Discord channel, like, just now. today.
84 Is it, though? The "always a complicated person [who has] shifted in [his] emphases in a way [I] don't like" story was true, of course, but it elided the substantive reasons _why_ I didn't like the new emphases, which I expect other people to be able to see, too.
86 (As far as [the](https://www.lesswrong.com/posts/fnEWQAYxcRnaYBqaZ/initiation-ceremony) [Bayesian](https://www.lesswrong.com/posts/ZxR8P8hBFQ9kC8wMy/the-failures-of-eld-science) [Conspiracy](https://www.lesswrong.com/posts/xAXrEpF5FYjwqKMfZ/class-project) [stories](https://www.lesswrong.com/posts/kXAb5riiaJNrfR8v8/the-ritual) [went](https://www.lesswrong.com/posts/yffPyiu7hRLyc7r23/final-words), I think there's a significant narrative contrast between Brennan _seeking_ knowledge from the master _beisutsukai_, and Keltham, Merrin, and Thellim being _protected from_ knowledge by the Keepers. Neither the Bayesian Conspiracy nor the Keepers are publishing open-access textbooks, but at least the Conspiracy isn't claiming that their secretiveness is _for others' benefit_.)
88 It's notable that Yudkowsky listed "still having posts like [Meta-Honesty](https://www.lesswrong.com/posts/xdwbX9pFEr7Pomaxv/meta-honesty-firming-up-honesty-around-its-edge-cases)" as an exculpatory factor here. The thing is, I [wrote a _critique_ of "Meta-Honesty"](https://www.lesswrong.com/posts/MN4NRkMw7ggt9587K/firming-up-not-lying-around-its-edge-cases-is-less-broadly). It was well-received (being [cited as a good example in the introductory post for the 2019 Less Wrong Review](https://www.lesswrong.com/posts/QFBEjjAvT6KbaA3dY/the-lesswrong-2019-review), for instance). I don't think I could have written a similarly impassioned critique of anything from the Sequences era, because the stuff from the Sequences era still looked correct to me. To me, "Meta-Honesty" was evidence _for_ Yudkowsky having relinquished his Art and lost his powers, not evidence that his powers were still intact.
90 I didn't have that response thought through in real time. At the time, I just agreed:
92 > **zackmdavis** — 11/29/2022 10:39 PM
94 > I'm not done working through the hate-warp
96 (This being a reference to part of _Planecrash_ in which [Keltham tells Carissa to be aware of her un-dath ilani tendency to feel "hatred that warps reality to be more hateable"](https://www.glowfic.com/replies/1882822#reply-1882822).)
98 > **Eliezer** — 11/29/2022 10:40 PM
99 > so one thing hasn't changed: the message that you, yourself, should always be trying to infer the true truth, off the information you already have.
100 > if you know you've got a hate-warp I don't know why you're running it and not trying to correct for it
101 > are you in fact also explicitly aware that the people who talk to you a lot about "gaslighting" are, like, insane?
102 > **zackmdavis** — 11/29/2022 10:42 PM
103 > I'm not really part of Vassar's clique anymore, if that's what you mean
104 > **Eliezer** — 11/29/2022 10:44 PM
105 > it looks from outside here like they stomped really heavy footprints all over your brain that have not healed or been filled in
106 > **zackmdavis** — 11/29/2022 10:49 PM
107 > it looks from inside here that the thing I'm not healed from is the thing where, as Oliver Habryka put it, I "should expect that depending on the circumstances community leaders might make up sophisticated stories for why pretty obviously true things are false" ([https://www.lesswrong.com/posts/juZ8ugdNqMrbX7x2J/challenges-to-yudkowsky-s-pronoun-reform-proposal?commentId=he8dztSuBBuxNRMSY](https://www.lesswrong.com/posts/juZ8ugdNqMrbX7x2J/challenges-to-yudkowsky-s-pronoun-reform-proposal?commentId=he8dztSuBBuxNRMSY)), and Michael and Ben and Jessica were _really_ helpful for orienting me to that particular problem, even if I disagree with them about a lot of other things and they seem crazy in other ways
108 > (rule thinkers in, not out)
110 (I was pleased to get the link to Habryka's comment in front of Yudkowsky, if he hadn't already seen it.)
112 > **Eliezer** — 11/29/2022 10:55 PM
113 > the most harm they did you was to teach you to see malice where you should have seen mortality
114 > noninnocent error is meaningfully different from innocent error; and noninnocent error is meaningfully different from malice
115 > Keltham deduced the lack of masochists in dath ilan by asking the question, "Why would Civilization have kept this information from me?", _ruling out_ or actually not even thinking of such ridiculous hypotheses as "Because it was fun", and settling on the obvious explanation that explained _why Keltham would have wanted Civilization to do that for him_—masochists not existing or being incredibly rare and unaffordable to him. You looked at this and saw malice everywhere; you couldn't even see _the fictional world_ the author was trying to give you _direct description about_. You didn't say that you disbelieved in the world; you could not see what was being _described_.
117 (When a literary critic proposes a "dark" interpretation of an author's world, I think it's implied that they're expressing disbelief in the "intended" world; the fact that I was impudently refusing to buy the benevolent interpretation wasn't because I didn't understand it.)
119 > Hate-warp like this is bad for truth-perception; my understanding of the situation is that it's harm done to you by the group you say you left. I would read this as being a noninnocent error of that group; that they couldn't get what they wanted from people who still had friends outside their own small microculture, and noninnocently then decided that this outer culture was bad and people needed to be pried loose from it. They tried telling some people that this outer culture was gaslighting them and maliciously lying to them and had to be understood in wholly adversarial terms to break free of the gaslighting; that worked on somebody, and made a new friend for them; so their brain noninnocently learned that it ought to use arguments like that again, so they must be true.
120 > This is a sort of thing I super did not do because I _understood_ it as a failure mode and Laid My Go Stones Against Ever Actually Being A Cult; I armed people with weapons against it, or tried to, but I was optimistic in my hopes about how much could actually be taught.
121 > **zackmdavis** — 11/29/2022 11:20 PM
122 > Without particularly defending Vassar _et al._ or my bad literary criticism (sorry), _modeling the adversarial component of non-innocent errors_ (as contrasted to "had to be understood in wholly adversarial terms") seems very important. (Maybe lying is "worse" than rationalizing, but if you can't hold people culpable for rationalization, you end up with a world that's bad for broadly the same reasons that a world full of liars is bad: we can't steer the world to good states if everyone's map is full of falsehoods that locally benefitted someone.)
123 > **Eliezer** — 11/29/2022 11:22 PM
124 > Rationalization sure is a huge thing! That's why I considered important to discourse upon the science of it, as was then known; and to warn people that there were more complicated tangles than that, which no simple experiment had shown yet.
125 > **zackmdavis** — 11/29/2022 11:22 PM
127 > **Eliezer** — 11/29/2022 11:23 PM
128 > It remains something that mortals do, and if you cut off anybody who's ever done that, you'll be left with nobody. And also importantly, people making noninnocent errors, if you accuse them of malice, will look inside themselves and correctly see that this is not how they work, and they'll stop listening to the (motivated) lies you're telling them about themselves.
129 > This also holds true if you make up overly simplistic stories about 'ah yes well you're doing that because you're part of $woke-concept-of-society' etc.
130 > **zackmdavis** — 11/29/2022 11:24 PM
131 > I think there's _also_ a frequent problem where you try to accuse people of non-innocent errors, and they motivatedly interpret _you_ as accusing malice
132 > **Eliezer** — 11/29/2022 11:25 PM
133 > Then invent new terminology. I do that all the time when existing terminology fails me.
134 > Like I literally invented the term 'noninnocent error' right in this conversation.
135 > **zackmdavis** — 11/29/2022 11:27 PM
136 > I've tried this, but maybe it wasn't good enough, or I haven't been using it consistently enough: [https://www.lesswrong.com/posts/sXHQ9R5tahiaXEZhR/algorithmic-intent-a-hansonian-generalized-anti-zombie](https://www.lesswrong.com/posts/sXHQ9R5tahiaXEZhR/algorithmic-intent-a-hansonian-generalized-anti-zombie)
137 > I should get ready for bed
138 > I will endeavor to edit out the hate-warp from my memoir before publishing, and _probably_ not talk in this server
139 > **Eliezer** — 11/29/2022 11:31 PM
140 > I think you should worry first about editing the hate-warp out of yourself, but editing the memoir might be useful practice for it. Good night.
142 It turned out that I was lying about probably not talking in the server anymore. (Hedging with the word "probably" didn't make the claim true, and of course I wasn't _consciously_ lying, but that hardly seems exculpatory.) The fact that Yudkowsky had been replying to me at length—explaining why my literary criticism was nuts, but in a way that respected my humanity and expected me to be able to hear it—implied that I was apparently in his "I can cheaply save him from crazy people like Michael Vassar" bucket, rather than the "AI timelines and therefore life is too short" bucket. Maybe my presence wasn't tresspassing; maybe I had a right to speak.
144 The next day, I belatedly pointed out that "Keltham thought that not learning about masochists he can never have, was obviously in retrospect what he'd have wanted Civilization to do" seemed to contradict "one thing hasn't changed: the message that you, yourself, should always be trying to infer the true truth". In the first statement, it didn't sound like Keltham thinks it's good that Civilization didn't tell him so that he could figure it out for himself (in accordance with the discipline of "you, yourself, always trying to infer the truth"). It sounded like he was better off not knowing—better off having a _less accurate self-model_ (not having the concept of "obligate romantic sadism"), better off having a _less accurate world-model_ (thinking that masochism isn't real).
146 In response to someone positing that dath ilani were choosing to be happier but less accurate predictors, I said that I read a blog post once about why you actually didn't want to do that, linking to [an Internet Archive copy of "Doublethink (Choosing to Be Biased)"](https://web.archive.org/web/20080216204229/https://www.overcomingbias.com/2007/09/doublethink-cho.html) from 2008[^hanson-conceit]—at least, that was _my_ attempted paraphrase; it was possible that I'd extracted a simpler message from it than the author intended.
148 [^hanson-conceit]: I was enjoying the conceit of referring to Sequences posts as being from "Robin Hanson's blog in 2008", as a way of emphasizing the distinction between my respect for the material and my contempt for the man Yudkowsky had become.
150 A user called Harmless explained the loophole. "Doublethink" was pointing out that decisions that optimize the world for your preferences can't come from nowhere: if you avoid painful thoughts in your map, you damage your ability to steer away from painful outcomes in the territory. However, there was no rule that all the information-processing going into decisions that optimize the world for your preferences had to take place in _your brain_ ...
152 I saw where they were going and completed the thought: you could build a Friendly AI or a Civilization to see all the dirty things for you, that would make you unhappy to have to see yourself.
154 Yudkowsky clarified his position:
156 > My exact word choices often do matter: I said that you should always be trying to _infer_ the truth. With the info you already have. In dath ilan if not in Earth, you might decline to open a box labeled "this info will make you permanently dissatisfied with sex" if the box was labeled by a prediction market.
157 > Trying to avoid inferences seems to me much more internally costly than declining to click on a spoiler box.
159 I understood the theory, but I was still extremely skpetical of the practice, assuming the eliezera were even remotely human. Yudkowsky described the practice of "keeping BDSM secret and trying to prevent most sadists from discovering what they are—informing them only when and if they become rich enough or famous enough that they'd have a high probability of successfully obtaining a very rare masochist" as a "basically reasonable policy option that [he] might vote for, not to help the poor dear other people, but to help [his] own counterfactual self."
161 I wasn't sure what my wordcount and "diplomacy" "budget limits" for the server were, but I couldn't let go; I kept the thread going on subsequent days. There was something I felt I should be able to convey, if I could just find the right words.
163 When [Word of God](https://tvtropes.org/pmwiki/pmwiki.php/Main/WordOfGod) says, "trying to prevent most [_X_] from discovering what they are [...] continues to strike me as a basically reasonable policy option", then, separately from the particular value of _X_, I expected people to jump out of their chairs and say, "No! This is wrong! Morally wrong! People can stand what is true about themselves, because they are already doing so!"
165 And to the extent that I was the only person jumping out of my chair, and there was a party-line response of the form, "Ah, but if it's been decreed by authorial fiat that these-and-such probabilities and utilities take such-and-these values, then in this case, self-knowledge is actually bad under the utilitarian calculus," I wasn't disputing the utilitarian calculus. I was wondering—here I used the "🐛" bug emoji customarily used in Glowfic culture to indicate uncertainty about the right words to use—_who destroyed your souls?_
169 > it feels powerfully relevant to me that the people of whom I am saying this _are eliezera_. I get to decide what they'd want because, unlike with Earth humans, I get to put myself in their shoes. it's plausible to me that the prediction markets say that I'd be sadder if I was exposed to the concept of sadism in a world with no masochists. if so, while I wouldn't relinquish my Art and lose my powers by trying to delude myself about that once I'd been told, I'd consider it a friendly act to keep the info from me—_because_ I have less self-delusional defenses than a standard Earthling, really—and a hostile act to tell me; and if you are telling me I don't get to make that decision for myself because it's evil, and if you go around shouting it from the street corners in dath ilan, then yeah I think most cities don't let you in.
171 I wish I had thought to ask if he'd have felt the same way in 2008.
173 A user called Ajvermillion continued to be baffled at my skepticism: if the author specifies that the world of the story is simple in this-and-such direction, on what grounds could I _disagree_?
175 I admitted, again, that there was a sense in which I couldn't argue with authorial fiat. But I thought that an author's choice of assumptions reveals something about what they think is true in our world, and commenting on that should be fair game for literary critics. Suppose someone wrote a story and said, "in the world portrayed in this story, everyone is super-great at _kung fu_, and they could beat up everyone from our Earth, but they never have to practice at all."
177 (Yudkowsky retorted, "...you realize you're describing like half the alien planets in comic books? when did Superman ever get depicted as studying kung fu?" I wish I had thought to admit that, yes, I _did_ hold Eliezer Yudkowsky to a higher standard of consilient worldbuilding than DC Comics. Would he rather I _didn't_?)
179 Something about innate _kung fu_ world seems fake in a way that seems like a literary flaw. It's not just about plausibility. Fiction often incorporates unrealistic elements in order to tell a story that has relevance to real human lives. Innate _kung fu_ skills are scientifically plausible[^instinct] in a way that faster-than-light travel is not, but throwing faster-than-light travel into the universe so that you can do a [space opera](https://tvtropes.org/pmwiki/pmwiki.php/Main/SpaceOpera) doesn't make the _people_ fake in the way that Superman's fighting skills are fake.
181 [^instinct]: All sorts of other instinctual behaviors exist in animals; I don't see why skills humans have to study for years as a "martial art" couldn't be coded into the genome.
183 Maybe it was okay for Superman's fighting skills to be fake from a literary perspective (because realism along that dimension is not what Superman is _about_), but if the Yudkowskian ethos exulted intelligence as ["the power that cannot be removed without removing you"](https://www.lesswrong.com/posts/SXK87NgEPszhWkvQm/mundane-magic), readers had grounds to demand that the dath ilani's thinking skills be real, and a world that's claimed by authorial fiat to be super-great at epistemic rationality, but where the people don't have a will-to-truth stronger than their will-to-happiness, felt fake to me. I couldn't _prove_ that it was fake. I agreed with Harmless's case that, technically, as far as the Law went, you could build a Civilization or a Friendly AI to see all the ugly things that you preferred not to see.
185 But if you could—would you? And more importantly, if you would—could you?
187 It was possible that the attitude I was evincing here was just a difference between the eliezera out of dath ilan and the Zackistani from my medianworld, and that there was nothing more to be said about it. But I didn't think the thing was a _genetic_ trait of the Zackistani! _I_ got it from spending my early twenties obsessively re-reading blog posts that said things like, ["I believe that it is right and proper for me, as a human being, to have an interest in the future [...] One of those interests is the human pursuit of truth [...] I wish to strengthen that pursuit further, in this generation."](https://www.lesswrong.com/posts/anCubLdggTWjnEvBS/your-rationality-is-my-business)
189 There were definitely communities on Earth where I wasn't allowed in because of my tendency to shout things from street corners, and I respected those people's right to have a safe space for themselves. But those communities didn't call themselves _rationalists_, weren't pretending be to be inheritors of the great tradition of E. T. Jaynes and Richard Feynman and Robin Hanson. And if they _did_, I think I would have a false advertising complaint against them.
191 "[The eleventh virtue is scholarship. Study many sciences and absorb their power as your own](https://www.yudkowsky.net/rational/virtues) ... unless a prediction market says that would make you less happy," just didn't have the same ring to it. Neither did "The first virtue is curiosity. A burning itch to know is higher than a solemn vow to pursue truth. But higher than both of those, is trusting your Society's institutions to tell you which kinds of knowledge will make you happy"—even if you stipulated by authorial fiat that your Society's institutions are super-competent, such that they're probably right about the happiness thing.
193 There's a scene in _Atlas Shrugged_ that I think illustrates [the mood dath ilan is missing](https://www.econlib.org/archives/2016/01/the_invisible_t.html).[^atlas-shrugged-ref] Our heroine Dagny expresses a wish to be kept ignorant for the sake of her own happiness. "Oh, if only I didn't have to hear about it!" she says. "If only I could stay here and never know what they're doing to the railroad, and never learn when it goes!"
195 John Galt isn't having it. "You'll have to hear about it," he says, in "that ruthless tone, peculiarly his, which sounded implacable by being simple, devoid of any emotional value, save the quality of respect for facts", listing the disasters fated to befall the railroad before concluding, "Nobody stays here by faking reality in any manner whatever."
197 "Thank you. You're right," Dagny says, noting the contrast of Galt's tough-minded honesty with "the world's code that worshipped white lies as an act of mercy".
199 [^atlas-shrugged-ref]: In Part Three, Chapter II, "The Utopia of Greed".
201 When I quoted this scene in the server, other participants were baffled that I seemed to be appealing to Ayn Rand's authority. I was actually going for a _reverse_ appeal-to-authority: if _Ayn Rand_ understood that facing reality is virtuous, why didn't the 2020s "rationalists"? I didn't think the disdain for "Earth people" was justified, when Earth's philosophy of rationality (as exemplified by Ayn Rand) was doing better than dath ilan's on this critical dimension.
203 I wondered if the issue had to do with what Yudkowsky had [identified as the problem of non-absolute rules](https://www.lesswrong.com/posts/xdwbX9pFEr7Pomaxv/meta-honesty-firming-up-honesty-around-its-edge-cases#5__Counterargument__The_problem_of_non_absolute_rules_), where not-literally-absolute rules like "Don't kill" or "Don't lie" have to be stated as if they were absolutes in order to register to the human motivational system with sufficient force.
205 Technically, as a matter of decision theory, "sacred values" are crazy. It's easy to say—and feel with the passion of religious conviction—that it's always right to choose Truth and Life, and that no one could choose otherwise except wrongly, in the vile service of Falsehood and Death. But reality presents us with quantitative choices over uncertain outcomes, in which everything trades off against everything else under the [von Neumann–Morgenstern axioms](https://en.wikipedia.org/wiki/Von_Neumann%E2%80%93Morgenstern_utility_theorem); if you had to choose between the Life of millions and a small, unimportant Truth, you'd probably choose Life—but more importantly, the very fact that you might have to choose means that Truth and Life can't both be infinitely sacred to you, and must be measured on a common scale with lesser goods like mere Happiness.
207 I knew that. The other people in the chatroom knew that. So to the extent that the argument amounted to me saying "Don't lie" (about, _e.g._, the existence of masochism), and them saying "Don't lie unless the badness of lying is outweighed by the goodness of increased happiness", why was I so confident that I was in the right, when they were wisely acknowledging the trade-offs under the Law, and I was sticking to my (incoherent) sacred value of Truth? Didn't they obviously have the more sophisticated side of the argument?
209 The problem was that, in my view, the people who weren't talking about Truth as if it were a sacred value were being _wildly recklessly casual_ about harms from covering things up, as if they didn't see the [non-first-order](https://www.lesswrong.com/posts/wyyfFfaRar2jEdeQK/entangled-truths-contagious-lies) [harms](https://www.lesswrong.com/posts/XTWkjCJScy2GFAgDt/dark-side-epistemology) _at all_. I felt I had to appeal to the lessons for children about how Lying Is Bad, because if I tried to make a more sophisticated argument about it being _quantitatively_ crazy to cover up psychology facts that make people sad, I would face a brick wall of "authorial fiat declares that the probabilities and utilities are specifically fine-tuned such that ignorance is good."
211 A user called RationalMoron asked if I was appealing to a terminal value. Did I think people should have accurate self-models even if they didn't want to?
213 Obviously I wasn't going to use a universal quantifier over all possible worlds and all possible minds, but in human practice, yes: people who prefer to believe lies about themselves are doing the wrong thing; people who lie to their friends to keep them happy are doing the wrong thing. People can stand what is true, because they are already doing so. I realized that this was a children's lesson without very advanced math, but I thought it was a better lesson than, "Ah, but what if a _prediction market_ says they can't???"
215 I maintained that the fact that the eliezera prefer not to know that there are desirable sexual experiences that they can't have, contradicted April's earlier claim (which had received a Word of God checkmark-emoji) that "it's not that the standards are being dropped[;] it's that there's an even higher standard far beyond what anyone on earth has accomplished".
217 Apparently I struck a nerve. Yudkowsky started "punching back":
219 > **Eliezer** — 12/08/2022 12:45 PM
220 > Do zacki have no concept of movie spoilers, such that all movies are just designed not to rely on uncertainty for dramatic tension? Do children have to be locked in individual test rooms because they can't comprehend the concept of refusing to look at other children's answer sheets because it's evidence and you should observe it? Do adults refuse to isolate the children so they can have practice problems, because you can't stop them from learning the answer to skill-building problems, only the legendary evil alien eliezera would do that? Obviously they don't have surprise parties.
221 > It's noticeably more extreme than the _Invention of Lying_ aliens, who can still have nudity taboos
222 > I'd also note that I think in retrospect (only after having typed it) that Zack could not have generated these examples of other places where society refrains from observation, and that I think this means I am tracking the thing Zack fears in a way that Zack cannot because his thinking is distorted and he is arguing rather than seeing; and this, not verbally advocating for "truth", is more what respect for truth really is.
224 I thought the "you could not have generated the answer I just told you" gambit was a pretty dirty argumentative trick on Yudkowsky's part. (Given that I could, how would I be able to prove it? This was itself a good use-case for concealing spoilers.)
226 As it happened, however, I had already considered the case of spoilers as a class of legitimate infohazards, and was prepared to testify that I had already thought of it, and explain why I thought hiding spoilers were relevantly morally different from the coverups I was objecting to: the previous night, 7 December 2022, I had had a phone call with Anna Salamon,[^evidence-of-independent-generation] in which I had cited dath ilan's [practice of letting children figure out heliocentrism for themselves](https://www.glowfic.com/replies/1777588#reply-1777588) as not being objectionable in the way the sadism/masochism coverup was.
228 [^evidence-of-independent-generation]: I counted myself lucky to be able to point to Anna as a potential witness to defend myself against the "could not have generated" trick—as a matter of principle, not because I seriously expected anyone to care enough to go ask Anna if she remembered the conversation the same way.
230 I also mentioned that when I had used spoiler blocks on the _Atlas Shrugged_ quote I had posted upthread, I had briefly considered making some kind of side-remark noting that the spoiler blocks were also a form of information-hiding, but couldn't think of anything funny or relevant enough (which, if my self-report could be trusted, showed that I had independently generated the idea of spoilers being an example of hiding information—but I didn't expect other people to uncritically believe my self-reports).
232 As for the claim that my thinking was distorted and I was arguing instead of seeing, it was definitely true that I was _motivated to look for_ criticisms of Yudkowsky and dath ilan, for personal reasons outside the scope of the server, and I thought it was great for people to notice this and take it into account. I hoped to nevertheless be competent to only report real criticisms and not fake criticisms. (Whether I succeeded, of course, was up to the reader to decide.)
236 > only half the battle even if you could do it. you're also not reporting any facts/arguments on the other side, which is a much larger and visible gap to me, and has a lot to do with why I'm not presently considering this criticism from a peer despite your spoken adherence to virtues I value. **QUESTION FOR ZACK ONLY, NOBODY ELSE ANSWER OR SAY ANYTHING ABOUT IT IN THIS MAIN CHANNEL:** What are some of the ways that Planecrash valorizes truth, as you, yourself, see that virtue?
238 I chose not to ask why it was relevant whether or not I was a "peer." If we're measuring IQ[^iq-comparison] or fiction-writing ability[^fiction-comparison] or contributions to AI alignment[^alignment-comparison], I'm obviously _not_ his peer. It didn't seem like that was necessary when one could just [evaluate my arguments about dath ilan on their own merits](https://www.lesswrong.com/posts/5yFRd3cjLpm3Nd6Di/argument-screens-off-authority). (Also, you'd think that people who are obsessed with prediction markets—whose trustworthiness does not stem from individual traders having unimpeachably perfect cognition—would be able to appreciate the value of the somewhat older [social technology of "debate"](https://www.lesswrong.com/posts/SX6wQEdGfzz7GKYvp/rationalist-discourse-is-like-physicist-motors).) But I wasn't going to be so impertinent to point that out when the master was testing me (!) and I was eager to pass the test.
240 [^iq-comparison]: [143](https://twitter.com/ESYudkowsky/status/1076154502326013952) _vs._ [131](/images/wisc-iii_result.jpg).
242 [^fiction-comparison]: Several [highly-acclaimed](https://www.lesswrong.com/posts/HawFh7RvDM4RyoJ2d/three-worlds-collide-0-8) [stories](https://www.yudkowsky.net/other/fiction/the-sword-of-good) [including the world's most popular _Harry Potter_ fanfiction](https://www.hpmor.com/) _vs._ a [few](/2018/Jan/blame-me-for-trying/) [blog](http://zackmdavis.net/blog/2016/05/living-well-is-the-best-revenge/) [vignettes](https://www.lesswrong.com/posts/dYspinGtiba5oDCcv/feature-selection) and a [_My Life as a Teenage Robot_ fanfic](https://archive.ph/WdydM) with double-digit Favorites on _fanfiction.net_.
244 [^alignment-comparison]: Founder of the field _vs._ author of some questionably relevant blog comments.
246 I said that I'd like to take an hour to compose a _good_ answer. (It was 10:26 _p.m._) If I tried to type something off-the-cuff on the timescale of five minutes, it wasn't going to be of similar quality as my criticisms, because, as I had just admitted, I had _totally_ been running a biased search for criticisms—or did the fact that I had to ask that mean I had already failed the test?
250 > I mean, yeah, in fact the greater test is already having that info queued, but conversely it's even worse if you think back or reread and people are not impressed with the examples you find. I cannot for politeness lie and deny that if you did it in five minutes it would be _more_ impressive, but I think that it is yet the correct procedure to take your time.
252 (As an aside—this isn't something I thought or said at the time—I _do_ think it makes sense to run an asymmetric search for flaws in some contexts, even though it would be disastrous to only look on one side of the argument when considering a belief you're uncertain about. Code reviewers often only comment in detail on flaws or bugs that they find, and say only "LGTM" (looks good to me) when they don't find any. Why? Because the reviewers aren't necessarily trying to evaluate "This code is good" as an abstract belief[^low-stakes]; they're trying to improve the code, and there's an asymmetry in payoffs where eliminating a flaw is an improvement, whereas identifying something the code does right just means the author was doing their job. If you didn't trust a reviewer's competence and thought they were making spurious negative reviews, you might legitimately test them by asking them to argue what's _good_ about a pull request that they just negatively reviewed, but I don't think it should be concerning if they asked for some extra time.)
254 [^low-stakes]: For typical low-stakes business software in the "move fast and break things" regime. In applications where bugs are more costly, you do want to affirmatively verify "the code is good" as a belief.
256 I said that I also wanted to propose a reframing: the thing that the present thread was complaining about was a lack of valorization of truth-_telling_, honesty, wanting _other_ people to have accurate maps. Or maybe that was covered by "as you, yourself, see that virtue"?
258 Yudkowsky said that he would accept that characterization of what the thread was about if my only objection was that dath ilan didn't tell Keltham about BSDM, and that I had no objection to Keltham's judgement that in dath ilan, he would have preferred not to know.
260 I expounded for some more paragraphs about why I _did_ object to Keltham's judgement, and then started on my essay exam—running with my "truth-telling" reframing.
262 I wanted to nominate the part where the Project Lawful Conspiracy is unveiled: I thought I remembered Keltham saying something about how Carissa's deception was the worst thing anyone could have done to him—that is, that the fact that someone he trusted was putting him in a fake reality was _itself_ considered a harm, separately from the fact that Cheliax is evil. I re-read pages 74 onwards of the ["What the Truth Can Destroy"](https://www.glowfic.com/posts/5930) thread, and didn't see Keltham saying the thing I thought he said (maybe it happened in the next thread, or I had misremembered), but found two more things to submit as answers to my lit exam, which I posted at 12:30 _a.m._ (so I had actually taken two hours, rather than the one I had asked for).
264 First, I liked how [Snack Service intervenes to stage](https://www.glowfic.com/replies/1811461#reply-1811461) a ["truth and reconciliation commission"](https://en.wikipedia.org/wiki/Truth_and_Reconciliation_Commission_(South_Africa)) for Keltham and his paramours, on the grounds that it's necessary for Asmodeus and Cayden Caliean and Adabar and Keltham to make their best decisions. People testifying in public (with the Chelaxians and Oririons present, as one would at a trial) reflects a moral about the importance of common knowledge, _shared_ maps. The testimony being public ensured that not just that Keltham got to know what had been done to him, but that his paramours and counterparties would _know that he knows_. There was something honorable about getting things on the public record like that, in the end, even while Snack Service was willing to participate in the conspiracy _before_ the jig was up.
266 Second, I liked Korva's speech about why she hates Keltham, and how Keltham not only takes it in stride, but also asks to buy the right to take Korva with him to Osirion. When Abrogail expresses surprise that Keltham would want Korva, Keltham cites a dath ilani proverb about advice that's easier to get from people who aren't friends with you. This reflects an understanding that your friends wanting to be nice to you can be a source of distortions; Keltham specifically values Korva _as a critic_.
268 The next day, I added that I realized that I had missed a huge opportunity to successfully reply on a five-minute time scale (to pass "the greater test [of] already having that info queued"): the "in _Planecrash_" part of the prompt made me think I had to find something in Keltham's story (which is why I took another two hours to hand in my essay), but other threads within the dath ilan Glowfic continuity should obviously count for the purpose of the test, and I did in fact already have cached thoughts about how Thellim's contempt for Jane Austen characters beautifully mirrored my contempt for protecting people from psychology facts that would hurt their feelings. I could _prove_ that I already had it cached (if not queued, as evidenced by my remembering it the next day), because I had mentioned it both in the conversation leading to the present thread, and in my memoir draft.
272 > so I think that you're looking an awful lot at what _characters say_ and nearly not at all at what the universe does. this plausibly reflects a deep flaw in your art, because it sure does seem to me that you are a lot better at noticing what people say about truth in words, detecting whose monkey-side they seem to be on, than you are imo at carefully weighing up both sides of things as is the art of finding-truth-in-reality. it plausibly also reflects some people who ill-shaped you, pointing you at the fictional characters and angering you at their spoken words and verbal thoughts, as was advantageous to them, and not pointing you towards, like, looking at the messages in the fiction itself rather than the words spoken by characters, because that would not have served their ill purpose of alienating you and turning you into an angry thing more useful for their purposes. (I would not ordinarily use language like this but I regret that it is the language you have now seemingly been ill-shaped to speak, for another's usefulness.)
273 > if I ask you, not what any _character says_, not even what any _societies say_, but _what happens in Planecrash_ and what the _causal process_ there seems to think about matters important to you, what do you see?
275 As a _quick_ reply to the followup question (posted within 19 minutes of it being asked), I said that Cheliax was at a structural disadvantage in its conflict with the forces of Good, because learning how to think inevitably turns mortals away from Asmodeus's will.
277 But I was more interested in replying to the part about me being ill-shaped to another's purpose. (I said, after sleeping on it,[^async-discord] that I wouldn't have considered that on-topic for the fiction server, but if _he_ thought it was on-topic, then it made sense for me to reply.)
279 [^async-discord]: Discord lends itself quite well to a mix of synchronous and asynchronous communication, depending on when people happen to be at their computers.
281 I said that he seemed really stuck on this hypothesis that it was Michael Vassar's fault that I'd been shaped into an alienated and angry thing.
283 To be clear, I totally agreed that I had been shaped into an alienated and an alienated and angry thing. Obviously. But speaking of people "look[ing] inside themselves and correctly see[ing] that this is not how they work" (as Yudkowsky had said earlier), I thought he was getting the causality all wrong.
285 It seemed to _me_ that the reason I had become an alienated and angry thing is because I had been shaped by [making an extraordinary effort](https://www.lesswrong.com/posts/GuEsfTpSDSbXFiseH/make-an-extraordinary-effort) since 2016 to respond to a class of things that included Yudkowsky "mak[ing] up sophisticated stories for why pretty obviously true things are false"—again referencing [Oliver Habryka's comment on "Challenges to Yudkowsky's Pronoun Reform Proposal."](https://www.lesswrong.com/posts/juZ8ugdNqMrbX7x2J/challenges-to-yudkowsky-s-pronoun-reform-proposal?commentId=he8dztSuBBuxNRMSY)
287 That's the context in which it wasn't surprising that my Art had involved some amount of specialization in "detecting whose monkey-side they seem to be on." In a world where monkeys are trying to cover up otherwise-obvious truths, successfully blowing the whistle on them involves being sensitive to their monkey games; figuring out the truth they're trying to cover up is the easy part. The whistleblowing-skill of of promoting otherwise-obvious things to _common_ knowledge in opposition to a Power trying to prevent common knowledge, is different from the science-skill of figuring out organically-nonobvious things from scratch. It _makes sense_ for Alexander Solzhenitsyn and [Andrey Kolmogorov](https://slatestarcodex.com/2017/10/23/kolmogorov-complicity-and-the-parable-of-lightning/)—or for that matter, John Galt and Robert Stadler—to have developed different crystalized skills.
289 (Indeed, it even makes sense for Kolmogorov and Stadler to _not_ develop some skills, because the skills would show up under [Detect Thoughts](https://www.d20pfsrd.com/magic/all-spells/d/detect-thoughts/).)
291 If it was all Michael's fault for "extensively meta-gas[lighting me] into believing that everyone generally and [him] personally [were] engaging in some kind of weird out-in-the-open gaslighting", I asked, echoing Yudkowsky's language from earlier (with appropriate quotation marks and brackets),[^gaslighting] then _how come Oli could see it, too?_
293 [^gaslighting]: In particular, Yudkowsky was the one who introduced the word _gaslighting_ into the conversation; I hadn't previously used the word myself.
297 > I think if you asked Oli about the state of reality with respect to this whole affair, he'd have a very different take from your take, _if you're still able to hear differences instead of only those similarities you demand._
299 That sounded like an easy enough experimental test! I wrote Habryka an email explaining the context, and asking him what "very different take" he might have, if any. (I resisted the temptation to start a [Manifold market](https://manifold.markets/) first.) As I mentioned in the email, I didn't expect to have a very different take from him _about the state of reality_. ("Zack is (still?!) very upset about this, but Oli mostly doesn't care" is a values-difference, not a disagreement about the state of reality.) I didn't think I disagreed with _Yudkowsky_ much about the state of reality! (In his own account, he thought it was ["sometimes personally prudent [...] to post your agreement with Stalin about things you actually agree with Stalin about"](/images/yudkowsky-personally_prudent_and_not_community-harmful.png), and I believed him; I was just unhappy about some of the side-effects of his _prudence_.)
301 Oliver didn't reply. (I might have guessed the wrong email address, out of the two I had on file for him?) I don't blame him; it might have been timelessly ungrateful of me to ask.[^timeless-ingratitude]
303 [^timeless-ingratitude]: One reason people are reluctant to make on-the-record statements in politically charged contexts is because they're afraid the statements will be _used_ to drag them into more political fights later. Habryka had already done me a huge favor by being brave enough to state the obvious in March; I had no right to demand anything more of him.
305 Regarding my quick reply about Cheliax's structural disadvantage, Yudkowsky said it was "okay as one element", but complained that the characters had already observed it out loud, and that I "didn't name any concrete sequence of events that bore it out or falsified it." He continued:
307 > I think you could find a lot more than this if your brain were still able to see incongruent facts as well as only congruent facts. what does the underlying reality of Planecrash think about your Most Important Issues? what is shown, but maybe never even told at all? you gave the reply of somebody who can _only_ see social realities and _only_ what people say and not what _just happens, at all_, even inside a story, you didn't say _a thing that happened._
309 At this point, I was a bit suspicious that _any_ answer that wasn't exactly whatever he was thinking of would be dismissed as too social or too inferentially close to something one of the characters had said. What did it mean for the _universe_ to say something about valorizing truth?
311 The original prompt ("What are some of the ways _Planecrash_ valorizes truth") had put me into 11th-grade English class mode; the revision "if I ask you, not what any _character_ says [...]" made me think the 11th-grade English teacher expected a different answer. Now the revised–revised prompt "what does the underlying reality of _Planecrash_ think about your Most Important Issues?", with the previous rebukes in my context window, was making me think I should be reaching for an act of philosophical [Original Seeing](https://www.lesswrong.com/posts/SA79JMXKWke32A3hG/original-seeing), rather than trying to be a diligent schoolstudent playing the 11th-grade English class game. I thought about it ... and I _saw something_.
313 _Thesis_: the universe of _Planecrash_ is saying that virtue ethics—including, as a special case, my virtue ethics about it being good to tell the truth and reveal information—are somewhat unnatural.
315 In the story, the god Adabar values trading fairly, even with those who can't verify that their partners are keeping up their end of the deal,[^trade-verification] and also wants to promote fair trading _elsewhere_ in Reality (as contrasted to just being fair Himself).
317 [^trade-verification]: Significantly, this is somewhat "unnatural" behavior according to Yudkowsky's view of decision theory. Ideal agents are expected to cooperate with agents whose cooperation is _conditional_ on their own cooperation, not simply those that cooperate with them: you "should" defect against a rock with the word "COOPERATE" painted on it, and you "shouldn't" trade for what you could just as easily steal. See §6 of ["Robust Cooperation in the Prisoner's Dilemma: Program Equilibrium via Provability Logic"](https://arxiv.org/abs/1401.5577).
319 Adabar is kind of a weirdo. He's not a vanishly rare freak (whose specification would require lots of uncompressible information); there _is_ a basin of attraction in the space of pre-gods, where creatures who develop a computationally efficient "fairness" heuristic in their ancestral environment and reify that into their utilityfunction when they ascend to divinity, but it's not a _huge_ basin of attraction; most gods aren't like Adabar.
321 It's the same thing with honesty. Generic consequentialists have no reason to "tell the truth" to agents with different utility functions when they're not under compact and being compensated for the service. Why _would_ you emit signals that other agents can interpret as a map that reflects the territory? [You can't get more paperclips that way!](https://arbital.com/p/not_more_paperclips/)
323 I had previously written about this in ["Commucation Requires Common Interests or Differential Signal Costs"](https://www.lesswrong.com/posts/ybG3WWLdxeTTL3Gpd/communication-requires-common-interests-or-differential); you needed some common interests in order for flexible, "digital" language to exist at all. ("Digital" language being that for which the relationship between signals and meaning can be arbitrary, in contrast to costly signaling, where me expending resources at least tells you that I could afford those resources.)
325 It's _possible_ for imperfectly deceptive social organisms to develop a taste for "honesty" as a computationally efficient heuristic for navigating to Pareto improvements in the ancestral environment, which _might_ get reified into the utilityfunction as they ascend—but that's an Adabar-class weird outcome, not the default outcome.
327 So—insofar as my Most Important Issues revolved around an obsession with motivational transparency, wanting to live in a world that wasn't lying to me, wanting to _reveal information_ as an end in itself, unilaterally rather than only as part of a coordinated negotiation scheme, without necessarily being _paid_ for it, but just because it is _right_ ...
329 It seems like my answer to the question of, "What does the underlying causal process of _Planecrash_ think about your Most Important Issues; what are some the ways that _Planecrash_ valorizes truth-telling as you, yourself, see that virtue?" is, "It doesn't" (!). Truth-telling is a virtue ethic, and _Planecrash_ depicts a universe ruled by consequentialist gods who only implement virtue ethics insofar as that made it into their utilityfunction.
331 I realized, of course, that this certainly wasn't the answer Yudkowsky was looking for. But it seemed like a _better_ answer than me trying to play the schoolstudent. He asked what I saw when I looked at what the fictional universe was saying about my problems, and I looked, and _I saw something_. (Something philosophically substantive, definitely not just a social reality.) It seemed more honest to just report that, rather than keep trying to [guess the teacher's password](https://www.lesswrong.com/posts/NMoLJuDJEms7Ku9XS/guessing-the-teacher-s-password).
333 [("Hermione knew the correct answer wouldn't impress Professor Quirrell, but it was the correct answer, so she said it.")](https://hpmor.com/chapter/70)
335 So, after sleeping on it first, I posted the explanation of what I saw to the channel (including the parts about how the original prompts steered me, and that I realized that this wasn't the answer he was looking for).
337 The outcome was—silence. No response from Yudkowsky in several days. Maybe I shouldn't have ran with my Original Seeing answer? I showed the transcripts to a friend, who compared my answer about consequentialist gods to including a list of your country's war crimes in a high school essay assignment about patriotism; I had done a terrible job of emitting symbols that made me a good monkey, and a mediocre-at-best job of flipping the table (rejecting Yudkowsky's "pass my test before I recognize your criticism as legitimate" game) and picking a fight instead.
339 ("Don't look at me," he added, "I would've flipped the table at the beginning.")
341 I tried to explain that my third answer wasn't _just_ doubling down on my previous thesis: "my virtue ethics run against the grain of the hidden Bayesian structure of reality" wasn't an argument _in favor of_ my virtue ethics. My friend wasn't buying it; I still hadn't been fulfilling the original prompt.
343 He had me there. I had no more excuses after that: I had apparently failed the test. I was feeling pretty glum about this, and lamented my poor performance in the `#drama` channel of another Discord server (that Yudkowsky was also a member of). I had thought I was doing okay: I definitely _didn't_ say, "That's impossible because Big Yud and Linta are lying liars who hate Truth", and there were reasons why my Original Seeing answer made sense _to me_ as a thing to say, but _that wasn't what I was being tested on_. It _genuinely_ looked bad in context. I had failed in [my ambition to know how it looks](/2022/Dec/context-is-for-queens/#knowing-how-that-looks).
345 I think Yudkowsky saw the `#drama` messages in the other server (he left an emoji-reaction in the relevant timespan of messages) and took pity on me. (Negative feedback from a teacher is kinder than the teacher not even deigning to grade your assignment at all.)
347 As examples of the kind of thing he was looking for, he cited Keltham letting Carissa wait before telling him disturbing things about Golarion, or talking himself out of taking another Owl's Wisdom or putting on a cognitive-enhancement headband on account of his squeamishness about mind-altering interventions. If Keltham had been more proactive about seeking knowledge, he could have uncovered the Conspiracy earlier; the universe punished his cowardice.
349 Or consider Peranza, who awakens to seeing the evil of Asmodeanism—but manages to get out a critical warning to the Good god Iomedae, and ends up being rescued from punishment in Hell; the universe rewarded her bravery. This is a big theme, Yudkowsky said; I shouldn't have had to look in weird side corners to dredge up something exotic to say; my initial answers were "really small on the scale of a story whose central conflict is that Cheliax is hiding the truth from Keltham and Asmodeus is hiding the truth from Cheliax."
351 In characteristically condescending fashion, he said that he was worried about "the possibility that earthlings are only capable of hearing what the characters said to each other, because to ask what the universe thinks is some kind of direction of thought that Twitter has trained out of them", and hoped that readers don't "come away with the wordless sense of the universe being a place that rewards you for not looking places."
353 Regarding the intended exam answers about the universe's treatment of Keltham and Peranza—fair enough; I'll acknowledge that I didn't do great on the literary exam as assigned. Other participants in the chatroom, and readers of this memoir, should correspondingly update their beliefs about my competence. When I tried to do Original Seeing about what the universe of _Planecrash_ was saying, it came out in a particular _shape_ (characteristic of my recent preoccupations), and a more powerful mind would be able to do different shapes; I could protest that the prompts didn't do enough to steer me away from that (the use of the second person in "as you, yourself, see that virtue" and "your Most Important Issues" keeping me anchored on my own concerns), but that would be too much excuse-making for a mediocre exam performance.
355 (Peranza's pre-awakening username[^glowfic-username] was 'not-looking-there'! My 11th-grade English class algorithm probably would have gotten there if I had just given it more compute, instead of running with my philosophy insight!)
357 [^glowfic-username]: "Glowfic" stories were originally hosted on DreamWidth (a LiveJournal clone), with each character's dialogue and actions being posted from "their own" account (and therefore their own username, typically distinct from the character's own name). When the bespoke _glowfic.com_ website launched, the convention of characters having usernames was retained.
359 On the other hand, however poorly my exam performance reflected on other people's estimates of my competence and the question of whether Yudkowsky should consider my criticisms of dath ilan as coming from a "peer", it still doesn't invalidate my criticisms of dath ilan, which can, still, be evaluated on their own merits.
361 (Was I a fool to so submissively agree to be tested, given that Yudkowsky could predictably find some grounds to dismiss me as a mere earthling? Should I have tried to negotiate—I'm happy to take your test, but only if _you_ reply to my argument that spoiler protections are morally different from coverups?)
363 Slightly later in the thread, Yudkowsky would say that the story of _Planecrash_ was that of Keltham's innocence born of growing up in an incredibly high-functioning Society leading him to wrongfully trust Cheliax, which was not dath ilan. Someone who characterized the story as endorsing outsourcing one's epistemology, without noting that most of the story is about trusting someone wrongly and being accordingly punished by reality, would be behaving like the hostile critics of _Harry Potter and the Methods of Rationality_ who harp on [Draco wanting to rape Luna](https://hpmor.com/chapter/7) without mentioning Harry's reaction to that. "[I]f you leave out very clear information like that, which contradicts the politicized story you want to tell, that's basically tantamount to lying, falling short of it only under the most technical-words versions of what it means to tell the truth", Yudkowsky said.
365 I agree that it would be wrong to summarize the dath ilan mythos as endorsing trusting authorities in general, but this would seem somewhat beside the point of my criticism. The work depicts Keltham suffering from trusting Cheliax, which is not dath ilan, and lampoons Earth, which is not dath ilan, but my point was that _dath ilan itself is not trustworthy_ (except by ludicrous authorial assertion). The universe of _Planecrash_ (like [almost all](https://en.wikipedia.org/wiki/Almost_all) universes) doesn't itself reward you for not looking places. But dath ilan as a Society _absolutely_ punishes you for looking places[^punishment] if you expect to tell anyone about it—if you expect to [live in truth](https://www.vhlf.org/havel-in-the-media/living-in-truth/), rather than join the conspiracy to coordinate on false maps for the greater good.
367 [^punishment]: I mean "punish" in a colloquial sense, just that there are things most dath ilani get to do, like living in most cities, that my analogue in dath ilan wouldn't be allowed to do on account of his tendency to shout truths from street corners. I understand that there's a decision-theoretic sense in which this doesn't count as a "punishment", because dath ilan is only trying to advance its _own_ interests in preventing the spread of what it considers infohazards; the "punishment" makes sense for them whether or not I change my policy in response to it.
369 Regarding the lit exam, Yudkowsky added that he wished he had paid more attention to my re-framing, where "[he] said 'valorizes truth' and [I] repeated back 'valorizes truth-telling'". I pointed out that I had marked that as a proposed revision; I thought I was proposing a change rather than repeating. But maybe you don't get to propose changes when someone is testing you. He then gave a nice speech (in the style of C. S. Lewis's _The Screwtape Letters_) about the dangers of focusing on truth-telling:
371 > so if you have an awareness of you in how people can be broken, where it's possible to redirect them into infinite loops, how they can be induced to press the anger button over and over, then you can perhaps see how somebody setting out to break Zack Davis would get him to focus on truth-telling rather than truth-seeking. for the Way of searching out truth within yourself is one of calm, balance, questioning not 'what society tells you' but also your own thoughts, and also sometimes answering those questions and moving on to different ones; the operation, not of firmly rooting your feet, nor finding somewhere to hover forever uncertainly in place and immovable in that proud ignorance, but of picking up your feet and putting them back down, over and over, the uncomfortable operation of not staying in the same mental place, which most people find some way or another to reject. it valorizes calm, and balance, and these are not useful states of mind to people who would like you frantically doing something useful to them.
372 > when you get somebody to turn outward and away from Reality and towards their fellow monkeys and focus on truth-telling, then, their fellow monkeys being imperfect, there will always be something in which to explode into fury; so this is a useful state of mind to inculcate in somebody, lending itself to constant outrage at a world where somebody has once said a thing that if you look at it hard could be misleading or teach the wrong lesson, it misled you, how dare they!
373 > so by all means if you would like to destroy a rationalist, teach them anger and focus it on others' imperfect conformance to the principles they were once taught to hold dear to themselves
375 And you know, that's a fair criticism of me. It _is_ unhealthy to focus on other people's errors rather than perfecting oneself! I'm optimistic about rectifying this after I've gotten this Whole Dumb Story out of my system—to retire from this distasteful chore of criticizing Yudkowsky and "the community", and just go directly do the thing that I thought "the community" was for, in the time we have left.
377 But, as I pointed out, it was significant that the particular problem to which my Art had been shaped (in some ways) and misshaped (in others) wasn't just a matter of people being imperfect. Someone at the 2021 Event Horizon Independence Day party had told me that people couldn't respond to my arguments because of the obvious political incentives. And so, the angry question I wanted to ask, since I didn't immediately know how to rephrase it to not be doing the angry monkey thing, was, did Yudkowsky think I was supposed to _take that lying down?_
381 > **Eliezer** — 12/17/2022 5:50 PM
382 > you sure are supposed to not get angry at the people who didn't create those political punishments
384 > they're living in Cheliax and you want them to behave like they're not in Cheliax and get arrested by the Church
385 > your issue is with Asmodeus. take it to Him, and if you can't take Him down then don't blame others who can't do that either.
387 I thought the game theory was subtler than that. As a member of the Chelaxian resistance, I didn't have a quarrel with ordinary citizens who kept their heads down and paid their taxes, but the more someone actively collaborated with the Asmodean regime more than they had to, the more they started to seem like a legitimate military target. Yudkowsky's history of repeatedly making sweeping public declarations about transgenderism while leaving out very clear information which contradicted the politicized story he wanted to tell made him an active collaborator, not an innocent taxpayer trying to keep his head down.[^topic-bounce]
389 [^topic-bounce]: It was arguably bad for me to be talking about my grivances against Yudkowsky in the fiction server (as contrasted to my complaints about dath ilan's worldbuilding, which were unambiguously on-topic). I hadn't consciously steered the conversation this way, but the conversation _bounced_ in a way that made it on-topic; that was technically not my fault, even if the elephant in my brain was optimizing for this outcome.
391 He couldn't claim that everyone does this and that it was therefore insane to expect better, because (picking an arbitrary example) Paul Christiano didn't make those kinds of political statements while claiming that everyone from Earth is insane unlike him (!) while claiming the right to ignore counterarguments on political grounds (!!), because Paul Christiano wasn't trying to be a religious leader.
393 I gave a parable (along the lines of ["it's not the incentives; it's you"](https://www.lesswrong.com/posts/5nH5Qtax9ae8CQjZ9/tal-yarkoni-no-it-s-not-the-incentives-it-s-you)) to illustrate the problem: a senior scientist publishes a paper with a conclusion favorable to an organization that gave his lab a grant. The scientist's faithful student points out a fatal flaw in the paper, and suggests publishing a retraction. The scientist says, "No, that would be bad for my career." The student gets angry. The scientist says, "You're supposed to not get angry at the people who didn't create those career incentives. That's insane. Your issue is with Moloch: the publish-or-perish incentives of grant-funded science. If you can't take that down, then don't blame others who can't do that, either."
395 That is, blaming people who commit fraud (where an innocent mistake _becomes_ fraud if you refuse to acknowledge it after it's been pointed out) is part of a strategy for defeating Moloch, an attempt to establish norms against fraud.
397 I successfully baited Lintamande, the other _Planecrash_ coauthor, into engaging with me. Lintamande said I was impossible to talk to about my special topic, because of the bizarre hostile performative thing I did in place of truthseeking inquiry; I imposed too stark of a trade-off on my interlocutors between successfully saying anything at all and not getting immediately whalloped with 30,000 words from me about how they fundamentally lack intellectual integrity; my behavior discouraged stifled genuine discussion in places where I might see it.
399 (I almost wondered whether this might be a good thing from a _realpolitik_ perspective. I'd prefer to argue people out of bad ideas, but if the threat of an argument disincentivized people from spreading them, did that not have similar consequences at lesser expense? The game theory went both ways; I'd _also_ been doing some amount of self-censoring to avoid fights.)
401 I said that Lintamande was giving good feedback on my social behavior—which, okay, I should work on that—but which didn't particularly bear on the substance of my position. I was super-fighty because I was super-traumatized, because there was social–political territory I was trying to defend. The thing I was trying to keep on Society's shared map was, Biological Sex Actually Exists and Is Sometimes Decision-Relevant; Biological Sex Actually Exists and Is Sometimes Decision-Relevant _Even When It Makes People Sad_; Biological Sex Actually Exists and Is Sometimes Decision-Relevant _Even When a Prediction Market Says It Will Make People Sad_.
403 Lintamande said they agreed with those claims; Yudkowsky concurred with a "+1" emoji.
405 I didn't think it was fair, I continued, for Yudkowsky to pull his everyone-but-me-is-retarded shtick while simultaneously pretending that Gendered Pronouns For Everyone and Asking To Leave The System Is Lying was the crux of trans rights debates.
407 Lintamande said they didn't read Yudkowsky as having made any claims about the the crux of trans rights debates, just the narrow claim about pronouns.
409 I said that when he knocks down bad arguments for sex-based pronouns without addressing the good ones, I read him as (not necessarily consciously) playing a ["Weak Men Are Superweapons"](https://slatestarcodex.com/2014/05/12/weak-men-are-superweapons/) political strategy: carefully choosing what arguments to respond to in order to mood-affiliate with the winning team, without technically saying anything false. A lot of TERFs would be overjoyed to compromise on pronouns if pronouns weren't a wedge or proxy for the other issues that anyone cared about.
411 Lintamande pointed out that other people thought about this topic less often than I did. Maybe they didn't carefully select which arguments to respond to in order to cleverly mood-affiliate, so much as get prompted by the surrounding Society to think about pronouns, do so, and come up with an answer. They thought it was plausible that Yudkowsky would have given the same verdict on pronoun conventions if someone had posed the question to him as a hypothetical in 2004. Just because pronouns had become a political proxy debate, didn't mean that it was disingenuous to just talk about pronouns without reference to the proxy debate.
413 I didn't think that was what was actually going on.
416 * The input-output behavior of consciously trying to agree with Stalin and "getting prompted by the surrounding Society" are similar, as I noted in "Tails, Never Heard of Her"; it's not a coincidence that he didn't write this in 2004
417 * _speaking of the year 2004_; the thing I'm at war with is that I don't think he would _dare_ publish the same essay today The 2004 mailing list post was almost _mocking_ the guy, for being so naïve, for not seeing the type mismatch between the deep structure of reality, and mentalistic fantasies vaguely gestured at with English words.
418 * And the 2016–2021 posts _couldn't even acknolwedge that Biological Sex Actually Exists_. Did Yudkowsky expect us not to _notice_?? Coming from anyone else in the world, I wouldn't have minded. But the _conjunction_ of these political games and the eliezera racial supremacy rhetoric was just _insulting_.
419 * April said she didn't think the "Changing Emotions" argument was making claims relevant to trans people. The only claim you really needed was that it was reasonable for cis men and trans men, and cis women and trans women, to be in the same category for the purposes of general social interaction; whether you would need post-Singularity tech to make a biological male indistinguishable from a cis woman had little bearing on what we should make of trans women. That was interesting. April's profile said she was 19 years old and transfeminine. April saying that the essay wasn't making claims relevant to trans people; but _I_ thought it was relevant in 2008. (My dream of a fiction of animosity between a Yudkowsky-like and Lynn Conway-like character.)
420 * someone said "the word in their language doesn't match the word in yours"; and got a +1 emoji from Big Yud; I resisted the temptation to say "So ... I can define a word any way I want"; I call a killthread.
projects / Ultimately_Untrue_Thought.git / blob