Okay, being scared was entirely appropriate, but what I mean is that I was scared, and concluded that shaping the Singularity was _not my problem_, as contrasted to being scared, then facing up to the responsibility anyway. After a 2013 sleep-deprivation-induced psychotic episode which [featured](http://zackmdavis.net/blog/2013/03/religious/) [futurist](http://zackmdavis.net/blog/2013/04/prodrome/)-[themed](http://zackmdavis.net/blog/2013/05/relativity/) [delusions](http://zackmdavis.net/blog/2013/05/relevance/), I wrote to Anna, Michael, and some MIRI employees who had been in my contacts for occasional contract work, that "my current plan [was] to just try to forget about _Less Wrong_/MIRI for a long while, maybe at least a year, not because it isn't technically the most important thing in the world, but because I'm not emotionally stable enough think about this stuff anymore" (Subject: "to whom it may concern"). When I got a real programming job and established an income for myself, I [donated to CfAR rather than MIRI](http://zackmdavis.net/blog/2016/12/philanthropy-scorecard-through-2016/), because public rationality was something I could be unambiguously enthusiastic about, and doing anything about AI was not.
-At the time, it seemed fine for the altruistically-focused fraction of my efforts to focus on rationality, and to leave the save/destroy/take-over the world stuff to other, more emotionally-stable people, in accordance with the principle of comparative advantage. Yudkowsky had written his Sequences as a dependency for explaining [the need for friendly AI](https://www.lesswrong.com/posts/GNnHHmm8EzePmKzPk/value-is-fragile), ["gambl[ing] only upon the portion of the activism that would flow to [his] own cause"](https://www.lesswrong.com/posts/9jF4zbZqz6DydJ5En/the-end-of-sequences), but rationality was supposed to be the [common interest of many causes](https://www.lesswrong.com/posts/4PPE6D635iBcGPGRy/rationality-common-interest-of-many-causes). Even if I wasn't working or donating to MIRI specifically, I was still _helping_, a good citizen according to the morality of my tribe.
+At the time, it seemed fine for the altruistically-focused fraction of my efforts to focus on rationality, and to leave the save/destroy/take-over the world stuff to other, more emotionally-stable people, in accordance with the principle of comparative advantage. Yudkowsky had written his Sequences as a dependency for explaining [the need for friendly AI](https://www.lesswrong.com/posts/GNnHHmm8EzePmKzPk/value-is-fragile), ["gambl\[ing\] only upon the portion of the activism that would flow to \[his\] own cause"](https://www.lesswrong.com/posts/9jF4zbZqz6DydJ5En/the-end-of-sequences), but rationality was supposed to be the [common interest of many causes](https://www.lesswrong.com/posts/4PPE6D635iBcGPGRy/rationality-common-interest-of-many-causes). Even if I wasn't working or donating to MIRI specifically, I was still _helping_, a good citizen according to the morality of my tribe.
But fighting for public epistemology is a long battle; it makes more sense if you have _time_ for it to pay off. Back in the late 'aughts and early 'tens, it looked like we had time. We had these abstract philosophical arguments for worrying about AI, but no one really talked about _timelines_. I believed the Singularity was going to happen in the 21st century, but it felt like something to expect in the _second_ half of the 21st century.
Now it looks like we have—less time? Not just tautologically because time has passed (the 21st century is one-fifth over—closer to a quarter over), but because of new information from the visible results of the deep learning revolution.[^second-half] Yudkowsky seemed particularly [spooked by AlphaGo](https://www.lesswrong.com/posts/7MCqRnZzvszsxgtJi/christiano-cotra-and-yudkowsky-on-ai-progress?commentId=gQzA8a989ZyGvhWv2) [and AlphaZero](https://intelligence.org/2017/10/20/alphago/) in 2016–2017, not because superhuman board game players were dangerous, but because of what it implied about the universe of algorithms.
-There had been a post in the Sequences that made fun of "the people who just want to build a really big neural net." These days, it's increasingly looking like just building a really big neural net ... [actually works](https://www.gwern.net/Scaling-hypothesis)?—which seems like bad news; if it's "easy" for non-scientific-genius engineering talent to shovel large amounts of compute into the birth of powerful minds that we don't understand and don't know how to control, then it would seem that the world is soon to pass outside of our understanding and control.
+In part of the Sequences, Yudkowsky had been [dismissive of people who aspired to build AI without understanding how intelligence works](https://www.lesswrong.com/posts/fKofLyepu446zRgPP/artificial-mysterious-intelligence)—for example, by being overly impressed by the [surface analogy](https://www.lesswrong.com/posts/6ByPxcGDhmx74gPSm/surface-analogies-and-deep-causes) between artificial neural networks and the brain. He conceded the possibility of brute-forcing AI (if natural selection had eventually gotten there with no deeper insight, so could we) but didn't consider it a default and especially not a desirable path. (["If you don't know how your AI works, that is not good. It is bad."](https://www.lesswrong.com/posts/fKofLyepu446zRgPP/artificial-mysterious-intelligence))
+
+These days, it's increasingly looking like making really large neural nets ... [actually works](https://www.gwern.net/Scaling-hypothesis)?—which seems like bad news; if it's "easy" for non-scientific-genius engineering talent to shovel large amounts of compute into the birth of powerful minds that we don't understand and don't know how to control, then it would seem that the world is soon to pass outside of our understanding and control.
[^second-half]: In an unfinished slice-of-life short story I started writing _circa_ 2010, my protagonist (a supermarket employee resenting his job while thinking high-minded thoughts about rationality and the universe) speculates about "a threshold of economic efficiency beyond which nothing human could survive" being a tighter bound on future history than physical limits (like the heat death of the universe), and comments that "it imposes a sense of urgency to suddenly be faced with the fabric of your existence coming apart in ninety years rather than 10<sup>90</sup>."
I pointed out that that's exactly what one would expect if the Vassar/breakdown correlation was mostly a selection effect rather than causal—that is, if the causal graph was the fork "prone-to-psychosis ← underlying-bipolar-ish-condition → gets-along-with-Michael".
-I had also had a sleep-deprivation-induced-psychotic-break-with-hospitalization in February 2013, and shortly thereafter, I remember Anna remarking that I was sounding a lot like Michael. But I hadn't been talking to Michael at all beforehand! (My previous email conversation with him had been in 2010.) So what could Anna's brain have been picking up on, when she said that? My guess: there was some underlying dimension of psychological variation (psychoticism? bipolar?—you tell me; this is supposed to be Scott's professional specialty) where Michael and I were already weird/crazy in similar ways, and sufficiently bad stressors could push me further along that dimension (enough for Anna to notice). Was Scott also going to blame Yudkowsky for making people autistic?
+I had also had a sleep-deprivation-induced-psychotic-break-with-hospitalization in February 2013, and shortly thereafter, I remember Anna remarking that I was sounding a lot like Michael. But I hadn't been talking to Michael at all beforehand! (My previous email conversation with him had been in 2010.) So what could Anna's brain have been picking up on, when she said that? My guess: there was some underlying dimension of psychological variation (psychoticism? bipolar?—you tell me; this is supposed to be Scott's professional specialty) where Michael and I were already weird/crazy in similar ways, and sufficiently bad stressors could push me further along that dimension (enough for Anna to notice). Was Scott also going to blame Yudkowsky for making people [autistic](https://twitter.com/ESYudkowsky/status/1633396201427984384)?
Concerning the lightning parable, Scott said that from his perspective, the point of "Kolmogorov Complicity" was that, yes, people can be crazy, but that we have to live in Society without spending all our time freaking out about it. If, back in the days of my ideological anti-sexism, the first ten Yudkowsky posts I had read had said that men and women are psychologically different for biological reasons and that anyone who denies this is a mind-killed idiot—which Scott assumed Yudkowsky did think—he could imagine me being turned off. It was probably good for me and the world that that wasn't my first ten experiences of the rationalist community.
On 1 April 2022, Yudkowsky published ["MIRI Announces New 'Death With Dignity' Strategy"](https://www.lesswrong.com/posts/j9Q8bRmwCgXRYAgcJ/miri-announces-new-death-with-dignity-strategy), a cry of despair in the guise of an April Fool's Day post. MIRI didn't know how to align a superintelligence, no one else did either, but AI capabilities work was continuing apace. With no credible plan to avert almost-certain doom, the most we could do now was to strive to give the human race a more dignified death, as measured in log-odds of survival: an alignment effort that doubled the probability of a valuable future from 0.0001 to 0.0002 was worth one information-theoretic bit of dignity.
-In a way, "Death With Dignity" isn't really an update. Yudkowsky had always refused to give a probability of success, while maintaining that Friendly AI was ["impossible"](https://www.lesswrong.com/posts/nCvvhFBaayaXyuBiD/shut-up-and-do-the-impossible). Now, he says the probability is approximately zero.
+In a way, "Death With Dignity" isn't really an update. Yudkowsky had always refused to name a "win" probability, while maintaining that Friendly AI was ["impossible"](https://www.lesswrong.com/posts/nCvvhFBaayaXyuBiD/shut-up-and-do-the-impossible). Now, he says the probability is approximately zero.
Paul Christiano, who has a much more optimistic picture of humanity's chances, nevertheless said that he liked the "dignity" heuristic. I like it, too. It—takes some of the pressure off. I [made an analogy](https://www.lesswrong.com/posts/j9Q8bRmwCgXRYAgcJ/miri-announces-new-death-with-dignity-strategy?commentId=R59aLxyj3rvjBLbHg): your plane crashed in the ocean. To survive, you must swim to shore. You know that the shore is west, but you don't know how far. The optimist thinks the shore is just over the horizon; we only need to swim a few miles and we'll probably make it. The pessimist thinks the shore is a thousand miles away and we will surely die. But the optimist and pessimist can both agree on how far we've swum up to this point, and that the most dignified course of action is "Swim west as far as you can."
(I asked Anna how Yudkowsky could stand the Glowfic people. She said she thought Eliezer could barely stand anyone. That makes sense, I said.)
-Everyone in dath ilan receives rationality training from childhood, but knowledge and training deemed psychologically hazardous to the general population is safeguarded by an order of [Keepers of Highly Unpleasant Things it is Sometimes Necessary to Know](https://www.glowfic.com/replies/1612937#reply-1612937). AGI research takes place in a secret underground city; some unspecified form of social engineering steers the _hoi polloi_ away from thinking about the possibility of AI.
+Everyone in dath ilan receives rationality training from childhood,[^category-vindication] but knowledge and training deemed psychologically hazardous to the general population is safeguarded by an order of [Keepers of Highly Unpleasant Things it is Sometimes Necessary to Know](https://www.glowfic.com/replies/1612937#reply-1612937). AGI research takes place in a secret underground city; some unspecified form of social engineering steers the _hoi polloi_ away from thinking about the possibility of AI.
+
+[^category-vindication]: On the topic of dath ilan's rationality training, I did appreciate [this passage about the cognitive function of categorization](https://www.glowfic.com/replies/1779051#reply-1779051):
+
+ > Dath ilani kids get told to not get fascinated with the fact that, in principle, 'bounded-agents' with finite memories and finite thinking speeds, have any considerations about mapping that depend on what they want. It doesn't mean that you get to draw in whatever you like on your map, because it's what you want. It doesn't make reality be what you want.
+
+ Vindication! (This showed that Yudkowsky _does_ understand what was at issue in the "... Not Man for the Categories" dispute, even if I can't be credited with winning the argument for political reasons.)
Something that annoyed me about the portrayal of dath ilan was their incredibly casual attitude towards hiding information for some alleged greater good, seemingly without considering that [there are benefits and not just costs to people knowing things](http://benjaminrosshoffman.com/humility-argument-honesty/).
In the same story, Merrin is dressed up as a member of a fictional alien species, the Sparashki, due to having been summoned to the hospital from a fan convention with no time to change outfits. An agent from Exception Handling [tells Merrin that the bureau's Fake Conspiracy section is running an operation to plant evidence that Sparashki are real](https://glowfic.com/replies/1860952#reply-1860952), and asks Merrin not to contradict this, and Merrin just ... goes along with it.
-It's in-character for Merrin to go along with it, because she's a pushover. My question is, why is it okay that Exception Handling has a Fake Conspiracies section (!), any more than it would have been if FTX or Enron explicitly had a Fake Accounting department?
+It's in-character for Merrin to go along with it, because she's a pushover. My question is, why is it okay that Exception Handling explicitly has a Fake Conspiracies section (!), any more than it would have been if FTX or Enron explicitly had a Fake Accounting department?
+
+Is it okay because the idea of humanoid aliens walking around in real life is sufficiently implausible that no one is "really fooled"? But this seems to be contradicted by the text, in which [the narrator tells us that Exception Handling deliberately does things that wouldn't otherwise make sense in order to make it harder to understand what's actually going on in Civilization](https://glowfic.com/replies/1860955#reply-1860955) (presumably, in order to protect the secret underground AGI alignment project). Even if very few dath ilani take the "Sparashki are real" ruse literally, it's still portrayed as successfully _kicking up epistemic dust_. If it were really the case that no one was being fooled about anything, then Exception Handling wouldn't have a strategic reason to do it.
-(Is it because dath ilan are the [designated good guys](https://tvtropes.org/pmwiki/pmwiki.php/Main/DesignatedHero)? Well, [so was FTX](https://forum.effectivealtruism.org/posts/sdjcH7KAxgB328RAb/ftx-ea-fellowships).)
+Is it okay because dath ilan are the [designated good guys](https://tvtropes.org/pmwiki/pmwiki.php/Main/DesignatedHero)? Well, [so was FTX](https://forum.effectivealtruism.org/posts/sdjcH7KAxgB328RAb/ftx-ea-fellowships).
As another notable example of dath ilan hiding information for the alleged greater good, in Golarion, Keltham discovers that he's a sexual sadist, and deduces that Civilization has deliberately prevented him from realizing this, because there aren't enough corresponding masochists to go around in dath ilan. Having concepts for "sadism" and "masochism" as variations in human psychology would make sadists like Keltham sad about the desirable sexual experiences they'll never get to have, so Civilization arranges for them to _not be exposed to knowledge that would make them sad, because it would make them sad_ (!!).
I said that I thought people were missing this idea that the reason "truth is better than lies; knowledge is better than ignorance" is such a well-performing [injunction](https://www.lesswrong.com/posts/dWTEtgBfFaz6vjwQf/ethical-injunctions) in the real world (despite the fact that there's no law of physics preventing lies and ignorance from having beneficial consequences), is because [it protects against unknown unknowns](https://www.lesswrong.com/posts/E7CKXxtGKPmdM9ZRc/of-lies-and-black-swan-blowups). Of course an author who wants to portray an ignorance-maintaining conspiracy as being for the greater good, can assert by authorial fiat whatever details are needed to make it all turn out for the greater good, but _that's not how anything works in real life_.
-I started a new thread to complain about the attitude I was seeing (Subject: "Noble Secrets; Or, Conflict Theory of Optimization on Shared Maps"). When fiction in this world, _where I live_, glorifies Noble Lies, that's a cultural force optimizing for making shared maps less accurate, I explained. As someone trying to make shared maps _more_ accurate, this force was hostile to me and mine. I understood that "secrets" and "lies" are not the same thing, but if you're a consequentialist thinking in terms of what kinds of optimization pressures are being applied to shared maps, [it's the same issue](https://www.lesswrong.com/posts/YptSN8riyXJjJ8Qp8/maybe-lying-can-t-exist): I'm trying to steer _towards_ states of the world where people know things, and the Keepers of Noble Secrets are trying to steer _away_ from states of the world where people know things. That's a conflict. I was happy to accept Pareto-improving deals to make the conflict less destructive, but I wasn't going to pretend the pro-ignorance forces were my friends just because they self-identified as "rationalists" or "EA"s. I was willing to accept secrets around nuclear or biological weapons, or AGI, on "better ignorant than dead" grounds, but the "protect sadists from being sad" thing wasn't a threat to life; it was _just_ coddling people who can't handle reality, which made _my_ life worse.
+I started a new thread to complain about the attitude I was seeing (Subject: "Noble Secrets; Or, Conflict Theory of Optimization on Shared Maps"). When fiction in this world, _where I live_, glorifies Noble Lies, that's a cultural force optimizing for making shared maps less accurate, I explained. As someone trying to make shared maps _more_ accurate, this force was hostile to me and mine. I understood that "secrets" and "lies" are not the same thing, but if you're a consequentialist thinking in terms of what kinds of optimization pressures are being applied to shared maps, [it's the same issue](https://www.lesswrong.com/posts/YptSN8riyXJjJ8Qp8/maybe-lying-can-t-exist): I'm trying to steer _towards_ states of the world where people know things, and the Keepers of Noble Secrets are trying to steer _away_ from states of the world where people know things. That's a conflict. I was happy to accept Pareto-improving deals to make the conflict less destructive, but I wasn't going to pretend the pro-ignorance forces were my friends just because they self-identified as "rationalists" or "EA"s. I was willing to accept secrets around nuclear or biological weapons, or AGI, on "better ignorant than dead" grounds, but the "protect sadists from being sad" thing wasn't a threat to anyone's life; it was _just_ coddling people who can't handle reality, which made _my_ life worse.
I wasn't buying the excuse that secret-Keeping practices that wouldn't be okay on Earth were somehow okay on dath ilan, which was asserted by authorial fiat to be sane and smart and benevolent enough to make it work. Alternatively, if I couldn't argue with authorial fiat: the reasons why it would be bad on Earth (even if it wouldn't be bad in the author-assertion paradise of dath ilan) are reasons why _fiction about dath ilan is bad for Earth_.
-And just—back in the 'aughts, I said, Robin Hanson had this really great blog called _Overcoming Bias_. (You probably haven't heard of it.) I wanted that _vibe_ back, of Robin Hanson's blog[^overcoming-bias] in 2008—the will to _just get the right answer_, without all this galaxy-brained hand-wringing about who the right answer might hurt.
+And just—back in the 'aughts, I said, Robin Hanson had this really great blog called _Overcoming Bias_. (You probably haven't heard of it.) I wanted that _vibe_ back, of Robin Hanson's blog in 2008—the will to _just get the right answer_, without all this galaxy-brained hand-wringing about who the right answer might hurt.
-[^overcoming-bias]: _Overcoming Bias_ had actually been a group blog then, but I was enjoying the æsthetic of saying "Robin Hanson's blog" (when what I had actually loved about _Overcoming Bias_ was Yudkowsky's Sequences) as a way of signaling contempt for the Yudkowsky of the current year.
+(_Overcoming Bias_ had actually been a group blog then, but I was enjoying the æsthetic of saying "Robin Hanson's blog" (when what I had actually loved about _Overcoming Bias_ was Yudkowsky's Sequences) as a way of signaling contempt for the Yudkowsky of the current year.)
I would have expected a subculture descended from the memetic legacy of Robin Hanson's blog in 2008 to respond to that tripe about protecting people from the truth being a form of "recognizing independent agency" with something like—
The other chatroom participants mostly weren't buying what I was selling.
-A user called April wrote that "the standard dath ilani has internalized almost everything in the sequences": "it's not that the standards are being dropped[;] it's that there's an even higher standard far beyond what anyone on earth has accomplished". (This received a checkmark emoji-react from Yudkowsky, an indication of his agreement.)
+A user called April wrote that "the standard dath ilani has internalized almost everything in the sequences": "it's not that the standards are being dropped[;] it's that there's an even higher standard far beyond what anyone on earth has accomplished". (This received a checkmark emoji-react from Yudkowsky, an indication of his agreement/endorsement.)
Someone else said he was "pretty leery of 'ignore whether models are painful' as a principle, for Earth humans to try to adopt," and went on to offer some thoughts for Earth. I continued to maintain that it was ridiculous that we were talking of "Earth humans" as if there were any other kind—as if rationality in the Yudkowskian tradition wasn't something to aspire to in real life.
Not-actually-plausible conspiracies that everyone is in on (like "Sparashki are real") are a [superstimulus](https://www.lesswrong.com/posts/Jq73GozjsuhdwMLEG/superstimuli-and-the-collapse-of-western-civilization) like zero-calorie sweetener: engineered to let everyone enjoy the thrill of lying, without doing any real damage to shared maps.
-In "For No Laid Course Prepare", Merrin's coworkers falsely maintain to outsiders that Merrin always cosplays as a Sparashki while on duty. "This is not considered a lie, in that it would be universally understood and expected that no one in this social circumstance would tell the truth," the narrator tells us. The language used here is strikingly similar to that of one of the corrupt executives in _Moral Mazes_: "We lie all the time, but if everyone knows that we're lying, is a lie really a lie?"
+In "For No Laid Course Prepare", Merrin's coworkers falsely maintain to outsiders that Merrin always cosplays as a Sparashki while on duty. ["This is not considered a lie, in that it would be universally understood and expected that no one in this social circumstance would tell the truth,"](https://www.glowfic.com/replies/1857346#reply-1857346) the narrator tells us. The language used here is strikingly similar to that of one of the corrupt executives in [_Moral Mazes_](https://thezvi.wordpress.com/2019/05/30/quotes-from-moral-mazes/): "We lie all the time, but if everyone knows that we're lying, is a lie really a lie?"
-But if it were true that [everyone knew](https://thezvi.wordpress.com/2019/07/02/everybody-knows/), what would be _function_ of saying the false thing? On dath ilan (if not in Earth boardrooms), one supposes the answer is "Because it's fun"? But that just prompts the followup question: but what is the function of the brain giving out a "fun" reward in this context? It seems like at _some_ point, there has to be the expectation of _some_ cognitive system (although possibly not an entire "person") taking the signals literally.[^funny-or-powerful-falsehood]
+But if [everyone really knew](https://thezvi.wordpress.com/2019/07/02/everybody-knows/), what would be _function_ of saying the false thing? On dath ilan (if not in Earth boardrooms), one supposes the answer is "Because it's fun"? But that just prompts the followup question: but what is the function of the brain giving out a "fun" reward in this context? It seems like at _some_ point, there has to be the expectation of _some_ cognitive system (although possibly not an entire "person") taking the signals literally.[^funny-or-powerful-falsehood]
[^funny-or-powerful-falsehood]: This is why, when I notice myself misrepresenting my actual beliefs or motivations because I think it's funny or rhetorically powerful, I often take care to disclaim it immediately, precisely because I _don't_ think that "everybody knows"; I'm not going to give up on humor or powerful rhetoric, but I'm also not going to delude myself into thinking it's "zero-calorie": people who don't "get the joke" _are_ going to be misled, and I don't think it's unambigously "their fault" for not being able to read my "intent" to arbitrary precision. But maybe dath ilan is (by authorial fiat) sufficiently good at achieving common knowledge in large groups that they _can_ pull off a zero-calorie "everyone knows" conspiracy without damaging shared maps?
-The existence of such a widespread sneakiness/"taste for deception" trait among the eliezera, in conjunction with their culture just not particularly valuing public knowledge (because they assume everything important is being handled by the Keepers), explains the recurring conspiracies and coverups, like the Ordinary Merrin Conspiracy, Exception Handling's fabrication of evidence for real Sparashki, the sadism/masochism coverup, and [the village that deliberately teaches anti-redhead bigotry to children in order to test the robustness of dath ilan's general humanism indoctrination](https://www.lesswrong.com/posts/uyBeAN5jPEATMqKkX/lies-told-to-children-1).
+The existence of such a widespread sneakiness/"taste for deception" trait among the eliezera, in conjunction with their culture just not particularly valuing public knowledge (because they assume everything important is being handled by the Keepers), explains the recurring conspiracies and coverups, like the Ordinary Merrin Conspiracy, Exception Handling's fabrication of evidence for Sparashki being real, the sadism/masochism coverup, and [the village that deliberately teaches anti-redhead bigotry to children in order to test the robustness of dath ilan's general humanism indoctrination](https://www.lesswrong.com/posts/uyBeAN5jPEATMqKkX/lies-told-to-children-1).
+
+I stress (at least now, even if I didn't do a good job of explaining it at the time) that this hypothesis _doesn't_ require dath ilani to be cartoon villains who hate knowledge and want people to be ignorant. Just that, as a result of the widespread sneakiness trait and their outsourcing information-process to the Keepers, in the course of trying to accomplish other things, plans-that-involve-conspiracies are often higher in their search ordering than plans-that-involve-keeping-people-informed.
+
+I claimed that there was a hidden-core-of-rationality thing about a culture that values living in truth, that the dath ilani didn't have. In previous discussion of the Sparashki example, a user called lc had written, "If you see someone wearing an elf costume at work and conclude elves are real and make disastrous decisions based on that conclusion you are mentally deranged". And indeed, you would be mentally deranged if you did that _on Earth_, because we don't have an elves-are-real conspiracy on Earth.
+
+In elves-are-real conspiracy-world, you (Whistleblower) see someone (Conspirator) wearing an elf costume at work and say, "Nice costume." They say, "What costume?" You say, "I see that you're dressed like an elf, but elves aren't real." They say, "What do you mean? Of course elves are real. I'm right here." You say, "You know exactly what I mean."
+
+It would appear that there's a conflict between Conspirator (who wants to maintain a social reality in which they're an elf, because it's fun, and the conspiracy is sufficiently outlandish that it's assumed that no one is "really" being deceived) and Whistleblower (who wants default social reality to map to actual reality; make-believe is fine at a designated fandom convention which has designated boundaries, but let's be serious at work, where your coworkers are trying to make a living and haven't opted-in to this false social reality).
-I stress (at least now, even if I didn't do a good job of explaining it at the time) that this hypothesis _doesn't_ require the eliezera to be cartoon villains who hate knowledge and want people to be ignorant. Just that, in the course of trying to accomplish other things, plans-that-involve-conspiracies are often higher in their search ordering than plans-that-involve-keeping-people-informed.
+I was skeptical that a culture where people collude to maintain a fake social reality at their job in a hospital, and everyone else is expected to play along because it's fun, really has this living-in-truth thing. People play those social-reality games on Earth, too, and when _they_ say no one is being deceived, they're _definitely_ lying about that, and I doubted that the eliezera were actually built that differently.
A user called ajvermillion asked why I was being so aggressively negative about dath ilan. He compared it to Keltham's remark about how [people who grew up under a Lawful Evil government were disposed to take a more negative view of paternalism](https://www.glowfic.com/replies/1874754#reply-1874754) than they do in dath ilan, where paternalism basically works fine because dath ilan is benevolent.
-This question put me in a somewhat awkward position: it was a legitimate question that I felt I had to answer, that I had no way of answering honestly without at least _alluding_ to my prior greviances against Yudkowsky—which were off-topic for the server.
+This question put me in a somewhat awkward position: it was a legitimate question that I felt I had to answer, that I had no way of answering honestly without at least _alluding_ to my prior greviances against Yudkowsky ... which were off-topic for the server. (Again, I had told myself that I was here to comment on the story, not to prosecute my greviances.)
I tried to explain, briefly. Someone who might be even _more_ paranoid about abuses of power than someone who grew up with a Lawful Evil government, is someone who grew up under a power structure that put on a _good show_ of being clean and nice, but was actually corrupt and mean.
-Yudkowsky had this whole marketing image of him being uniquely sane and therefore uniquely benevolent, and because his Sequences were so life-changingly good, I _actually fell for it_. There was a long Dumb Story that was off-topic and I hadn't finished writing up, but basically, I had what I claimed were very strong reasons not to trust the guy anymore; I think he cares a lot about not explicitly _lying_, but what made the Sequences special is that they articulated a vastly higher standard than that, that he has no intention of living up to.
+Yudkowsky had this whole marketing image of him being uniquely sane and therefore uniquely benevolent, and because his Sequences were so life-changingly good, I _actually fell for it_. There was a long Dumb Story (this Story) that was off-topic and I hadn't then finished writing up, but basically, I had what I claimed were very strong reasons not to trust the guy anymore; I think he cares a lot about not explicitly _lying_, but what made the Sequences special is that they articulated a vastly higher standard than that, that he had no intention of living up to.
+
+And so, yeah, insofar as fiction about dath ilan functioned as marketing material for Yudkowsky's personality cult that I thought was damaging people like me (in some ways, while simultaneously helping us in other ways), I had an incentive to come up with literary criticism that paints dath ilan negatively?
+
+It was great for ajvermillion to notice this! It _would_ be bad if my brain were configured to come up with dath-ilan-negative literary criticism, and for me to _simultaneously_ present myself as an authority on dath ilan whom you should trust. But if dath-ilan-negative literary criticism was undersupplied for structural reasons (because people who like a story are selected for not seeing things the story is doing that are Actually Bad), and my brain was configured to generate it anyway (because I disliked the person Yudkowsky had become, in contrast to the person he was in 2008), it seemed pro-social for me to post it, for other people to take or leave according to their own judgement?
+
+Yudkowsky soon entered the thread again, initially replying to someone else. (I remarked parenthetically that his appearance made me think I should stop wasting time snarking in his fiction server and just finish my memoir already.) We had a brief back-and-forth:
+
+> **Eliezer** — 11/29/2022 10:33 PM
+> the main thing I'd observe contrary to Zack's take here, is that Keltham thought that not learning about masochists he can never have, was obviously in retrospect what he'd have wanted Civilization to do, or do unless and until Keltham became rich enough to afford a masochist and then he could be told
+> in other words, Keltham thought he was obviously being treated the way that counterfactual fully-informed Keltham would have paid Governance to treat not-yet-informed Keltham
+> that this obeys the social contract that Keltham thought he had, is part of why Keltham is confident that the logic of this particular explanation holds together
+> **zackmdavis** — 11/29/2022 10:35 PM
+> the level of service that Keltham is expecting is _not the thing I learned from Robin Hanson's blog in 2008_
+> **Eliezer** — 11/29/2022 10:36 PM
+> I am sorry that some of the insane people I attracted got together and made each other more insane and then extensively meta-gaslit you into believing that everyone generally and me personally was engaging in some kind of weird out-in-the-open gaslighting that you could believe in if you attached least-charitable explanations to everything we were doing
+
+It was pretty annoying that Yudkowsky was still attributing my greviances to Michael's malign influence—as if the gender identity revolution was something I would otherwise have just _taken lying down_. In the counterfactual where Michael had died in 2015, I think something like my February 2017 breakdown would have likely happened anyway. (Between August 2016 and January 2017, I sent Michael 14 emails, met with him once, and watched 60% of South Park season 19 at his suggestion, so he was _an_ influence on my thinking during that period, but not a disproportionately large one compared to everything else I was doing at the time.) How would I have later reacted to the November 2018 "hill of meaning" Tweets (assuming they weren't butterfly-effected away in this counterfactual)? It's hard to say. Maybe, if that world's analogue of my February 2017 breakdown had gone sufficiently badly (with no Michael to visit me in the psych ward or help me make sense of things afterwards), I would have already been a broken man, and not even sent Yudkowsky an email. In any case, I feel very confident that my understanding of the behavior of "everyone generally and [Yudkowsky] personally" would not have been _better_ without Michael _et al._'s influence.
+
+> [cont'd]
+> you may recall that this blog included something called the "Bayesian Conspiracy"
+> they won't tell you about it, because it interferes with the story they were trying to drive you insaner with, but it's so
+> **zackmdavis** — 11/29/2022 10:37 PM
+> it's true that the things I don't like about modern Yudkowsky were still there in Sequences-era Yudkowsky, but I think they've gotten _worse_
+> **Eliezer** — 11/29/2022 10:39 PM
+> well, if your story is that I was always a complicated person, and you selected some of my posts and liked the simpler message you extracted from those, and over time I've shifted in my emphases in a way you don't like, while still having posts like Meta-Honesty and so on... then that's a pretty different story than the one you were telling in this Discord channel, like, just now. today.
+
+Is it, though? The "always a complicated person [who has] shifted in [his] emphases in a way [I] don't like" story was true, of course, but it elided the substantive reasons _why_ I didn't like the new emphases, which could presumably be evaluated on their own merits.
+
+It's interesting that Yudkowsky listed "still having posts like Meta-Honesty" as an exculpatory factor here. The thing is, I [wrote a _critique_ of Meta-Honesty](https://www.lesswrong.com/posts/MN4NRkMw7ggt9587K/firming-up-not-lying-around-its-edge-cases-is-less-broadly). It was well-received (being [cited as a good example in the introductory post for the 2019 Less Wrong Review](https://www.lesswrong.com/posts/QFBEjjAvT6KbaA3dY/the-lesswrong-2019-review), for instance). I don't think I could have written a similarly impassioned critique of anything from the Sequences era, because the stuff from the Sequences era still looked _correct_ to me. To me, "Meta-Honesty" was evidence _for_ Yudkowsky having relinquished his Art and lost his powers, not evidence that his powers were still intact.
+
+I didn't have that response thought through in real time. At the time, I just agreed:
+
+> **zackmdavis** — 11/29/2022 10:39 PM
+> It is!
+> I'm not done working through the hate-warp
+
+(This being a reference to part of _Planecrash_ in which [Keltham tells Carissa to be aware of her un-dath ilani tendency to feel "hatred that warps reality to be more hateable"](https://www.glowfic.com/replies/1882822#reply-1882822).)
+
+> **Eliezer** — 11/29/2022 10:40 PM
+> so one thing hasn't changed: the message that you, yourself, should always be trying to infer the true truth, off the information you already have.
+> if you know you've got a hate-warp I don't know why you're running it and not trying to correct for it
+> are you in fact also explicitly aware that the people who talk to you a lot about "gaslighting" are, like, insane?
+> **zackmdavis** — 11/29/2022 10:42 PM
+> I'm not really part of Vassar's clique anymore, if that's what you mean
+> **Eliezer** — 11/29/2022 10:44 PM
+> it looks from outside here like they stomped really heavy footprints all over your brain that have not healed or been filled in
+> **zackmdavis** — 11/29/2022 10:49 PM
+> it looks from inside here that the thing I'm not healed from is the thing where, as Oliver Habryka put it, I "should expect that depending on the circumstances community leaders might make up sophisticated stories for why pretty obviously true things are false" ([https://www.lesswrong.com/posts/juZ8ugdNqMrbX7x2J/challenges-to-yudkowsky-s-pronoun-reform-proposal?commentId=he8dztSuBBuxNRMSY](https://www.lesswrong.com/posts/juZ8ugdNqMrbX7x2J/challenges-to-yudkowsky-s-pronoun-reform-proposal?commentId=he8dztSuBBuxNRMSY)), and Michael and Ben and Jessica were _really_ helpful for orienting me to that particular problem, even if I disagree with them about a lot of other things and they seem crazy in other ways
+> (rule thinkers in, not out)
+
+(I was pleased to get the link to Habryka's comment in front of Yudkowsky, if he hadn't already seen it.)
+
+> **Eliezer** — 11/29/2022 10:55 PM
+> the most harm they did you was to teach you to see malice where you should have seen mortality
+> noninnocent error is meaningfully different from innocent error; and noninnocent error is meaningfully different from malice
+> Keltham deduced the lack of masochists in dath ilan by asking the question, "Why would Civilization have kept this information from me?", _ruling out_ or actually not even thinking of such ridiculous hypotheses as "Because it was fun", and settling on the obvious explanation that explained _why Keltham would have wanted Civilization to do that for him_—masochists not existing or being incredibly rare and unaffordable to him. You looked at this and saw malice everywhere; you couldn't even see _the fictional world_ the author was trying to give you _direct description about_. You didn't say that you disbelieved in the world; you could not see what was being _described_.
+
+(When a literary critic proposes a "dark" interpretation of an author's world, I think it's implied that they're expressing disbelief in the "intended" world; the fact that I was impudently refusing to buy the benevolent interpretation wasn't because I didn't understand it.)
+
+> Hate-warp like this is bad for truth-perception; my understanding of the situation is that it's harm done to you by the group you say you left. I would read this as being a noninnocent error of that group; that they couldn't get what they wanted from people who still had friends outside their own small microculture, and noninnocently then decided that this outer culture was bad and people needed to be pried loose from it. They tried telling some people that this outer culture was gaslighting them and maliciously lying to them and had to be understood in wholly adversarial terms to break free of the gaslighting; that worked on somebody, and made a new friend for them; so their brain noninnocently learned that it ought to use arguments like that again, so they must be true.
+> This is a sort of thing I super did not do because I _understood_ it as a failure mode and Laid My Go Stones Against Ever Actually Being A Cult; I armed people with weapons against it, or tried to, but I was optimistic in my hopes about how much could actually be taught.
+> **zackmdavis** — 11/29/2022 11:20 PM
+> Without particularly defending Vassar _et al._ or my bad literary criticism (sorry), _modeling the adversarial component of non-innocent errors_ (as contrasted to "had to be understood in wholly adversarial terms") seems very important. (Maybe lying is "worse" than rationalizing, but if you can't hold people culpable for rationalization, you end up with a world that's bad for broadly the same reasons that a world full of liars is bad: we can't steer the world to good states if everyone's map is full of falsehoods that locally benefitted someone.)
+> **Eliezer** — 11/29/2022 11:22 PM
+> Rationalization sure is a huge thing! That's why I considered important to discourse upon the science of it, as was then known; and to warn people that there were more complicated tangles than that, which no simple experiment had shown yet.
+> **zackmdavis** — 11/29/2022 11:22 PM
+> yeah
+> **Eliezer** — 11/29/2022 11:23 PM
+> It remains something that mortals do, and if you cut off anybody who's ever done that, you'll be left with nobody. And also importantly, people making noninnocent errors, if you accuse them of malice, will look inside themselves and correctly see that this is not how they work, and they'll stop listening to the (motivated) lies you're telling them about themselves.
+> This also holds true if you make up overly simplistic stories about 'ah yes well you're doing that because you're part of $woke-concept-of-society' etc.
+> **zackmdavis** — 11/29/2022 11:24 PM
+> I think there's _also_ a frequent problem where you try to accuse people of non-innocent errors, and they motivatedly interpret _you_ as accusing malice
+> **Eliezer** — 11/29/2022 11:25 PM
+> Then invent new terminology. I do that all the time when existing terminology fails me.
+> Like I literally invented the term 'noninnocent error' right in this conversation.
+> **zackmdavis** — 11/29/2022 11:27 PM
+> I've tried this, but maybe it wasn't good enough, or I haven't been using it consistently enough: [https://www.lesswrong.com/posts/sXHQ9R5tahiaXEZhR/algorithmic-intent-a-hansonian-generalized-anti-zombie](https://www.lesswrong.com/posts/sXHQ9R5tahiaXEZhR/algorithmic-intent-a-hansonian-generalized-anti-zombie)
+> I should get ready for bed
+> I will endeavor to edit out the hate-warp from my memoir before publishing, and _probably_ not talk in this server
+> **Eliezer** — 11/29/2022 11:31 PM
+> I think you should worry first about editing the hate-warp out of yourself, but editing the memoir might be useful practice for it. Good night.
+
+It turned out that I was lying about probably not talking in the server anymore. (Hedging with the word "probably" didn't make the claim true, and of course I wasn't _consciously_ lying, but that hardly seems exculpatory.)
+
+The next day, I belatedly pointed out that "Keltham thought that not learning about masochists he can never have, was obviously in retrospect what he'd have wanted Civilization to do" seemed to contradict "one thing hasn't changed: the message that you, yourself, should always be trying to infer the true truth". In the first statement, it didn't sound like Keltham thinks it's good that Civilization didn't tell him so that he could figure it out for himself (in accordance with the discipline of "you, yourself, always trying to infer the truth"). It sounded like he was better off not knowing—better off having a _less accurate self-model_ (not having the concept of "obligate romantic sadism"), better off having a _less accurate world-model_ (thinking that masochism isn't real).
+
+In response to someone positing that dath ilani were choosing to be happier but less accurate predictors, I said that I read a blog post once about why you actually didn't want to do that, linking to [an Internet Archive copy of "Doublethink (Choosing to Be Biased)"](https://web.archive.org/web/20080216204229/https://www.overcomingbias.com/2007/09/doublethink-cho.html) from 2008[^hanson-conceit]—at least, that was _my_ attempted paraphrase; it was possible that I'd extracted a simpler message from it than the author intended.
+
+[^hanson-conceit]: I was really enjoying the "Robin Hanson's blog in 2008" conceit.
+
+A user called Harmless explained the loophole. "Doublethink" was pointing out that decisions that optimize the world for your preferences can't come from nowhere: if you avoid painful thoughts in your map, you damage your ability to steer away from painful outcomes in the territory. However, there was no rule that all the information-processing going into decisions that optimize the world for your preferences had to take place in _your brain_ ...
+
+I saw where they were going and completed the thought: you could build a Friendly AI or a Civilization to see all the dirty things for you, that would make you unhappy to have to see yourself.
+
+Yudkowsky clarified his position:
+
+> My exact word choices often do matter: I said that you should always be trying to _infer_ the truth. With the info you already have. In dath ilan if not in Earth, you might decline to open a box labeled "this info will make you permanently dissatisfied with sex" if the box was labeled by a prediction market.
+> Trying to avoid inferences seems to me much more internally costly than declining to click on a spoiler box.
+
+I understood the theory, but I was still extremely skpetical of the practice, assuming the eliezera were even remotely human. Yudkowsky described the practice of "keeping BDSM secret and trying to prevent most sadists from discovering what they are—informing them only when and if they become rich enough or famous enough that they'd have a high probability of successfully obtaining a very rare masochist" as a "basically reasonable policy option that [he] might vote for, not to help the poor dear other people, but to help [his] own counterfactual self."
+
+The problem I saw with this is that becoming rich and famous isn't a purely random exogenous event. In order to make an informed decision about whether or not to put in the effort to try to _become_ rich and famous (as contrasted to choosing a lower-risk or more laid-back lifestyle), you need accurate beliefs about the perks of being rich and famous.
+
+The dilemma of whether to make more ambitious economic choices in pusuit of sexual goals was something that _already_ happens to people on Earth, rather than being hypothetical. I once met a trans woman who spent a lot of her twenties and thirties working very hard to get money for various medical procedures. I think she would be worse off under a censorship regime run by self-styled Keepers who thought it was kinder to prevent _poor people_ from learning about the concept of "transsexualism".
+
+Further discussion established that Yudkowsky was (supposedly) already taking into account that class of distortion on individuals' decisions, but that the empirical setting of probabilities and utilities happened to be such that ignorance came out on top.
+
+I wasn't sure what my wordcount and "diplomacy" "budget limits" for the server were, but I couldn't let go; I kept the thread going on subsequent days. There was something I felt I should be able to convey, if I could just find the right words.
+
+When [Word of God](https://tvtropes.org/pmwiki/pmwiki.php/Main/WordOfGod) says, "trying to prevent most [_X_] from discovering what they are [...] continues to strike me as a basically reasonable policy option", then, separately from the particular value of _X_, I expected people to jump out of their chairs and say, "No! This is wrong! Morally wrong! People can stand what is true about themselves, because they are already doing so!"
+
+And to the extent that I was the only person jumping out of my chair, and there was a party-line response of the form, "Ah, but if it's been decreed by authorial fiat that these-and-such probabilities and utilities take such-and-these values, then in this case, self-knowledge is actually bad under the utilitarian calculus," I wasn't disputing the utilitarian calculus. I was wondering—here I used the "🐛" bug emoji customarily used in Glowfic culture to indicate uncertainty about the right words to use—_who destroyed your souls?_
+
+Yudkowsky replied:
+
+> it feels powerfully relevant to me that the people of whom I am saying this _are eliezera_. I get to decide what they'd want because, unlike with Earth humans, I get to put myself in their shoes. it's plausible to me that the prediction markets say that I'd be sadder if I was exposed to the concept of sadism in a world with no masochists. if so, while I wouldn't relinquish my Art and lose my powers by trying to delude myself about that once I'd been told, I'd consider it a friendly act to keep the info from me—_because_ I have less self-delusional defenses than a standard Earthling, really—and a hostile act to tell me; and if you are telling me I don't get to make that decision for myself because it's evil, and if you go around shouting it from the street corners in dath ilan, then yeah I think most cities don't let you in.
+
+I wish I had thought to ask if he'd have felt the same way in 2008.
+
+Ajvermillion was still baffled at my skepticism: if the author specifies that the world of the story is simple in this-and-such direction, on what grounds could I _disagree_?
+
+I admitted, again, that there was a sense in which I couldn't argue with authorial fiat. But I thought that an author's choice of assumptions reveals something about what they think is true in our world, and commenting on that should be fair game for literary critics. Suppose someone wrote a story and said, "in the world portrayed in this story, everyone is super-great at _kung fu_, and they could beat up everyone from our Earth, but they never have to practice at all."
+
+(Yudkowsky retorted, "...you realize you're describing like half the alien planets in comic books? when did Superman ever get depicted as studying kung fu?" I wish I had thought to admit that, yes, I _did_ hold Eliezer Yudkowsky to a higher standard of consilient worldbuilding than DC Comics. Would he rather I _didn't_?)
+
+Something about innate _kung fu_ world seems fake in a way that seems like a literary flaw. It's not just about plausibility. Fiction often incorporates unrealistic elements in order to tell a story that has relevance to real human lives. Innate _kung fu_ skills are scientifically plausible[^instinct] in a way that faster-than-light travel is not, but throwing faster-than-light travel into the universe so that you can do a [space opera](https://tvtropes.org/pmwiki/pmwiki.php/Main/SpaceOpera) doesn't make the _people_ fake in the way that Superman's fighting skills are fake.
+
+[^instinct]: All sorts of other instinctual behaviors exist in animals; I don't se why skills humans have to study for years as a "martial art" couldn't be coded into the genome.
+
+Maybe it was okay for Superman's fighting skills to be fake from a literary perspective (because realism along that dimension is not what Superman is _about_), but if the Yudkowskian ethos exulted intelligence as ["the power that cannot be removed without removing you"](https://www.lesswrong.com/posts/SXK87NgEPszhWkvQm/mundane-magic), readers had grounds to demand that the dath ilani's thinking skills be real, and a world that's claimed by authorial fiat to be super-great at epistemic rationality, but where the people don't have a will-to-truth stronger than their will-to-happiness, felt fake to me. I couldn't _prove_ that it was fake. I agreed with Harmless's case that, _technically_, as far as the Law went, you could build a Civilization or a Friendly AI to see all the ugly things that you preferred not to see.
+
+But if you could—would you? And more importantly, if you would—could you?
+
+It was possible that the attitude I was evincing here was just a difference between the eliezera out of dath ilan and the Zackistani from my medianworld, and that there was nothing more to be said about it. But I didn't think the thing was a _genetic_ trait of the Zackistani! _I_ got it from spending my early twenties obsessively re-reading blog posts that said things like, ["I believe that it is right and proper for me, as a human being, to have an interest in the future [...] One of those interests is the human pursuit of truth [...] I wish to strengthen that pursuit further, in this generation."](https://www.lesswrong.com/posts/anCubLdggTWjnEvBS/your-rationality-is-my-business)
+
+There were definitely communities on Earth where I wasn't allowed in because of my tendency to shout things from street corners, and I respected those people's right to have a safe space for themselves.
+
+But those communities ... didn't call themselves _rationalists_, weren't _pretending_ be to be inheritors of the great tradition of E. T. Jaynes and Robin Dawes and Richard Feynman. And if they _did_, I think I would have a false advertising complaint against them.
+
+"[The eleventh virtue is scholarship. Study many sciences and absorb their power as your own](https://www.yudkowsky.net/rational/virtues) ... unless a prediction market says that would make you less happy," just didn't have the same ring to it. Neither did "The first virtue is curiosity. A burning itch to know is higher than a solemn vow to pursue truth. But higher than both of those, is trusting your Society's institutions to tell you which kinds of knowledge will make you happy"—even if you stipulated by authorial fiat that your Society's institutions are super-competent, such that they're probably right about the happiness thing.
+
+Attempting to illustrate [the mood I thought dath ilan was missing](https://www.econlib.org/archives/2016/01/the_invisible_t.html), I quoted (with Discord's click-to-reveal spoiler blocks around the more plot-relevant sentences) the scene from _Atlas Shrugged_ where our heroine Dagny expresses a wish to be kept ignorant for the sake of her own happiness, and gets shut down by John Galt—and Dagny _thanks_ him.[^atlas-shrugged-ref]
+
+> "[...] Oh, if only I didn't have to hear about it! If only I could stay here and never know what they're doing to the railroad, and never learn when it goes!"
+>
+> "You'll have to hear about it," said Galt; it was that ruthless tone, peculiarly his, which sounded implacable by being simple, devoid of any emotional value, save the quality of respect for facts. "You'll hear the whole course of the last agony of Taggart Transcontinental. You'll hear about every wreck. You'll hear about every discontinued train. You'll hear about every abandoned line. You'll hear about the collapse of the Taggart Bridge. Nobody stays in this valley except by a full, conscious choice based on a full, conscious knowledge of every fact involved in his decision. Nobody stays here by faking reality in any manner whatever."
+>
+> She looked at him, her head lifted, knowing what chance he was rejecting. She thought that no man of the outer world would have said this to her at this moment—she thought of the world's code that worshipped white lies as an act of mercy—she felt a stab of revulsion against that code, suddenly seeing its full ugliness for the first time [...] she answered quietly, "Thank you. You're right."
+
+[^atlas-shrugged-ref]: In Part Three, Chapter II, "The Utopia of Greed".
+
+This (probably predictably) failed to resonate with other server participants, who were baffled why I seemed to be appealing to Ayn Rand's authority.
+
+I was actually going for a _reverse_ appeal-to-authority: if _Ayn Rand_ understood that facing reality is virtuous, why didn't the 2020s "rationalists"? Wasn't that undignified? I didn't think the disdain for "Earth people" (again, as if there were any other kind) was justified, when Earth's philosophy of rationality (as exemplified by Ayn Rand or Robert ["Get the Facts"](https://www.goodreads.com/quotes/38764-what-are-the-facts-again-and-again-and-again) Heinlein) was doing better than dath ilan's on this critical dimension.
+
+But if people's souls had been damaged such that they didn't have the "facing reality is virtuous" gear, it wasn't easy to install the gear by talking at them.
+
+Why was I so sure _my_ gear was correct?
+
+I wondered if the issue had to do with what Yudkowsky had [identified as the problem of non-absolute rules](https://www.lesswrong.com/posts/xdwbX9pFEr7Pomaxv/meta-honesty-firming-up-honesty-around-its-edge-cases#5__Counterargument__The_problem_of_non_absolute_rules_), where not-literally-absolute rules like "Don't kill" or "Don't lie" have to be stated _as if_ they were absolutes in order to register to the human motivational system with sufficient force.
+
+Technically, as a matter of decision theory, "sacred values" are crazy. It's easy to say—and feel with the passion of religious conviction—that it's always right to choose Truth and Life, and that no one could choose otherwise except wrongly, in the vile service of Falsehood and Death. But reality presents us with quantitative choices over uncertain outcomes, in which everything trades off against everything else under the [von Neumann–Morgenstern axioms](https://en.wikipedia.org/wiki/Von_Neumann%E2%80%93Morgenstern_utility_theorem); if you had to choose between a small, unimportant Truth and the Life of millions, you'd probably choose Life—but more importantly, the very fact that you might have to choose, means that Truth and Life can't both be infinitely sacred to you, and must be measured on a common scale with lesser goods like mere Happiness.
+
+I knew that. The other people in the chatroom knew that. So to the extent that the argument amounted to me saying "Don't lie" (about the existence of masochism), and them saying "Don't lie unless the badness of lying is outweighed by the goodness of increased happiness", why was I so confident that I was in the right, when they were wisely acknowledging the trade-offs under the Law, and I was sticking to my (incoherent) sacred value of Truth? Didn't they obviously have the more sophisticated side of the argument?
+
+The problem was that, in my view, the people who weren't talking about Truth as if it were a sacred value were being _wildly recklessly casual_ about harms from covering things up, as if they didn't see the non-first-order harms _at all_. I felt I had to appeal to the lessons for children about how Lying Is Bad, because if I tried to make a more sophisticated argument about it being _quantitatively_ crazy to cover up psychology facts that make people sad, I would face a brick wall of "authorial fiat declares that the probabilities and utilities are specifically fine-tuned such that ignorance is good".
+
+Even if you specified by authorial fiat that "latent sadists could use the information to decide whether or not to try to become rich and famous" didn't tip the utility calculus in itself, [facts are connected to each other](https://www.lesswrong.com/posts/wyyfFfaRar2jEdeQK/entangled-truths-contagious-lies); there were _more consequences_ to the coverup, more ways in which better-informed people could make better decisions than worse-informed people.
+
+What about the costs of all the other recursive censorship you'd have to do to keep the secret? (If a biography mentioned masochism in passing along with many other traits of the subject, you'd need to either censor the paragraphs with that detail, or censor the whole book. Those are real costs, even under a soft-censorship regime where people can give special consent to access "Ill Advised" products.) Maybe latent sadists could console themselves with porn if they knew, or devote their careers to making better sex robots, just as people on Earth with non-satisfiable sexual desires manage to get by. (I _knew some things_ about this topic.) What about dath ilan's heritage optimization (read: eugenics) program? Are they going to try to breed more masochists, or fewer sadists, and who's authorized to know that? And so on.
+
+Or imagine a world where male homosexuality couldn't be safely practiced due to super-AIDS. (I know very little about BDSM.) I still think men with that underlying predisposition would be better off _having a concept_ of "homosexuality" (even if they couldn't practice it), rather than the concept itself being censored. There are also other systematic differences that go along with sexual orientation (the "feminine gays, masculine lesbians" thing); if you censor the _concept_, you're throwing away that knowledge.
+
+(When I had brought up the super-AIDS hypothetical in the chat, Ajvermillion complained that I was trying to bait people into self-cancelling by biting the bullet on suppressing homosexuality. I agreed that the choice of example was engineered to activate people's progressive moral intuitions about gay rights—it was great for him to notice that—but I thought that colliding philosophical intuitions like that was intellectually productive; it wasn't an attempt to gather blackmail material.)
+
+A user called RationalMoron asked if I was appealing to a terminal value. Did I think people should have accurate self-models even if they didn't want to?
+
+Obviously I wasn't going to use a universal quantifier over all possible worlds and all possible minds, but in human practice, yes: people who prefer to believe lies about themselves are doing the wrong thing; people who lie to their friends to keep them happy are doing the wrong thing. People can stand what is true, because they are already doing so. I realized that this was a children's lesson without very advanced math, but I thought it was a better lesson than, "Ah, but what if a _prediction market_ says they can't???"
+
+I maintained that the fact that the eliezera prefer not to know that there are desirable sexual experiences that they can't have, contradicted April's earlier claim (which had received a Word of God checkmark-emoji) that "it's not that the standards are being dropped[;] it's that there's an even higher standard far beyond what anyone on earth has accomplished".
+
+Apparently I struck a nerve. Yudkowsky started "punching back":
+
+> **Eliezer** — 12/08/2022 12:45 PM
+> Do zacki have no concept of movie spoilers, such that all movies are just designed not to rely on uncertainty for dramatic tension? Do children have to be locked in individual test rooms because they can't comprehend the concept of refusing to look at other children's answer sheets because it's evidence and you should observe it? Do adults refuse to isolate the children so they can have practice problems, because you can't stop them from learning the answer to skill-building problems, only the legendary evil alien eliezera would do that? Obviously they don't have surprise parties.
+> It's noticeably more extreme than the _Invention of Lying_ aliens, who can still have nudity taboos
+> I'd also note that I think in retrospect (only after having typed it) that Zack could not have generated these examples of other places where society refrains from observation, and that I think this means I am tracking the thing Zack fears in a way that Zack cannot because his thinking is distorted and he is arguing rather than seeing; and this, not verbally advocating for "truth", is more what respect for truth really is.
+
+I thought the "you could not have generated the answer I just told you" gambit was a pretty dirty argumentative trick on Yudkowsky's part. (Given that I could, how would I be able to prove it?—this was itself a good use-case for concealing spoilers.)
+
+As it happened, however, I _had_ already considered the case of spoilers as a class of legitimate infohazards, and was prepared to testify that I had already thought of it, and explain why I thought hiding spoilers were relevantly morally different from the coverups I was objecting to. The previous night, 7 December 2022, I had had a phone call with Anna Salamon,[^evidence-of-independent-generation] in which I (remembered that I) had cited dath ilan's [practice of letting children figure out heliocentrism for themselves](https://www.glowfic.com/replies/1777588#reply-1777588) as not being objectionable in the way the sadism/masochism coverup was.
+
+[^evidence-of-independent-generation]: I was lucky to be able to point to Anna as a potential witness to defend myself against the "could not have generated" trick—as a matter of principle, not because I seriously expected anyone to care enough to go ask Anna if she remembered the conversation the same way.
+
+ I also mentioned that when I had used spoiler blocks on the _Atlas Shrugged_ quote I had posted upthread, I had briefly considered making some kind of side-remark noting that the spoiler blocks were also a form of information-hiding, but couldn't think of anything funny or relevant enough (which, if my self-report could be trusted, showed that I had independently generated the idea of spoilers being an example of hiding information—but I didn't expect other people to uncritically believe my self-reports).
+
+It seemed like the rationale for avoiding spoilers of movie plots or homework exercises had to do with the outcome being different if you got spoiled: you have a different æsthetic experience if you experience the plot twist in the 90th minute of the movie rather than the fourth paragraph of the _Wikipedia_ article. Dath ilan's sadism/masochism coverup didn't seem to have the same structure: when I try to prove a theorem myself before looking at how the textbook says to do it, it's not because I would be _sad about the state of the world_ if I looked at the textbook; it's because the temporary ignorance of working it out myself results in a stronger state of final knowledge.
+
+That is, the difference between "spoilers" (sometimes useful) and "coverups" (bad) had to do with whether the ignorant person is expected to eventually uncover the hidden information, and whether the ignorant person knows that there's hidden information that they're expected to uncover. In the case of the sadism/masochism coverup (in contrast to the cases of movie spoilers or homework exercises), it seemed like neither of these conditions pertained. (Keltham knows that the Keepers are keeping secrets, but he seems to actively have beliefs about human psychology that imply masochism is implausible; it seems more like he has a false map, rather than a blank spot on his map for the answer to the homework exercise to be filled in.) I thought that was morally relevant.
+
+(Additionally, I would have hoped that my two previous mentions in the thread of supporting keeping nuclear, bioweapon, and AI secrets should have already made it clear that I wasn't against _all_ cases of Society hiding information, but to further demonstrate my ability to generate counterexamples, I mentioned that I would also admit _threats_ as a class of legitimate infohazard: if I'm not a perfect decision theorist, I'm better off if Tony Soprano just doesn't have my email address to begin with, if I don't trust myself to calculate when I "should" ignore his demands.)
+
+As for the claim that my thinking was distorted and I was arguing instead of seeing, it was definitely true that I was _motivated to look for_ criticisms of Yudkowsky and dath ilan, for personal reasons outside the scope of the server, and I thought it was great for people to notice this and take it into account. I hoped to nevertheless be competent to only report real criticisms and not fake criticisms. (Whether I succeeded, of course, was up to the reader to decide.)
+
+Yudkowsky replied:
+
+> only half the battle even if you could do it. you're also not reporting any facts/arguments on the other side, which is a much larger and visible gap to me, and has a lot to do with why I'm not presently considering this criticism from a peer despite your spoken adherence to virtues I value. **QUESTION FOR ZACK ONLY, NOBODY ELSE ANSWER OR SAY ANYTHING ABOUT IT IN THIS MAIN CHANNEL:** What are some of the ways that Planecrash valorizes truth, as you, yourself, see that virtue?
+
+I didn't ask why it was relevant whether or not I was a "peer." If we're measuring IQ (143 _vs._ [131](/images/wisc-iii_result.jpg)), or fiction-writing ability (several [highly-acclaimed](https://www.lesswrong.com/posts/HawFh7RvDM4RyoJ2d/three-worlds-collide-0-8) [stories](https://www.yudkowsky.net/other/fiction/the-sword-of-good) [including the world's most popular _Harry Potter_ fanfiction](https://www.hpmor.com/) _vs._ a [_My Life as a Teenage Robot_ fanfiction](https://archive.ph/WdydM) with double-digit favorites and a [few](/2018/Jan/blame-me-for-trying/) [blog](http://zackmdavis.net/blog/2016/05/living-well-is-the-best-revenge/) [vignettes](https://www.lesswrong.com/posts/dYspinGtiba5oDCcv/feature-selection) here and there), or contributions to AI alignment (founder of the field _vs._ author of some dubiously relevant blog comments), I'm obviously _not_ his peer. It didn't seem like that was necessary when one could just [evaluate my arguments about dath ilan on their own merits](https://www.lesswrong.com/posts/5yFRd3cjLpm3Nd6Di/argument-screens-off-authority). But I wasn't going to be so impertinent to point that out when the master was testing me (!) and I was eager to pass the test.
+
+I said that I'd like to take an hour to compose a _good_ answer. (It was 10:26 _p.m._) If I tried to type something off-the-cuff on the timescale of five minutes, it wasn't going to be of similar quality as my criticisms, because, as I had just admitted, I had _totally_ been running a biased search for criticisms—or did the fact that I had to ask that mean I had already failed the test?
+
+Yudkowsky replied:
+
+> I mean, yeah, in fact the greater test is already having that info queued, but conversely it's even worse if you think back or reread and people are not impressed with the examples you find. I cannot for politeness lie and deny that if you did it in five minutes it would be _more_ impressive, but I think that it is yet the correct procedure to take your time.
+
+(As an aside—this isn't something I thought or said at the time—I _do_ think it makes sense to run an asymmetric search for flaws in some contexts, even though it would be disastrous to only look on one side of the argument when considering a belief you're uncertain about. Code reviewers often only comment in detail on flaws or bugs that they find, and say only "LGTM" (looks good to me) when they don't find any. Why? Because the reviewers aren't necessarily trying to evaluate "This code is good" as an abstract belief[^low-stakes]; they're trying to improve the code, and there's an asymmetry in payoffs where eliminating a flaw is an improvement, whereas identifying something the code does right just means the author was doing their job. If you didn't trust a reviewer's competence and thought they were making spurious negative reviews, you might legitimately test them by asking them to argue what's _good_ about a pull request that they just negatively reviewed, but I don't think it should be concerning if they asked for some extra time.)
+
+[^low-stakes]: For typical low-stakes business software in the "move fast and break things" regime. In applications where bugs are more costly, you do want to affirmatively verify "the code is good" as a belief.
+
+I said that I also wanted to propose a reframing: the thing that the present thread was complaining about was a lack of valorization of truth-_telling_, honesty, wanting _other_ people to have accurate maps. Or maybe that was covered by "as you, yourself, see that virtue"?
+
+Yudkowsky said that he would accept that characterization of what the thread was about if my only objection was that dath ilan didn't tell Keltham about BSDM, and that I had no objection to Keltham's judgement that in dath ilan, he would have preferred not to know.
+
+I expounded for some more paragraphs about why I _did_ object to Keltham's judgement, and then started on my essay exam—running with my "truth-telling" reframing.
+
+I wanted to nominate the part where the Conspiracy is unveiled—I thought I remembered Keltham saying something about how Carissa's deception was the worst thing anyone could have done to him—that is, the fact that someone he trusted was putting him in a fake reality was _itself_ considered a harm, separately from the fact that Cheliax is evil. I re-read pages 74 onwards of the ["What the Truth Can Destroy"](https://www.glowfic.com/posts/5930) thread, and didn't see Keltham saying the thing I thought he said (maybe it happened in the next thread, or I had misremembered), but found two more things to submit as answers to my lit exam, which I posted at 12:30 _a.m._ (so I had actually taken two hours rather than the one I had asked for).
+
+First, I liked how [Snack Service intervenes to stage](https://www.glowfic.com/replies/1811461#reply-1811461) a "truth and reconciliation commission" for Keltham and his paramours, on the grounds that it's necessary for Asmodeus and Cayden Caliean and Adabar and Keltham to make their best decisions. People testifying in public (with the Chelaxians and Oririons present, as one would at a trial) reflects a moral about the importance of common knowledge, _shared_ maps. The testimony being public ensured that not just that Keltham got to know what's been done to him, but that his paramours and counterparties _know that he knows_. There was something honorable about getting things on the public record like that, in the end, even while Snack Service was willing to participate in the conspiracy _before_ the jig was up.
+
+Second, I liked Korva's speech about why she hates Keltham, and how Keltham not only takes it in stride, but also asks to buy the right to take Korva with him to Osirion. When Abrogail expresses surprise that Keltham would want Korva, Keltham cites a dath ilani proverb about advice that's easier to get from people who aren't friends with you. This reflects an understanding that your friends wanting to be nice to you can be a source of distortions; Keltham specifically values Korva _as a critic_.
+
+The next day, I added that I realized that I had missed a huge opportunity to successfully reply on a five-minute time scale (to pass "the greater test [of] already having that info queued"): the "in _Planecrash_" part of the prompt made me think I had to find something in Keltham's story (which is why I took another two hours to hand in my essay), but other threads within the dath ilan Glowfic continuity should obviously count for the purpose of the test, and I did in fact already have cached thoughts about how Thellim's contempt for Jane Austen characters beautifully mirrored my contempt for protecting people from psychology facts that would hurt their feelings. I could _prove_ that I already had it cached (if not queued, as evidenced by my remembering it the next day), because I had mentioned it both in the conversation leading to the present thread, and in my memoir draft.
+
+Yudkowsky replied:
+
+> so I think that you're looking an awful lot at what _characters say_ and nearly not at all at what the universe does. this plausibly reflects a deep flaw in your art, because it sure does seem to me that you are a lot better at noticing what people say about truth in words, detecting whose monkey-side they seem to be on, than you are imo at carefully weighing up both sides of things as is the art of finding-truth-in-reality. it plausibly also reflects some people who ill-shaped you, pointing you at the fictional characters and angering you at their spoken words and verbal thoughts, as was advantageous to them, and not pointing you towards, like, looking at the messages in the fiction itself rather than the words spoken by characters, because that would not have served their ill purpose of alienating you and turning you into an angry thing more useful for their purposes. (I would not ordinarily use language like this but I regret that it is the language you have now seemingly been ill-shaped to speak, for another's usefulness.)
+> if I ask you, not what any _character says_, not even what any _societies say_, but _what happens in Planecrash_ and what the _causal process_ there seems to think about matters important to you, what do you see?
+
+As a _quick_ reply to the followup question (posted within 19 minutes of it being asked), I said that Cheliax was at a structural disadvantage in its conflict with the forces of Good, because learning how to think inevitably turns mortals away from Asmodeus's will.
+
+But I was _more_ interested in replying to the part about me being ill-shaped to another's purpose. (I said that I wouldn't have considered that on-topic for the fiction server, but if _he_ thought it was on-topic, then it made sense for me to reply—and I did so at 12:26 _p.m._ the next day, after some time to think. Discord lends itself quite well to a mix of synchronous and asynchronous communication, depending on when people happen to be at their computers.)
+
+I said that he seemed _really_ stuck on this hypothesis that it was Michael Vassar's fault that I'd been shaped into an alienated and angry thing.
+
+To be clear, I totally agreed that I had been shaped into an alienated and an alienated and angry thing. Obviously. But speaking of people "look[ing] inside themselves and correctly see[ing] that this is not how they work" (as Yudkowsky had said earlier), I thought he was getting the causality all wrong.
+
+It seemed to _me_ that the reason I had become an alienated and angry thing is because I had been shaped by [making an extraordinary effort](https://www.lesswrong.com/posts/GuEsfTpSDSbXFiseH/make-an-extraordinary-effort) since 2016 to respond to a class of things that included Yudkowsky "mak[ing] up sophisticated stories for why pretty obviously true things are false"—again referencing Oliver Habryka's comment on "Challenges to Yudkowsky's Pronoun Reform Proposal."
+
+That's the context in which it wasn't surprising that my Art had involved some amount of specialization in "detecting whose monkey-side they seem to be on." In a world where monkeys are trying to cover up otherwise-obvious truths, successfully blowing the whistle on them involves being sensitive to their monkey games; figuring out the truth they're trying to cover up is the easy part. The whistleblowing-skill of of promoting otherwise-obvious things to _common_ knowledge in opposition to a Power trying to prevent common knowledge, is different from the science-skill of figuring out organically-nonobvious things from scratch. It _makes sense_ for Alexander Solzhenitsyn and Andrey Kolmogorov—or for that matter, John Galt and Robert Stadler—to have developed different crystalized skills.
+
+(Indeed, it even makes sense for Kolmogorov and Stadler to _not_ develop some skills, because the skills would show up under Detect Thoughts.)
+
+If it was all Michael's fault for "extensively meta-gas[lighting me] into believing that everyone generally and [him] personally [were] engaging in some kind of weird out-in-the-open gaslighting", I asked, echoing Yudkowsky's language from earlier (with appropriate quotation marks and brackets),[^gaslighting] then _how come Oli could see it, too?_
+
+[^gaslighting]: In particular, Yudkowsky was the one who introduced the word _gaslighting_ into the conversation; I hadn't previously used the word myself.
+
+Yudkowsky replied:
+
+> I think if you asked Oli about the state of reality with respect to this whole affair, he'd have a very different take from your take, _if you're still able to hear differences instead of only those similarities you demand._
+
+That sounded like an easy enough experimental test! I wrote Habryka an email explaining the context, and asking him what "very different take" he might have, if any. (I resisted the temptation to start a [Manifold market](https://manifold.markets/) first.) As I mentioned in the email, I didn't expect to have a very different take from Habryka _about the state of reality_. ("Zack is (still?!) very upset about this, but Oli mostly doesn't care" isn't a disagreement about the state of reality.) I didn't think I disagreed with _Yudkowsky_ much about the state of reality.
+
+He didn't reply. (I might have guessed the wrong email address, out of the two I had on file for him?) I don't blame him. It might have been timelessly ungrateful of me to ask. (The reason people are reluctant to make on-the-record statements in politically charged contexts is because they're afraid the statements will be _used_ to drag them into more political fights later. He had already done me a huge favor by being brave enough to state the obvious in March; I had no right to demand anything more of him.)
+
+Regarding my quick reply about Cheliax's structural disadvantage, Yudkowsky said it was "okay as one element", but complained that the characters had already observed it out loud, and that I "didn't name any concrete sequence of events that bore it out or falsified it." He continued:
+
+> I think you could find a lot more than this if your brain were still able to see incongruent facts as well as only congruent facts. what does the underlying reality of Planecrash think about your Most Important Issues? what is shown, but maybe never even told at all? you gave the reply of somebody who can _only_ see social realities and _only_ what people say and not what _just happens, at all_, even inside a story, you didn't say _a thing that happened._
+
+At this point, I was a bit suspicious that _any_ answer that wasn't exactly whatever he was thinking of would be dismissed as too social or too inferentially close to something one of the characters had said. What did it mean for the _universe_ to say something about valorizing truth?
+
+The original prompt ("What are some of the ways _Planecrash_ valorizes truth") had put me into 11th-grade English class mode; the revision "if I ask you, not what any _character_ says [...]" made me think the 11th-grade English teacher expected a different answer. The revised–revised prompt "what does the underlying reality of _Planecrash_ think about your Most Important Issues?", with the previous rebukes in my context window, made me think I should be reaching for an act of philosophical [Original Seeing](https://www.lesswrong.com/posts/SA79JMXKWke32A3hG/original-seeing), rather than trying to be a diligent schoolstudent playing the 11th-grade English class game. I pondered the matter ... and I _thought of something_.
+
+_Thesis_: the universe of _Planecrash_ is saying that virtue ethics—including, as a special case, my virtue ethics about it being good to tell the truth and reveal information—are somewhat unnatural.
+
+In the story, the god Adabar values trading fairly, even with those who can't verify their partners are keeping up their end of the deal, and also wants to promote fair trading _elsewhere_ in Reality (as contrasted to just being fair Himself).
+
+Adabar is kind of a weirdo. He's not a vanishly rare freak (whose specification would require lots of uncompressible information); there _is_ a basin of attraction in the space of pre-gods, where creatures who develop a computationally efficient "fairness" heuristic in their ancestral environment, reify that into their utilityfunction when they ascend, but it's not a _huge_ basin of attraction; most gods aren't like Adabar.
+
+It's the same thing with honesty. Generic consequentialists have no reason to "tell the truth" to agents with different utility functions when they're not under compact and being compensated for the service. Why _would_ you emit signals that other agents can interpret as a map that reflects the territory? [You can't get more paperclips that way!](https://arbital.com/p/not_more_paperclips/)
+
+I had previously written about this in ["Commucation Requires Common Interests or Differential Signal Costs"](https://www.lesswrong.com/posts/ybG3WWLdxeTTL3Gpd/communication-requires-common-interests-or-differential); you needed some common interests in order for flexible, "digital" language to exist at all. ("Digital" language where the relationship between signals and meaning can be arbitrary, in contrast to costly signaling, where me expending resources at least tell you that I could afford those resources.)
+
+It's _possible_ for imperfectly deceptive social organisms to develop a taste for "honesty" as a computationally efficient heuristic for navigating to Pareto improvements in the ancestral environment, which _might_ get reified into the utilityfunction as they ascend—but that's an Adabar-class weird outcome, not the default outcome.
+
+So—insofar as my Most Important Issues revolved around an obsession with motivational transparency, wanting to live in a world that wasn't lying to me, wanting to _reveal information_ as an end in itself, unilaterally rather than only as part of a coordinated negotiation scheme, without necessarily being _paid_ for it, but just because it is _right_ ...
+
+It seems like my answer to the question of, "What does the underlying causal process of _Planecrash_ think about your Most Important Issues; what are some the ways that _Planecrash_ valorizes truth-telling as you, yourself, see that virtue?" is, "It doesn't" (!). Truth-telling is a virtue ethic, and _Planecrash_ depicts a universe ruled by consequentialist gods who only implement virtue ethics insofar as that made it into their utilityfunction.
+
+I realized, of course, that this certainly wasn't the answer Yudkowsky was looking for. But it seemed like a _better_ answer than me trying to play the schoolstudent. He asked what I saw when I looked at what the fictional universe was saying about _my_ problems, and I looked, and _I saw something_. (Something philosophically substantive, not just a social reality.) It seemed more honest to just report that, rather than try to [guess the teacher's password](https://www.lesswrong.com/posts/NMoLJuDJEms7Ku9XS/guessing-the-teacher-s-password). So, after sleeping on it first, I posted the explanation of what I saw to the channel (including the parts about how the original prompts steered me, and that I realized that this wasn't the answer he was looking for).
+
+[TODO:
+ * third bluebook didn't get a response
+ * I was feeling kind of glum about my poor performance; I had reasons that made sense to me for why I said the things I said, but that _wasn't what I was being tested on_; I definitely _didn't_ say "Nothing, because Big Yud and Linta are lying liars who hate truth"; I moped in Alicorner #drama
+ * Big Yud saw it (I know this because he left an emoji react in the relevant section of messages) and took pity on me
+]
+
+[TODO: derail with Lintamande]
+
+[TODO: knives, and showing myself out]
+
+------
+
+Anyway, that—briefly (I mean it)—is the Whole Dumb Story about how I wasted the last seven years of my life. It's probably not that interesting? Life goes on—for now. My dayjob contract expired at the end of 2022. In 2023, I've been finishing up this memoir, and posting some other ideas to _Less Wrong_. (I got into another slapfight about me being un-collaborative, which is not interesting enough to summarize.)
+
+After this, the AI situation is looking worrying enough, that I'm thinking I should try to do some more direct xrisk-reduction work, although I haven't definitely selected any particular job or project. (It probably won't matter, but it will be dignified.) Now that the shape of the threat is on the horizon, I think I'm less afraid of being directly involved. Something about having large language models to study in the 'twenties is—grounding, compared to the superstitious fears of the paperclip boogeyman of my nightmares in the 'teens.
+Like all intellectuals, as a teenager I imagined that I would write a book. It was always going to be about gender, but I was vaguely imagining a novel, which never got beyond vague imaginings. That was before the Sequences. I'm 35 years old now. I think my intellectual life has succeeded in ways I didn't know how to imagine, before. I think my past self would be proud of this blog—140,000 words of blog posts stapled together is _morally_ a book—once he got over the shock of heresy.
-[TODO: regrets and wasted time
+[TODO conclusion, cont'd—
* Do I have regrets about this Whole Dumb Story? A lot, surely—it's been a lot of wasted time. But it's also hard to say what I should have done differently; I could have listened to Ben more and lost faith Yudkowsky earlier, but he had earned a lot of benefit of the doubt?
+ * even young smart AGPs who can appreciate my work have still gotten pinkpilled
+ * Jonah had told me that my planning horizon was too short—like the future past a year wasn't real to me. (This plausibly also explains my impatience with college.) My horizon is starting to broaden as AI timelines shorten
* less drama (in my youth, I would have been proud that at least this vice was a feminine trait; now, I prefer to be good even if that means being a good man)
]
projects / Ultimately_Untrue_Thought.git / blobdiff