> When you're doubting one of your most cherished beliefs, close your eyes, empty your mind, grit your teeth, and **deliberately think about whatever hurts the most**. Don't rehearse standard objections whose standard counters would make you feel better. Ask yourself what _smart_ people who disagree would say to your first reply, and your second reply. Whenever you catch yourself flinching away from an objection you fleetingly thought of, drag it out into the forefront of your mind. **Punch yourself in the solar plexus. Stick a knife in your heart, and wiggle to widen the hole.** In the face of the pain, rehearse only this:
>
-> What is true is already so.
-> Owning up to it doesn't make it worse.
-> Not being open about it doesn't make it go away.
-> And because it's true, it is what is there to be interacted with. Anything untrue isn't there to be lived.
-> People can stand what is true, for they are already enduring it.
+>> What is true is already so.
+>> Owning up to it doesn't make it worse.
+>> Not being open about it doesn't make it go away.
+>> And because it's true, it is what is there to be interacted with. Anything untrue isn't there to be lived.
+>> People can stand what is true, for they are already enduring it.
Meanwhile, the dath ilan mythos depicts rationality itself as hazardous knowledge that people need to be protected from. Dath ilani Civilization is steered by a secretive order of [Keepers of Highly Unpleasant Things it is Sometimes Necessary to Know](https://www.glowfic.com/replies/1612937#reply-1612937). ["The universe is not so dark a place that everyone needs to become a Keeper to ensure the species's survival,"](https://glowfic.com/replies/1861879#reply-1861879) we're told. "Just dark enough that some people ought to." Ordinary dath ilani do receive rationality training, but it's implied to be deliberately crippled, featuring ["signposts around the first steps [towards becoming a Keeper], placed to warn dath ilani off starting down that path unless they mean it."](https://www.glowfic.com/replies/1799590#reply-1799590) The maxim that "That which can be destroyed by the truth should be" is described as being ["remembered as much for how it's false, as for how it's true, because among the things that truths can destroy is people."](https://www.glowfic.com/replies/1687922#reply-1687922)
Clearly, this is not a culture that cares about ordinary people being well-informed. Apparently, they believe that owning up to it _does_ make it worse, that the untrue _is_ there to be lived (for non-Keepers).
-We might say that the algorithm that designed dath ilan's Civilization can be seen as systematically preferring deception. When I speak of an algorithm preferring deception, [what I mean is](https://www.lesswrong.com/posts/fmA2GJwZzYtkrAKYJ/algorithms-of-deception) that given a social problem, candidate solutions that involve deceiving the populace seem to be higher in dath ilani Civilization's implicit search ordering than solutions that involve informing the populace. Solutions that work by means of telling the truth will be implemented only when solutions that work by means of deception are seen to fail.
+We might say that the algorithm that designed dath ilan's Civilization can be seen as systematically preferring deception. When I speak of an algorithm preferring deception, [what I mean is](https://www.lesswrong.com/posts/fmA2GJwZzYtkrAKYJ/algorithms-of-deception) that given a social problem, candidate solutions that involve deceiving the populace seem to be higher in dath ilani Civilization's implicit search ordering than solutions that involve informing the populace. If a solution adequately achieves its purpose by means of deception or secrecy, Civilization isn't going to reject it on that basis and keep searching for a solution that involves telling the truth.
Crucially, these are [functionalist](https://www.lesswrong.com/posts/sXHQ9R5tahiaXEZhR/algorithmic-intent-a-hansonian-generalized-anti-zombie) [criteria](http://benjaminrosshoffman.com/bad-faith-behavior-not-feeling/) of "preference" and "deception". It's about how Civilization is structured in a way that systematically encourages divergences between popular belief and reality. I'm _not_ positing that Civilization's Keepers and Legislators and Chief Executive are laughing maniacally and telling each other out loud, "I personally love it when non-Keepers have false beliefs; we need to do as much of that as possible—as a [terminal value](https://www.readthesequences.com/Terminal-Values-And-Instrumental-Values)!"
But the craft of literature isn't a matter of merely conveying a fictional reality that existed fully formed in the author's imagination in advance of writing it down. The craft is about producing text that readers can use to build up their own model of the fictional world. The exacting labor of [converting vague ideas into definite text](http://www.paulgraham.com/words.html) is the difference between writing and daydreaming. We can accept [Word of God](https://tvtropes.org/pmwiki/pmwiki.php/Main/WordOfGod) as supplementary material where the text of a story is ambiguous or silent on a point of interest, but some kind of [Death of the Author](https://tvtropes.org/pmwiki/pmwiki.php/Main/DeathOfTheAuthor) stance is ultimately necessary for making sense of literature in a world in which telepathy doesn't exist and authors do occasionally die. The text is not a mere pointer to the "real" work inside the author's head; the text _is_ the work. That's the only way the technology of writing can function.
-Moreover, a Death of the Author stance seems particularly important for evaluating medianworlds. What makes the exercise of constructing a medianworld interesting is the challenge of envisioning the details of a _realistic_ Society that would result given a population with an alternative [distribution of cognitive repertoires](/2020/Apr/book-review-human-diversity/), but where the same generalizations of biology, sociology, and economics that govern our own world are presumed to hold. If the world portrayed by the text [doesn't seem to hold together](/2022/Jun/comment-on-a-scene-from-planecrash-crisis-of-faith/) or has [unfortunate implications](https://tvtropes.org/pmwiki/pmwiki.php/Main/UnfortunateImplications) that the author doesn't acknowledge, it's the solemn duty of literary critics to point that out to less discerning readers.
+Moreover, a Death of the Author stance seems particularly important for evaluating medianworlds. What makes the exercise of constructing a medianworld interesting is the challenge of envisioning the details of a realistic Society that would result given a population with an alternative [distribution of cognitive repertoires](/2020/Apr/book-review-human-diversity/), but where the same generalizations of biology, sociology, and economics that govern our own world are presumed to hold. If the world portrayed by the text [doesn't seem to hold together](/2022/Jun/comment-on-a-scene-from-planecrash-crisis-of-faith/) or has [unfortunate implications](https://tvtropes.org/pmwiki/pmwiki.php/Main/UnfortunateImplications) that the author doesn't acknowledge, it's the solemn duty of literary critics to point that out to less discerning readers.
In analogy, mathematicians, like authors, are also in the business of creating imaginary worlds, but mathematical objects, once defined, can be examined on their own terms. A mathematician encountering [a deeply unsatisfying theorem about their new definition](https://blog.plover.com/math/major-screwups-4.html) understands that there can be no recourse in protesting, "But that's not how _I_ imagined it working." That would be failing to engage with the real difficulties of mathematical research. If you wanted different behavior, you should have written a better definition!
Similarly, an author who says, "In _my_ medianworld, fully automated luxury gay space communism with central planning just works, because the populace is so smart and nice, unlike _Earth people_, who are so mean and dumb that they have to use _markets_ to allocate scarce resources" is failing to engage with the real difficulties of the medianworld exercise. Readers would have a right to be skeptical.
-Authors, of course, have much more wiggle room than mathematicians to try to salvage their cherished ideas. Rather than being forced back to the drawing board by an unwanted implication, a fiction writer finds it all too easy to simply add another sentence denying it. But the author's total freedom to specify the text necessarily interacts with readers' attempts to imagine a self-consistent universe that "projects into" that text. Short of an explicitly omniscient narrator declaring "And then a miracle occured", discerning readers will tend to reach for interprerations of the text that make sense—even if making sense entails casting doubt on the narrator's editorial spin on the described events.
+Authors, of course, have much more wiggle room than mathematicians to try to salvage their cherished ideas. Rather than being forced back to the drawing board by an unwanted implication, a fiction writer finds it all too easy to simply add another sentence denying it. But the author's total freedom to specify the text necessarily interacts with readers' attempts to imagine a self-consistent universe that "projects into" that text. Short of an explicitly omniscient narrator declaring "And then a miracle occured", discerning readers will tend to reach for interprerations of the text that make sense—even if making sense entails casting doubt on the narrator's spin on the described events.
-Yudkowsky's 2009 story ["The Sword of Good"](https://www.yudkowsky.net/other/fiction/the-sword-of-good) is an incisive commentary on how unwary readers' moral compasses can be hijacked by author editorializing. If the story depicts our heroes wantonly slaughtering orcs, readers tend not to worry about the ethics of warfare: if the [designated hero](https://tvtropes.org/pmwiki/pmwiki.php/Main/DesignatedHero) is doing it, it presumably isn't a problem. But living creatures that are depicted as speaking language, having organized armies with complex tools, _&c._ are presumably sapient for the same reasons humans are. It's fundamentally fair game for "The Sword of Good" to point that out—at least, short of the narrator _explicitly_ declaring, "Despite appearances, the orcs are unconscious [philosophical zombies](https://www.lesswrong.com/posts/fdEWWr8St59bXLbQr/zombies-zombies); killing them has no moral significance." (But a story that did declare that would be highly unusual, and basically conceding the critic's point about stories that didn't!)
+Yudkowsky's 2009 story ["The Sword of Good"](https://www.yudkowsky.net/other/fiction/the-sword-of-good) is an incisive commentary on how unwary readers' moral compasses can be hijacked by authorial editorializing. If the story depicts our heroes wantonly slaughtering orcs, readers tend not to worry about the ethics of warfare: if the [designated hero](https://tvtropes.org/pmwiki/pmwiki.php/Main/DesignatedHero) is doing it, it presumably isn't a problem. But living creatures that are depicted as speaking language, having organized armies with complex tools, _&c._ are presumably sapient for the same reasons humans are. It's fair game for "The Sword of Good" to point that out—at least, short of the narrator _explicitly_ declaring, "Despite appearances, the orcs are unconscious [philosophical zombies](https://www.lesswrong.com/posts/fdEWWr8St59bXLbQr/zombies-zombies); killing them has no moral significance." (But a story that did declare that would be highly unusual, and basically conceding the critic's point about stories that didn't!)
-All I'm doing in this essay is holding the dath ilan mythos to the same standard that "The Sword of Good" holds classic fantasy tropes to. Maybe it's good to run a global conspiracy to keep people ignorant in order to protect their emotional well-being, and maybe those orcs deserved to die. But given a text that _does_ literally describe massive coverups or killing of human-like creatures, it's fundamentally fair game for literary critics to point that out, and prompt readers to rethink whether they should accept "it's good because the good guys are doing it" or "it's good because it's dath ilan, which is not Earth" as an implicit excuse.
+All I'm doing in this essay is holding the dath ilan mythos to the same standard that "The Sword of Good" holds classic fantasy tropes to. Maybe it's good to run a global conspiracy to keep people ignorant in order to protect their emotional well-being, and maybe those orcs deserved to die. But given a text that _does_ literally describe massive coverups or killing of human-like creatures, it's fundamentally fair game for literary critics to point that out, and prompt readers to think carefully about whether they should accept "it's fine because the good guys are doing it" or "it's fine because it's dath ilan, which is not Earth" as an implicit excuse.
-### History Screening
+### The History Screen
-[TODO—
+One of the distinctive features of dath ilani Civilization is that they've deleted their history. No one except a few specially cleared specialists ["in their own causally isolated bunker"](https://glowfic.com/replies/1688794#reply-1688794) is allowed to learn history before a point in a relatively recent past. (["[D]ecades ago, not millennia ago"](https://glowfic.com/replies/1789110#reply-1789110) as of Keltham's time.)
+
+This is an incredibly drastic measure, a civilizational lobotomy. (It perhaps bears worth mentioning that the most famous literary depiction of a Society that deleted its history is George Orwell's _Nineteen-Eighty-Four_, in which a totalitarian state maintains a Society in which "[n]othing exists except an endless present in which the Party is always right.") The full costs may not be obvious to many readers: if the immediate needs of living in the present don't directly require thinking about the past, what's the harm of not knowing?
+
+The problem is that [facts are causally entangled with each other](https://www.readthesequences.com/Entangled-Truths-Contagious-Lies). Forgetting one fact entails not just having to deal with not knowing that particular thing, but also not knowing everything it implies about other things in the world—which might be of little consequence for any one trivium, but is surely enormous for _all of human history_.
+
+In at least one case, the text depicts dath ilan as suffering practical consequences from its ignorance: a discussion of the eliezera's low happiness levels is cut short with ["How'd it happen? Nobody knows, at this point, they screened their history."](https://glowfic.com/replies/1812613#reply-1812613) (Generally, knowing how something happened is useful for figuring out how to remediate it.) Realistically, we can only surmise that there are many other cases where knowing history would be useful that the narrator hasn't gotten around to telling us.
+
+Given the costs, what could possibly justify the history screen, in an advanced Society that otherwise seems to value knowledge? A fantasy author could easily an invent an answer: maybe a psychic plague that spread through memories such that the infected must be not just physically quarantined, but forgotten. Medianworld authors have a tougher time insofar as we construe their genre as hard science fiction. Psychic plagues aren't real. What could _realistically_ justify the history screen?
+
+The answer the text gives leaves something to be desired. In "But Hurting People Is Wrong", we learn of the birth of modern dath ilan, when the Keepers noticed the AGI alignment problem and propose strict policy measures aimed at surviving the challenge.
- * Dath ilan placed its history behind a causal screen.
- * This is a drastic measure: consider all the costs of deleting history
- * The actual explanation turns out to be incredibly casual—"As long as they were doing that anyways": the assumption that the hoi polloi are better off not knowing where they came from isn't argued for at all
- * Doylist interpretation makes sense (Keltham encountering a mideval monarchy is more interesting if he doesn't know what a Queen is), but if it's that kind of universe, what's with the comments about dath ilan being superior to Earth? Topia-for-storytelling shouldn't be hailed as a eutopia
+For one, dath ilan's eugenics program is to optimize for intelligence even more than it already does. That makes sense: smarter alignment researchers have a better shot at solving the problem.
+
+For another, restrictions on the manufacture of computers are imposed. That makes sense: if computer hardware were too capable and easily available before alignment researchers had the [serial time](https://www.lesswrong.com/posts/FS6NCWzzP8DHp4aD4/do-earths-with-slower-economic-growth-have-a-better-chance) to think the problem through, that would increase the risk of accidentally unleashing an unaligned intelligence explosion.
+
+For another, digital surveilance is imposed. That makes sense: Governance would want to know about any rogue AGI projects, so they can be shut down by force.
+
+[And then we get this](https://www.glowfic.com/replies/1612939#reply-1612939):
-https://www.glowfic.com/replies/1612939#reply-1612939
> And so long as they were doing all that anyways, they might as well also carry out the less important but still useful operation of putting all of Civilization's past behind the most complete possible causal screen. That part wasn't as important, but still legitimately helpful; and doing it would help to overshadow the other changes, and lead to less attention going to the more dangerous places.
-]
+"So long as they were doing all that anyways, they might as well"! That's it? The only concrete benefit of enforcing ignorance (of history) we're told about is ... more ignorance (of the existence of the AGI project).
+
+Keeping _the project's_ existence secret likely makes sense, if the surveillance measures are deemed insufficient to prevent rogue AGI projects. But classifying _all of human history_ along with it should require some enormously compelling reason, and one just isn't suppplied.
+
+Is it that the Singularity is so important, the stakes so massive, that [no cost is too small to pay](https://tvtropes.org/pmwiki/pmwiki.php/Main/UtopiaJustifiesTheMeans) to make sure it goes well? That would be understandable. (As the saying goes, ["Shut up and multiply."](https://www.lesswrong.com/posts/r5MSQ83gtbjWRBDWJ/the-intuitions-behind-utilitarianism))
+
+But the word choices make it seem like that's not what's going on. "So long as they were doing all that anyways, they might as well" is not the language of a tough-minded utilitarian who understands the gravity of the crimes they're about to commit, but is determined to proceed [for the good of the many](https://tvtropes.org/pmwiki/pmwiki.php/Main/TheNeedsOfTheMany). "So long as they were doing all that anyways, they might as well" is the language of someone who's never considered that what they're about to do might be a crime, that [people knowing things as benefits as well as costs](http://benjaminrosshoffman.com/humility-argument-honesty/).
+
+It would be remiss to conclude the section without considering a [Doylist rather than Watsonian](https://tvtropes.org/pmwiki/pmwiki.php/Main/WatsonianVersusDoylist) interpretation of the history screen. Rumored Word of God[^wog-rumor] states that one reason for the history screen is that it makes a better story for dath ilani isekai protagonists to be more ignorant of other ways of life than the power of their civilization would otherwise imply; it's more interesting to watch Keltham encounter a medieval monarchy if he doesn't know what a queen is.
+
+[^wog-rumor]: I don't have a link for this; I think I heard Yudkowsky say it at a party.
+
+The Doylist storytelling case for the history screen is compelling insofar as the function of the dath ilan mythos is strictly to tell entertaining stories. However, such a prominent feature of the Society existing for storytelling reasons undermines the function of dath ilan as a utopian contrast to the real world, as empitomized by the incessant in- and out-of-story sneers at "Earth" (_i.e._, real) people.
+
+### The Ordinary Merrin Conspiracy
+
+The story of Merrin provides us with a case study on dath ilani culture's penchant for deception in a lower-stakes context. Merrin is a famous emergency medical technician of exceptional endurance, who obsessively improves her skills by training on simulated disasters when she's not on duty saving QALYs in real life. We are [told of an Ordinary Merrin Conspiracy](https://www.glowfic.com/replies/1764946#reply-1764946)
+
+> wherein Merrin has some weird psychological hangup about believing she is a totally normal and ordinary person or even something of a struggling low achiever, and everybody in Civilization is coordinating to pretend around her that ordinary normal people totally get their weird Exception Handling training scenarios televised to a million watchers on a weekly basis.
+
+It's like a reverse [Emperor Norton](https://en.wikipedia.org/wiki/Emperor_Norton) situation. (Norton was ordinary, but everyone around him colluded to make him think he was special; Merrin is special, but everyone around her colludes to make her think she's ordinary.)
+
+One is given the sense that from a dath ilani moral perspective, this is heartwarming. Merrin doesn't want to feel important, so everyone in her world colludes to conceal evidence from her than she's important. Isn't that thoughtful?
-### The Merrin Show
+From the moral perspective of an _Overcoming Bias_ reader in 2008 "on Earth" (_i.e._, in real life) who places value on authenticity and truth in human affairs, this is unsettling and even immoral, for the same reason [the Emperor Norton conspiracy was immoral](/2018/Feb/the-categories-were-made-for-man-to-make-predictions/#emperor-norton). Merrin is living a lie, and everyone she trusts is in on it!
-For example, we are told of an Ordinary Merrin Conspiracy centered around a famous medical technician with a psychological need to feel unimportant, of whom ["everybody in Civilization is coordinating to pretend around her"](https://www.glowfic.com/replies/1764946#reply-1764946) that her achievements are nothing special, which is deemed to be kindness to her. It's like a reverse [Emperor Norton](https://en.wikipedia.org/wiki/Emperor_Norton) situation. (Norton was ordinary, but everyone around him colluded to make him think he was special; Merrin is special, but everyone around her colludes to make her think she's ordinary.)
+The deception by the people in Merrin's life is especially galling in light of the fact that the situation that Merrin needs to be protected from is one of their own making. EMTs can't become famous on accident. If Merrin doesn't like attention, she should not have a TV show with a million viewers! (We can assume that most unusually skilled workers aren't celebrities in dath ilan, as in real life, not because their work is particularly secret, but because recording and editing it for public consumption is a cost that no one has particular reason to pay.) One almost wonders whether the deception is the point—that people in dath ilan enjoy "coordinating to pretend around" someone, and Merrin's psychological hangup presented them with a golden opportunity to put someone in a [_The Truman Show_](https://en.wikipedia.org/wiki/The_Truman_Show)-like psuedo-reality.
-But _as_ a rationalist, I condemn the Ordinary Merrin Conspiracy as _morally wrong_, for the same [reasons I condemn the Emperor Norton Conspiracy](/2018/Feb/the-categories-were-made-for-man-to-make-predictions/#emperor-norton). As [it was taught to me on _Overcoming Bias_ back in the 'aughts](https://www.lesswrong.com/posts/HYWhKXRsMAyvRKRYz/you-can-face-reality): what's true is already so. Denying it won't make it better. Acknowledging it won't make it worse. And _because_ it is true, it is what is there to be interacted with. Anything untrue isn't there to be lived. People can stand what is true, _because they are already doing so_.
+["For No Laid Course Prepare"](https://glowfic.com/posts/6263), the story about how Merrin came to the attention of dath ilan's bureau of Exception Handling, shows us more examples of dath ilani conspiracy culture. In the story, Merrin is dressed up as a member of a fictional alien species, the Sparashki, due to having been summoned to the hospital from a fan convention with no time to change outfits. Merrin's coworkers falsely maintain to outsiders that Merrin always cosplays as a Sparashki while on duty. ["This is not considered a lie, in that it would be universally understood and expected that no one in this social circumstance would tell the truth,"](https://www.glowfic.com/replies/1857346#reply-1857346) the narrator tells us. The language used here is strikingly similar to that of one of the corrupt executives in [_Moral Mazes_](https://thezvi.wordpress.com/2019/05/30/quotes-from-moral-mazes/): "We lie all the time, but if everyone knows that we're lying, is a lie really a lie?"
-In ["For No Laid Course Prepare"](https://glowfic.com/posts/6263), the story about how Merrin came to the attention of dath ilan's bureau of Exception Handling, we see the thoughts of a Keeper, Rittaen, who talks to Merrin. We're told that the discipline of modeling people mechanistically rather than [through empathy](https://www.lesswrong.com/posts/NLMo5FZWFFq652MNe/sympathetic-minds) is restricted to Keepers to prevent the risk of ["turning into an exceptionally dangerous psychopath"](https://glowfic.com/replies/1862201#reply-1862201). Rittaen [uses his person-as-machine Sight](https://glowfic.com/replies/1862204#reply-1862204) to infer that Merrin was biologically predisposed to learn to be afraid of having too much status.
+But if [everyone really knew](https://thezvi.wordpress.com/2019/07/02/everybody-knows/), what would be function of saying the false thing? On dath ilan (if not in real-world boardrooms), one supposes the answer is "Because it's fun"? But that just prompts the followup question: but what is the function of the brain giving out a "fun" reward in this context? It seems like at some point, there has to be the expectation of some cognitive system (although possibly not an entire "person") taking the signals literally.
-Notwithstanding that Rittaen can be Watsonianly assumed to have detailed neuroscience skills that the author Doylistically doesn't know how to write, I am entirely unimpressed by the assertion that this idea is somehow _dangerous_, a secret that only Keepers can bear, rather than something _Merrin herself should be clued into_. "It's not [Rittaen's] place to meddle just because he knows Merrin better than Merrin does," we're told.
+Indeed, in the described circumstances, it's hard to see how receivers of the Merrin-always-cosplays falsehood are supposed to figure out that the claim is not true. If it's universally understood and expected that no dath ilani would pass up a good trolling opportunity, it does not therefore follow that the people being trolled know it. (That's exactly what makes a good troll!)
-In the same story, Merrin is dressed up as a member of a fictional alien species, the Sparashki, due to having been summoned to the hospital from a fan convention with no time to change outfits. An agent from Exception Handling [tells Merrin that the bureau's Fake Conspiracy section is running an operation to plant evidence that Sparashki are real](https://glowfic.com/replies/1860952#reply-1860952), and asks Merrin not to contradict this, and Merrin just ... goes along with it.
+It's tempting to speculate that the eliezera are high in an "enjoying social deception" trait. Not-actually-plausible conspiracies that most people are in on are a [superstimulus](https://www.lesswrong.com/posts/Jq73GozjsuhdwMLEG/superstimuli-and-the-collapse-of-western-civilization) like zero-calorie sweetener: engineered to let everyone enjoy the thrill of lying, without doing serious damage to shared maps.
-It's in-character for Merrin to go along with it, because she's a pushover. My question is, why is it okay that Exception Handling explicitly has a Fake Conspiracies section (!), any more than it would have been if FTX or Enron explicitly had a Fake Accounting department?
+An agent from Exception Handling [tells Merrin that the bureau's Fake Conspiracy section is running an operation to plant evidence that Sparashki are real](https://glowfic.com/replies/1860952#reply-1860952), and asks Merrin not to contradict this.
-Is it okay because the idea of humanoid aliens walking around in real life is sufficiently implausible that no one is "really fooled"? But this seems to be contradicted by the text, in which [the narrator tells us that Exception Handling deliberately does things that wouldn't otherwise make sense in order to make it harder to understand what's actually going on in Civilization](https://glowfic.com/replies/1860955#reply-1860955) (presumably, in order to protect the secret underground AGI alignment project). Even if very few dath ilani take the "Sparashki are real" ruse literally, it's still portrayed as successfully _kicking up epistemic dust_. If it were really the case that no one was being fooled about anything, then Exception Handling wouldn't have a strategic reason to do it.
+One immediately wonders: why does Exception Handling have a Fake Conspiracies department? Why is it okay for a government agency to be pulling hoaxes on its own citizens, any more than it would have been if FTX or Enron in real life had a Fake Accounting department?
Is it okay because dath ilan are the [designated good guys](https://tvtropes.org/pmwiki/pmwiki.php/Main/DesignatedHero)? Well, [so was FTX](https://forum.effectivealtruism.org/posts/sdjcH7KAxgB328RAb/ftx-ea-fellowships).
-[TODO—
- * The Merrin Show
- * Merrin: a reverse Emperor Norton case
- * Overcoming Bias readers in 2008 would have found this offensive, not cute; Merrin is living a lie, and everyone is in on it.
- * Rittaen's claim that knowledge of psychology turns you into a psychopath seems dubious.
- * Exception Handling's Sparashki ruse is still kicking up epistemic dust even if few take it literally
- * "Everybody knows"
- * It has to bottom out in someone taking it literally
- * zero-calorie superstimulus theory
-]
+Is it okay because the idea of humanoid aliens walking around in real life is sufficiently implausible that no one is "really fooled"?
+
+The narrator soon tells us that [Exception Handling deliberately does things that wouldn't otherwise make sense in order to make it harder to understand what's actually going on in Civilization](https://glowfic.com/replies/1860955#reply-1860955) (presumably, in order to protect the secret AGI alignment project). But this is in tension with the potential rationale that no one is really being fooled. Even if very few dath ilani take the "Sparashki are real" ruse literally, it's still portrayed as successfully "kicking up epistemic dust": if it were really the case that no one was being fooled about anything, then Exception Handling wouldn't have a strategic reason to do it.
+
+Later in the story, we see the thoughts of a Keeper, Rittaen, who talks to Merrin. We're told that the discipline of modeling people mechanistically rather than [through empathy](https://www.lesswrong.com/posts/NLMo5FZWFFq652MNe/sympathetic-minds) is restricted to Keepers to prevent the risk of ["turning into an exceptionally dangerous psychopath"](https://glowfic.com/replies/1862201#reply-1862201). Rittaen [uses his person-as-machine Sight](https://glowfic.com/replies/1862204#reply-1862204) to infer that Merrin was biologically predisposed to learn to be afraid of having too much status.
+
+Notwithstanding that Rittaen can be Watsonianly assumed to have detailed neuroscience skills that the author Doylistically doesn't know how to write, the assertion that _knowing too much about psychology_ is dangerous says something about dath ilan. The explanation of Merrin's psychology that we do see isn't dangerous. (I read it, and I haven't become a psychopath.) I think a culture that shared the values of _Overcoming Bias_ readers in 2008 would consider that knowledge something _Merrin herself should be clued into_. Not dath ilan, though. "It's not [Rittaen's] place to meddle just because he knows Merrin better than Merrin does," we're told. But apparently it _is_ the place of everyone else who can see Merrin's shyness to form the Ordinary Merrin Conspiracy, to keep her from knowing herself?
+
+That's not an accusation of hypocrisy. The dath ilani are being consistent. In a Society that prizes freedom-from-infohazards over freedom-of-speech, it makes sense that telling people things would be meddling, but colluding to not tell them, isn't.
### Keltham and S/M
I'm more preoccupied by a [Doylistic](https://tvtropes.org/pmwiki/pmwiki.php/Main/WatsonianVersusDoylist) interpretation—that dath ilan's obsessive secret-Keeping reflects something deep about how the Yudkowsky of the current year relates to speech and information, in contrast to the Yudkowsky who wrote the Sequences. The Sequences had encouraged you—yes, _you_, the reader—to be as rational as possible. In contrast, the dath ilan mythos seems to portray advanced rationality as dangerous knowledge that people need to be protected from.
-
-
-
As another notable example of dath ilan hiding information for the alleged greater good, in Golarion, Keltham discovers that he's a sexual sadist, and deduces that Civilization has deliberately prevented him from realizing this, because there aren't enough corresponding masochists to go around in dath ilan. Having concepts for "sadism" and "masochism" as variations in human psychology would make sadists like Keltham sad about the desirable sexual experiences they'll never get to have, so Civilization arranges for them to _not be exposed to knowledge that would make them sad, because it would make them sad_ (!!).
What happens, I asked, to the occasional dath ilani free speech activists, with their eloquent manifestos arguing that Civilization would be better off coordinating on maps that reflect the territory, rather than coordinating to be a Keeper-managed zoo? (They _had_ to exist: in a medianworld centered on Yudkowsky, there are going to be a few weirdos who are +2.5 standard deviations on "speak the truth, even if your voice trembles" and −2.5 standard deivations on love of clever plots; this seems less weird than negative utilitarians, who were [established to exist](https://www.glowfic.com/replies/1789623#reply-1789623).) I _assumed_ they get dealt with somehow in the end (exiled from most cities? ... involuntarily cryopreserved?), but there had to be an interesting story about someone who starts out whistleblowing small lies (which Exception Handling allows; they think it's cute, and it's "priced in" to the game they're playing), and then just keeps _escalating and escalating and escalating_ until Governance decides to unperson him.
[...]
-
If we believe that [IQ research validates the "Jews are clever" stereotype](https://web.mit.edu/fustflum/documents/papers/AshkenaziIQ.jbiosocsci.pdf), I wondered if there's a distinct (albeit probably correlated) "enjoying deception" trait that validates the "Jews are sneaky" stereotype? If dath ilan is very high in this "sneakiness" trait (relative to Earth Jews), that would help explain all the conspiracies!
-Not-actually-plausible conspiracies that everyone is in on (like "Sparashki are real") are a [superstimulus](https://www.lesswrong.com/posts/Jq73GozjsuhdwMLEG/superstimuli-and-the-collapse-of-western-civilization) like zero-calorie sweetener: engineered to let everyone enjoy the thrill of lying, without doing any real damage to shared maps.
-
-In "For No Laid Course Prepare", Merrin's coworkers falsely maintain to outsiders that Merrin always cosplays as a Sparashki while on duty. ["This is not considered a lie, in that it would be universally understood and expected that no one in this social circumstance would tell the truth,"](https://www.glowfic.com/replies/1857346#reply-1857346) the narrator tells us. The language used here is strikingly similar to that of one of the corrupt executives in [_Moral Mazes_](https://thezvi.wordpress.com/2019/05/30/quotes-from-moral-mazes/): "We lie all the time, but if everyone knows that we're lying, is a lie really a lie?"
-
-But if [everyone really knew](https://thezvi.wordpress.com/2019/07/02/everybody-knows/), what would be _function_ of saying the false thing? On dath ilan (if not in Earth boardrooms), one supposes the answer is "Because it's fun"? But that just prompts the followup question: but what is the function of the brain giving out a "fun" reward in this context? It seems like at _some_ point, there has to be the expectation of _some_ cognitive system (although possibly not an entire "person") taking the signals literally.[^funny-or-powerful-falsehood]
-
-[^funny-or-powerful-falsehood]: This is why, when I notice myself misrepresenting my actual beliefs or motivations because I think it's funny or rhetorically powerful, I often take care to disclaim it immediately, precisely because I _don't_ think that "everybody knows"; I'm not going to give up on humor or powerful rhetoric, but I'm also not going to delude myself into thinking it's "zero-calorie": people who don't "get the joke" _are_ going to be misled, and I don't think it's unambigously "their fault" for not being able to read my "intent" to arbitrary precision. But maybe dath ilan is (by authorial fiat) sufficiently good at achieving common knowledge in large groups that they _can_ pull off a zero-calorie "everyone knows" conspiracy without damaging shared maps?
-
The existence of such a widespread sneakiness/"taste for deception" trait among the eliezera, in conjunction with their culture just not particularly valuing public knowledge (because they assume everything important is being handled by the Keepers), explains the recurring conspiracies and coverups, like the Ordinary Merrin Conspiracy, Exception Handling's fabrication of evidence for Sparashki being real, the sadism/masochism coverup, and [the village that deliberately teaches anti-redhead bigotry to children in order to test the robustness of dath ilan's general humanism indoctrination](https://www.lesswrong.com/posts/uyBeAN5jPEATMqKkX/lies-told-to-children-1).
I stress that this hypothesis _doesn't_ require dath ilani to be cartoon villains who hate knowledge and want people to be ignorant. Just that, as a result of the widespread sneakiness trait and their outsourcing information-process to the Keepers, in the course of trying to accomplish other things, plans-that-involve-conspiracies are often higher in their search ordering than plans-that-involve-keeping-people-informed.
(I was tempted to tag that as "epistemic status: low-confidence speculation", but that's _frequentist_ thinking—as if "Jews and gentiles are equally sneaky" were a "null hypothesis" that could only be rejected by data that would be sufficiently unlikely assuming that the null was true. Ha ha, that would be _crazy!_ Obviously, I should have a _prior_ on the effect size difference between the Jew and gentile sneakiness distributions, that can be updated as sneakiness data comes in. I think the mean of my prior distribution is at, like, _d_ ≈ 0.1? So it's not "low confidence"; it's "low confidence of the effect size being large enough to be of much practical significance".)
-
For context on why I have no sense of humor about this, on Earth (which _actually exists_, unlike dath ilan), when someone says "it's not lying, because no one _expected_ me to tell the truth in that situation", what's usually going on, [as Zvi Mowshowitz explains](https://thezvi.wordpress.com/2019/07/02/everybody-knows/), is that is that conspirators benefit from deceiving outsiders, and the claim that "everyone knows" is them lying to _themselves_ about the fact that they're lying.
(If _you_ got hurt by not knowing, well, it's not like anyone got hurt, because if you didn't know, then you weren't anyone.)
-
-
Okay, but if it were _actually true_ that everyone knew, what would be _function_ of saying the false thing? On dath ilan (if not in Earth boardrooms), I suppose the answer is "Because it's fun"? Okay, but what is the function of your brain giving out a "fun" reward in this context? It seems like at _some_ point, there has to be the expectation of _some_ cognitive system (although possibly not an entire "person") taking the signals literally.
That's why, when I _notice_ myself misrepresenting my actual beliefs or motivations because I think it's funny or rhetorically powerful (and it takes a special act of noticing; humans aren't built to be honest by default), I often take care to disclaim it immediately (as was observed in the message this is one a reply to), precisely because I _don't_ think that "everybody knows"; I'm not going to give up on humor or powerful rhetoric, but I'm also not going to delude myself into thinking it's "zero-calorie" (people who don't "get the joke" _are_ going to be misled, and I don't think it's unambigously "their fault" for not being able to read my "intent" to arbitrary precision)
But maybe dath ilan is sufficiently good at achieving common knowledge in large groups that they _can_ pull off a zero-calorie "everyone knows" conspiracy without damaging shared maps??
-
I'm still skeptical, especially given that we see them narratizing it as "not lying" (in the same words that corrupt executives on Earth use!), rather than _explicitly_ laying out the evopysch logic of sneakiness superstimuli, and the case that they know how to pull it off in a zero-calorie (trivial damage to shared maps) way.
In general, I think that "it's not lying because no one expected the truth" is something you would say as part of an attempted nearest-unblocked-strategy end run around a deontological constraint against "lying" (https://www.lesswrong.com/posts/MN4NRkMw7ggt9587K/firming-up-not-lying-around-its-edge-cases-is-less-broadly); I don't think it's something you would say if you _actually cared_ about shared maps being accurate
(natural language is very flexible, you can _come up with_ some interpretation)
-
You can cosplay an elf _at a designated fandom convention_, where people have temporarily _opted in_ to that false social reality. But I don't think you can cosplay an elf _at work_ (in "real life") and have everyone play along for extended periods of time, without dealing damage to real-life shared maps.
Similarly, I cosplay female characters at fandom conventions, and that's fun, and I'm glad that conventions exist, but I can't transition in "real life", because I don't expect anyone in real life to believe that I'm female, because it's _very obviously not true_. People will _pretend_ to believe it because they're terrified of being accused of transphobia, but _they are lying_, and the
Aslan / amputation of destiny
* An ethnographer might note that Americans believe themselves to be "the land of the brave and the home of the free", without being obliged for their ethnography to agree with this description. I'm taking the same stance towards dath ilan: as a literary critic, I don't have to share its Society's beliefs about itself.
+
+spoilers for the pleasure of discovering sex for themselves: https://glowfic.com/replies/1812613#reply-1812613
+
+being told eugenics prospects early as self-fulfilling prophecies, as a legitimate infohazard: https://glowfic.com/replies/1812614#reply-1812614
projects / Ultimately_Untrue_Thought.git / blobdiff