1 Title: On the Public Anti-Epistemology of dath ilan
5 Tags: bullet-biting, cathartic, Eliezer Yudkowsky, epistemic horror, my robot cult, worldbuilding
8 > All should be laid open to you without reserve, for there is not a truth existing which I fear, or would wish unknown to the whole world.
10 > —Thomas Jefferson, earthling
12 Eliezer Yudkowsky's fiction about the world of dath ilan (capitalization _sic_) aims to portray a smarter, saner, better-coordinated alternate version of Earth. Dath ilan had originally been introduced in a [2014 April Fool's Day post](https://yudkowsky.tumblr.com/post/81447230971/my-april-fools-day-confession), in which Yudkowsky "confessed" that the explanation for his seemingly implausible genius is that he's "actually" an ordinary person from dath ilan, where the ideas he presented to this world as his own were common knowledge. (This likely inspired the trope of a [_medianworld_](https://www.glowfic.com/replies/1619639#reply-1619639), a setting where the average person is like the author along important dimensions.)[^medianworlds]
14 [^medianworlds]: You might think that the thought experiment of imagining what someone's medianworld is like would only be interesting for people who are "weird" in our own world, thinking that our world is a medianworld for people who are normal in our world. But [in high-dimensional spaces, _most_ of the probability-mass is concentrated in a "shell" some distance around the mode](/2021/May/sexual-dimorphism-in-the-sequences-in-relation-to-my-gender-problems/#typical-point), because even though the per-unit-hypervolume probability _density_ is greatest at the mode, there's vastly more hypervolume in the hyperspace around it. The upshot is that typical people are atypical along _some_ dimensions, so normies can play the medianworld game, too. (Or they could, if the normies of our world were into worldbuilding.)
16 Dath ilan's cognitive superiority to the real world is a recurring theme in modern Yudkowsky's work. In the fictional canon, it's a focus of the story ["But Hurting People is Wrong"](https://www.glowfic.com/posts/4508), but even when not discussing fiction, Yudkowsky often makes sneering comments about "Earth" or "Earth people", apparently meant to disparage all actually existing humans for not living up to his fiction.
18 One is led to believe that people who were deeply inspired by Yudkowsky's [Sequences](https://www.readthesequences.com/) (a series of influential posts about rationality published on the _Overcoming Bias_ blog largely between 2007 and 2009) should regard dath ilan as a rationalist utopia. After all, on the terms of the 2014 April Fools' Day joke, that's where the knowledge came from.
20 And yet, for such a supposed rationalist utopia, it's remarkable the extent to which dath ilan's Society is portrayed as being organized around conspiracies to lie or otherwise cover up the truth—not just when forced to by dire matters of planetary security (as when keeping nuclear or AGI secrets), but seemingly for any somewhat plausible excuse whatsoever, including protecting the feelings of people who would be happier if kept ignorant. Evidently, there are _many_ truths existing which dath ilan fears and would wish unknown to the whole world.
22 The contrast to the [sense of life](http://aynrandlexicon.com/lexicon/sense_of_life.html) portrayed in the Sequences is striking. The Sequences emphasized that you—yes, you, the reader—had an interest in having an accurate world-model. On the subject of confronting unpleasant hypotheses, the Sequences [gave this advice](https://www.readthesequences.com/Avoiding-Your-Beliefs-Real-Weak-Points) (bolding mine):
24 > When you're doubting one of your most cherished beliefs, close your eyes, empty your mind, grit your teeth, and **deliberately think about whatever hurts the most**. Don't rehearse standard objections whose standard counters would make you feel better. Ask yourself what _smart_ people who disagree would say to your first reply, and your second reply. Whenever you catch yourself flinching away from an objection you fleetingly thought of, drag it out into the forefront of your mind. **Punch yourself in the solar plexus. Stick a knife in your heart, and wiggle to widen the hole.** In the face of the pain, rehearse only this:
26 >> What is true is already so.
27 >> Owning up to it doesn't make it worse.
28 >> Not being open about it doesn't make it go away.
29 >> And because it's true, it is what is there to be interacted with. Anything untrue isn't there to be lived.
30 >> People can stand what is true, for they are already enduring it.
32 Meanwhile, the dath ilan mythos depicts rationality itself as hazardous knowledge that people need to be protected from. Dath ilani Civilization is steered by a secretive order of [Keepers of Highly Unpleasant Things it is Sometimes Necessary to Know](https://www.glowfic.com/replies/1612937#reply-1612937). ["The universe is not so dark a place that everyone needs to become a Keeper to ensure the species's survival,"](https://glowfic.com/replies/1861879#reply-1861879) we're told. "Just dark enough that some people ought to." Ordinary dath ilani do receive rationality training, but it's implied to be deliberately crippled, featuring ["signposts around the first steps [towards becoming a Keeper], placed to warn dath ilani off starting down that path unless they mean it."](https://www.glowfic.com/replies/1799590#reply-1799590) The maxim that "That which can be destroyed by the truth should be" is described as being ["remembered as much for how it's false, as for how it's true, because among the things that truths can destroy is people."](https://www.glowfic.com/replies/1687922#reply-1687922)
34 Clearly, this is not a culture that cares about ordinary people being well-informed. Apparently, they believe that owning up to it _does_ make it worse, that the untrue _is_ there to be lived (for non-Keepers).
36 We might say that the algorithm that designed dath ilan's Civilization can be seen as systematically preferring deception. When I speak of an algorithm preferring deception, [what I mean is](https://www.lesswrong.com/posts/fmA2GJwZzYtkrAKYJ/algorithms-of-deception) that given a social problem, candidate solutions that involve deceiving the populace seem to be higher in dath ilani Civilization's implicit search ordering than solutions that involve informing the populace. If a solution adequately achieves its purpose by means of deception or secrecy, Civilization isn't going to reject it on that basis and keep searching for a solution that involves telling the truth.
38 Crucially, these are [functionalist](https://www.lesswrong.com/posts/sXHQ9R5tahiaXEZhR/algorithmic-intent-a-hansonian-generalized-anti-zombie) [criteria](http://benjaminrosshoffman.com/bad-faith-behavior-not-feeling/) of "preference" and "deception". It's about how Civilization is structured in a way that systematically encourages divergences between popular belief and reality. I'm _not_ positing that Civilization's Keepers and Legislators and Chief Executive are laughing maniacally and telling each other out loud, "I personally love it when non-Keepers have false beliefs; we need to do as much of that as possible—as a [terminal value](https://www.readthesequences.com/Terminal-Values-And-Instrumental-Values)!"
40 Rather, I'm positing that they don't care about non-Keepers having false beliefs. (They might care about [not technically lying](https://www.lesswrong.com/posts/PrXR66hQcaJXsgWsa/not-technically-lying), but that [turns out to be a weak constraint](https://www.lesswrong.com/posts/MN4NRkMw7ggt9587K/firming-up-not-lying-around-its-edge-cases-is-less-broadly).)
42 If you're in the business of coming up with clever plans to solve problems, and you don't care about people having false beliefs, you mostly end up with clever plans that work by means of giving people false beliefs that trick them into doing what you want them to do (perhaps without technically lying).
44 Why wouldn't you? There are more false maps than true maps. If you don't specifically care about [affirmatively telling the truth](http://benjaminrosshoffman.com/honesty-and-perjury/#Intent_to_inform), you mostly end up supplying false maps in order to control people's behavior by means of controling their information, because if you told them the truth, they wouldn't behave the way you want them to. Instrumental convergence is a harsh mistress.
46 ### Interlude: Methodology for Worldbuilding Criticism; Or, I Can't Argue With Authorial Fiat
48 At this point, some readers might object that this kind of "dark" interpretation of a fictional universe oversteps the authority of the literary critic. One imagines that Yudkowsky doesn't particularly think of dath ilan as a world governed by deception. What grounds could I possibly have to argue that it is, given that he's the author and I'm not? Isn't that just making up my own fictional world and substituting it for the "real" dath ilan defined by Yudkowsky's authorial intent?
50 But the craft of literature isn't a matter of merely conveying a fictional reality that existed fully formed in the author's imagination in advance of writing it down. The craft is about producing text that readers can use to build up their own model of the fictional world. The exacting labor of [converting vague ideas into definite text](http://www.paulgraham.com/words.html) is the difference between writing and daydreaming. We can accept [Word of God](https://tvtropes.org/pmwiki/pmwiki.php/Main/WordOfGod) as supplementary material where the text of a story is ambiguous or silent on a point of interest, but some kind of [Death of the Author](https://tvtropes.org/pmwiki/pmwiki.php/Main/DeathOfTheAuthor) stance is ultimately necessary for making sense of literature in a world in which telepathy doesn't exist and authors do occasionally die. The text is not a mere pointer to the "real" work inside the author's head; the text _is_ the work. That's the only way the technology of writing can function.
52 Moreover, a Death of the Author stance seems particularly important for evaluating medianworlds. What makes the exercise of constructing a medianworld interesting is the challenge of envisioning the details of a realistic Society that would result given a population with an alternative [distribution of cognitive repertoires](/2020/Apr/book-review-human-diversity/), but where the same generalizations of biology, sociology, and economics that govern our own world are presumed to hold. If the world portrayed by the text [doesn't seem to hold together](/2022/Jun/comment-on-a-scene-from-planecrash-crisis-of-faith/) or has [unfortunate implications](https://tvtropes.org/pmwiki/pmwiki.php/Main/UnfortunateImplications) that the author doesn't acknowledge, it's the solemn duty of literary critics to point that out to less discerning readers.
54 In analogy, mathematicians, like authors, are also in the business of creating imaginary worlds, but mathematical objects, once defined, can be examined on their own terms. A mathematician encountering [a deeply unsatisfying theorem about their new definition](https://blog.plover.com/math/major-screwups-4.html) understands that there can be no recourse in protesting, "But that's not how _I_ imagined it working." That would be failing to engage with the real difficulties of mathematical research. If you wanted different behavior, you should have written a better definition!
56 Similarly, an author who says, "In _my_ medianworld, fully automated luxury gay space communism with central planning just works, because the populace is so smart and nice, unlike _Earth people_, who are so mean and dumb that they have to use _markets_ to allocate scarce resources" is failing to engage with the real difficulties of the medianworld exercise. Readers would have a right to be skeptical.
58 Authors, of course, have much more wiggle room than mathematicians to try to salvage their cherished ideas. Rather than being forced back to the drawing board by an unwanted implication, a fiction writer finds it all too easy to simply add another sentence denying it. But the author's total freedom to specify the text necessarily interacts with readers' attempts to imagine a self-consistent universe that "projects into" that text. Short of an explicitly omniscient narrator declaring "And then a miracle occured", discerning readers will tend to reach for interprerations of the text that make sense—even if making sense entails casting doubt on the narrator's spin on the described events.
60 Yudkowsky's 2009 story ["The Sword of Good"](https://www.yudkowsky.net/other/fiction/the-sword-of-good) is an incisive commentary on how unwary readers' moral compasses can be hijacked by authorial editorializing. If the story depicts our heroes wantonly slaughtering orcs, readers tend not to worry about the ethics of warfare: if the [designated hero](https://tvtropes.org/pmwiki/pmwiki.php/Main/DesignatedHero) is doing it, it presumably isn't a problem. But living creatures that are depicted as speaking language, having organized armies with complex tools, _&c._ are presumably sapient for the same reasons humans are. It's fair game for "The Sword of Good" to point that out—at least, short of the narrator _explicitly_ declaring, "Despite appearances, the orcs are unconscious [philosophical zombies](https://www.lesswrong.com/posts/fdEWWr8St59bXLbQr/zombies-zombies); killing them has no moral significance." (But a story that did declare that would be highly unusual, and basically conceding the critic's point about stories that didn't!)
62 All I'm doing in this essay is holding the dath ilan mythos to the same standard that "The Sword of Good" holds classic fantasy tropes to. Maybe it's good to run a global conspiracy to keep people ignorant in order to protect their emotional well-being, and maybe those orcs deserved to die. But given a text that _does_ literally describe massive coverups or killing of human-like creatures, it's fundamentally fair game for literary critics to point that out, and prompt readers to think carefully about whether they should accept "it's fine because the good guys are doing it" or "it's fine because it's dath ilan, which is not Earth" as an implicit excuse.
64 ### The History Screen
66 One of the distinctive features of dath ilani Civilization is that they've deleted their history. No one except a few specially cleared specialists ["in their own causally isolated bunker"](https://glowfic.com/replies/1688794#reply-1688794) is allowed to learn history before a point in a relatively recent past. (["[D]ecades ago, not millennia ago"](https://glowfic.com/replies/1789110#reply-1789110) as of Keltham's time.)
68 This is an incredibly drastic measure, a civilizational lobotomy. (It perhaps bears worth mentioning that the most famous literary depiction of a Society that deleted its history is George Orwell's _Nineteen-Eighty-Four_, in which a totalitarian state maintains a Society in which "[n]othing exists except an endless present in which the Party is always right.") The full costs may not be obvious to many readers: if the immediate needs of living in the present don't directly require thinking about the past, what's the harm of not knowing?
70 The problem is that [facts are causally entangled with each other](https://www.readthesequences.com/Entangled-Truths-Contagious-Lies). Forgetting one fact entails not just having to deal with not knowing that particular thing, but also not knowing everything it implies about other things in the world—which might be of little consequence for any one trivium, but is surely enormous for _all of human history_.
72 In at least one case, the text depicts dath ilan as suffering practical consequences from its ignorance: a discussion of the eliezera's low happiness levels is cut short with ["How'd it happen? Nobody knows, at this point, they screened their history."](https://glowfic.com/replies/1812613#reply-1812613) (Generally, knowing how something happened is useful for figuring out how to remediate it.) Realistically, we can only surmise that there are many other cases where knowing history would be useful that the narrator hasn't gotten around to telling us.
74 Given the costs, what could possibly justify the history screen, in an advanced Society that otherwise seems to value knowledge? A fantasy author could easily an invent an answer: maybe a psychic plague that spread through memories such that the infected must be not just physically quarantined, but forgotten. Medianworld authors have a tougher time insofar as we construe their genre as hard science fiction. Psychic plagues aren't real. What could _realistically_ justify the history screen?
76 The answer the text gives leaves something to be desired. In "But Hurting People Is Wrong", we learn of the birth of modern dath ilan, when the Keepers noticed the AGI alignment problem and propose strict policy measures aimed at surviving the challenge.
78 For one, dath ilan's eugenics program is to optimize for intelligence even more than it already does. That makes sense: smarter alignment researchers have a better shot at solving the problem.
80 For another, restrictions on the manufacture of computers are imposed. That makes sense: if computer hardware were too capable and easily available before alignment researchers had the [serial time](https://www.lesswrong.com/posts/FS6NCWzzP8DHp4aD4/do-earths-with-slower-economic-growth-have-a-better-chance) to think the problem through, that would increase the risk of accidentally unleashing an unaligned intelligence explosion.
82 For another, digital surveilance is imposed. That makes sense: Governance would want to know about any rogue AGI projects, so they can be shut down by force.
84 [And then we get this](https://www.glowfic.com/replies/1612939#reply-1612939):
86 > And so long as they were doing all that anyways, they might as well also carry out the less important but still useful operation of putting all of Civilization's past behind the most complete possible causal screen. That part wasn't as important, but still legitimately helpful; and doing it would help to overshadow the other changes, and lead to less attention going to the more dangerous places.
88 "So long as they were doing all that anyways, they might as well"! That's it? The only concrete benefit of enforcing ignorance (of history) we're told about is ... more ignorance (of the existence of the AGI project).
90 Keeping _the project's_ existence secret likely makes sense, if the surveillance measures are deemed insufficient to prevent rogue AGI projects. But classifying _all of human history_ along with it should require some enormously compelling reason, and one just isn't suppplied.
92 Is it that the Singularity is so important, the stakes so massive, that [no cost is too small to pay](https://tvtropes.org/pmwiki/pmwiki.php/Main/UtopiaJustifiesTheMeans) to make sure it goes well? That would be understandable. (As the saying goes, ["Shut up and multiply."](https://www.lesswrong.com/posts/r5MSQ83gtbjWRBDWJ/the-intuitions-behind-utilitarianism))
94 But the word choices make it seem like that's not what's going on. "So long as they were doing all that anyways, they might as well" is not the language of a tough-minded utilitarian who understands the gravity of the crimes they're about to commit, but is determined to proceed [for the good of the many](https://tvtropes.org/pmwiki/pmwiki.php/Main/TheNeedsOfTheMany). "So long as they were doing all that anyways, they might as well" is the language of someone who's never considered that what they're about to do might be a crime, that [people knowing things as benefits as well as costs](http://benjaminrosshoffman.com/humility-argument-honesty/).
96 It would be remiss to conclude the section without considering a [Doylist rather than Watsonian](https://tvtropes.org/pmwiki/pmwiki.php/Main/WatsonianVersusDoylist) interpretation of the history screen. Rumored Word of God[^wog-rumor] states that one reason for the history screen is that it makes a better story for dath ilani isekai protagonists to be more ignorant of other ways of life than the power of their civilization would otherwise imply; it's more interesting to watch Keltham encounter a medieval monarchy if he doesn't know what a queen is.
98 [^wog-rumor]: I don't have a link for this; I think I heard Yudkowsky say it at a party.
100 The Doylist storytelling case for the history screen is compelling insofar as the function of the dath ilan mythos is strictly to tell entertaining stories. However, such a prominent feature of the Society existing for storytelling reasons undermines the function of dath ilan as a utopian contrast to the real world, as empitomized by the incessant in- and out-of-story sneers at "Earth" (_i.e._, real) people.
102 ### The Ordinary Merrin Conspiracy
104 The story of Merrin provides us with a case study on dath ilani culture's penchant for deception in a lower-stakes context. Merrin is a famous emergency medical technician of exceptional endurance, who obsessively improves her skills by training on simulated disasters when she's not on duty saving QALYs in real life. We are [told of an Ordinary Merrin Conspiracy](https://www.glowfic.com/replies/1764946#reply-1764946)
106 > wherein Merrin has some weird psychological hangup about believing she is a totally normal and ordinary person or even something of a struggling low achiever, and everybody in Civilization is coordinating to pretend around her that ordinary normal people totally get their weird Exception Handling training scenarios televised to a million watchers on a weekly basis.
108 It's like a reverse [Emperor Norton](https://en.wikipedia.org/wiki/Emperor_Norton) situation. (Norton was ordinary, but everyone around him colluded to make him think he was special; Merrin is special, but everyone around her colludes to make her think she's ordinary.)
110 One is given the sense that from a dath ilani moral perspective, this is heartwarming. Merrin doesn't want to feel important, so everyone in her world colludes to conceal evidence from her than she's important. Isn't that thoughtful?
112 From the moral perspective of an _Overcoming Bias_ reader in 2008 "on Earth" (_i.e._, in real life) who places value on authenticity and truth in human affairs, this is unsettling and immoral, for the same reason [the Emperor Norton Conspiracy was immoral](/2018/Feb/the-categories-were-made-for-man-to-make-predictions/#emperor-norton). Merrin is living a lie, and everyone she trusts is in on it!
114 The deception by the people in Merrin's life is especially galling in light of the fact that the situation that Merrin needs to be protected from is one of their own making. EMTs can't become famous on accident. If Merrin doesn't like attention, she should not have a TV show with a million viewers! (We can assume that most unusually skilled workers aren't celebrities in dath ilan, as in real life, not because their work is particularly secret, but because recording and editing it for public consumption is a cost that no one has particular reason to pay.) One almost wonders whether the deception is the point—that people in dath ilan enjoy "coordinating to pretend around" someone, and Merrin's psychological hangup presented them with a golden opportunity to put someone in a [_The Truman Show_](https://en.wikipedia.org/wiki/The_Truman_Show)-like psuedo-reality.
116 ["For No Laid Course Prepare"](https://glowfic.com/posts/6263), the story about how Merrin came to the attention of dath ilan's bureau of Exception Handling, shows us more examples of dath ilani conspiracy culture. In the story, Merrin is dressed up as a member of a fictional alien species, the Sparashki, due to having been summoned to the hospital from a fan convention with no time to change outfits. Merrin's coworkers falsely maintain to outsiders that Merrin always cosplays as a Sparashki while on duty. ["This is not considered a lie, in that it would be universally understood and expected that no one in this social circumstance would tell the truth,"](https://www.glowfic.com/replies/1857346#reply-1857346) the narrator tells us. The language used here is strikingly similar to that of one of the corrupt executives in [_Moral Mazes_](https://thezvi.wordpress.com/2019/05/30/quotes-from-moral-mazes/): "We lie all the time, but if everyone knows that we're lying, is a lie really a lie?"
118 But if [everyone really knew](https://thezvi.wordpress.com/2019/07/02/everybody-knows/), what would be function of saying the false thing? On dath ilan (if not in real-world boardrooms), one supposes the answer is "Because it's fun"? But that just prompts the followup question: but what is the function of the brain giving out a "fun" reward in this context? It seems like at some point, there has to be the expectation of some cognitive system (although possibly not an entire "person") taking the signals literally.
120 Indeed, in the described circumstances, it's hard to see how receivers of the Merrin-always-cosplays falsehood are supposed to figure out that the claim is not true. If it's universally understood and expected that no dath ilani would pass up a good trolling opportunity, it does not therefore follow that the people being trolled know it. (That's exactly what makes a good troll!)
122 It's tempting to speculate that the eliezera are high in an "enjoying social deception" trait. Not-actually-plausible conspiracies that most people are in on are a [superstimulus](https://www.lesswrong.com/posts/Jq73GozjsuhdwMLEG/superstimuli-and-the-collapse-of-western-civilization) like zero-calorie sweetener: engineered to let everyone enjoy the thrill of lying, without doing serious damage to shared maps.
124 An agent from Exception Handling [tells Merrin that the bureau's Fake Conspiracy section is running an operation to plant evidence that Sparashki are real](https://glowfic.com/replies/1860952#reply-1860952), and asks Merrin not to contradict this.
126 One immediately wonders: why does Exception Handling have a Fake Conspiracies department? Why is it okay for a government agency to be pulling hoaxes on its own citizens, any more than it would have been if FTX or Enron in real life had a Fake Accounting department?
128 Is it okay because dath ilan are the [designated good guys](https://tvtropes.org/pmwiki/pmwiki.php/Main/DesignatedHero)? Well, [so was FTX](https://forum.effectivealtruism.org/posts/sdjcH7KAxgB328RAb/ftx-ea-fellowships).
130 Is it okay because the idea of humanoid aliens walking around in real life is sufficiently implausible that no one is "really fooled"?
132 The narrator soon tells us that [Exception Handling deliberately does things that wouldn't otherwise make sense in order to make it harder to understand what's actually going on in Civilization](https://glowfic.com/replies/1860955#reply-1860955) (presumably, in order to protect the secret AGI alignment project). But this is in tension with the potential rationale that no one is really being fooled. Even if very few dath ilani take the "Sparashki are real" ruse literally, it's still portrayed as successfully "kicking up epistemic dust": if it were really the case that no one was being fooled about anything, then Exception Handling wouldn't have a strategic reason to do it.
134 Later in the story, we see the thoughts of a Keeper, Rittaen, who talks to Merrin. We're told that the discipline of modeling people mechanistically rather than [through empathy](https://www.lesswrong.com/posts/NLMo5FZWFFq652MNe/sympathetic-minds) is restricted to Keepers to prevent the risk of ["turning into an exceptionally dangerous psychopath"](https://glowfic.com/replies/1862201#reply-1862201). Rittaen [uses his person-as-machine Sight](https://glowfic.com/replies/1862204#reply-1862204) to infer that Merrin was biologically predisposed to learn to be afraid of having too much status.
136 Notwithstanding that Rittaen can be Watsonianly assumed to have detailed neuroscience skills that the author Doylistically doesn't know how to write, the assertion that _knowing too much about psychology_ is dangerous says something about dath ilan. The explanation of Merrin's psychology that we do see isn't dangerous. (I read it, and I haven't become a psychopath.) I think a culture that shared the values of _Overcoming Bias_ readers in 2008 would consider that knowledge something _Merrin herself should be clued into_. Not dath ilan, though. "It's not [Rittaen's] place to meddle just because he knows Merrin better than Merrin does," we're told. But apparently it _is_ the place of everyone else who can see Merrin's shyness to form the Ordinary Merrin Conspiracy, to keep her from knowing herself?
138 That's not an accusation of hypocrisy. The dath ilani are being consistent. In a Society that prizes freedom-from-infohazards over freedom-of-speech, it makes sense that telling people things would be meddling, but colluding to not tell them, isn't.
140 ### Keltham and the Sadism/Masochism Coverup
142 The story of Keltham's sexuality provides another case study on dath ilani Civilization's predilection for deception. In Golarion, Keltham discovers that he's a sexual sadist, and infers that ["not many sadists in dath ilan know what they are and Civilization tries to prevent us from finding out, because dath ilan does _not_ have masochists."](https://glowfic.com/replies/1735044#reply-1735044) (The notion of an evolved organism preferring pain in any context strikes him as implausible.) Word of God clarifies that masochists do exist at low frequencies, but that Keltham's guess is otherwise correct; there is a deliberate effort (by the Keepers?—the exact agency responsible isn't clear) to prevent public knowledge that sadism and masochism even exist.
144 The naïve utilitarian case for the coverup is straightforward. There aren't enough masochists to go around. People with a latent inclination towards sadism are better off not being self-aware about it, because if they knew what they wanted, they would feel sad about not being able to have it.
146 The non-naïve utilitarian case is less straightforward, when you consider the second-order effects of censoring scientific information about psychology. The existence of sadism and masochism are facts about the world that you would expect to be mentioned in the sexuality section of psychology textbooks. If dath ilan is censoring real conditions from its psychology textbooks in order to avoid emotional harm to people with those conditions, that's a constraint on the extent to which their Society is in touch with reality.
148 Again, [facts are connected to each other](https://www.lesswrong.com/posts/wyyfFfaRar2jEdeQK/entangled-truths-contagious-lies), which both
151 * They do manage to censor sex information for children, but "age" is a Schelling cateogry in human Society; you can limit epistemic damage somewhat
153 * What about the costs of all the other recursive censorship you'd have to do to keep the secret? (If a biography mentioned masochism in passing along with many other traits of the subject, you'd need to either censor the paragraphs with that detail, or censor the whole book. Those are real costs, even under a soft-censorship regime where people can give special consent to access "Ill Advised" products.) Maybe latent sadists could console themselves with porn if they knew, or devote their careers to making better sex robots, just as people on Earth with non-satisfiable sexual desires manage to get by. (I _knew some things_ about this topic.) What about dath ilan's heritage optimization (read: eugenics) program? Are they going to try to breed more masochists, or fewer sadists, and who's authorized to know that? And so on.
155 * transsexualism example
157 * "They're better off not knowing about masochists that they can't have" disregards other ways of dealing with it: fantasy, porn, inventing in better sex robots, implications for eugenic policy ...
159 * Difficulties of recursive censorship; do the people who can afford it have to stay in the closet for the benefit of people like Keltham? You could claim that they're so good at coordinating that they negotiated this, but how does the negotiation work when the people the deal allegedly benefits don't know about it?
161 * Word of God talks about "affording" a masochist—suggesting not just that classism and sex work are legal, but institutionalized??
163 * the empirical setting of probabilities and utilities just happening to settle such that ignorance is the best policy
165 * Keltham thought evolutionary psychology ruled out the possibility of masochists;
167 * It would be one thing if BDSM orientation was a self-fulfilling propehecy. You can't miss an addictive drug. The text seems to suggest that "obligate sadism" is a real thing that already exists. It's not analogous to a drug that you can't miss if no one ever invents it.
169 Comparisons to other possible non-practicable minority sexualities may be instructive on the perils of deleting concepts. Imagine a world where homosexuality couldn't be safely practiced because [an engineered plague made it amount to certain death](https://www.gregegan.net/MISC/MORAL/Moral.html). Would we then approve of censoring the concept itself, forbidding the knowledge that same-sex attraction was ever a thing? And homosexuality has other correlates. [(The stereotypes of masculine lesbians and feminine gay men are empirically grounded.)](/papers/lippa-gender-related_traits_in_gays.pdf) Given the plague, would it be better not to notice the clusters—for gay people to possibly be left with some vague sense that there's something not normal about them, and not have a concept to name it?
171 Or consider how our Society handles pedophilia. This is a sexuality that we do not want to be practiced, so we stigmatize it and criminalize its practice. We do not censor the concept. Given that the tendency exists (and is not merely a cultural self-fulfilling prophecy that wouldn't exist if it weren't named), it's better to have a concept about it that can be reasoned about, rather than a blank spot on our map. Imagine a young man noticing he's attracted to children—or for that matter, an abused child—having _no idea what was happening to them_, because Society's benevolent censors decided that it was better that people not have concepts for this aspect of reality. Is that going to end well? How do you get help for a problem you're not supposed to have a word for?
179 * Word of God says, "there's also a deliberate semiconspiracy whereby movies make standard kitchen knives look much deadlier and easier to wield than they actually are, so that people who suddenly go nuts won't improvise much more dangerous weapons than that"
180 * "too nuts to see past cached thoughts" presupposes a specific model of mental illness that I think is importantly backwards; while I was psychotic, my problem was the _opposite_ of not being able to "see past cached thoughts": on the contrary, I was having _lots_ of new ideas that I had never thought of before (<http://zackmdavis.net/blog/2017/02/cognitive-bayesian-therapy-i/>), and would never have thought of in my normal state—it's just that, after I got some sleep, I decided that all of my new ideas were clearly false, _which is why_ we call the psychotic state "crazy", and the normal state "sane".
181 * You can't leak to the general public without leaking to crazy people. Where do you think they come from?
182 * the mechanism by which the plan positively contributes to the goal, is deception: the plan works _because_ if people who want to commit violence have _less accurate beliefs_ about how to effectively commit violence, they will be less effective at their goal of committing violence.
183 * If you were just trying to reduce murders, is this hte plan you would pick? Obviously not. (Think about how you would solve the problem "on Earth"!) Making it easier to commit people is a much more direct mechanism. The algorithm that promoted the movie-misportrayal to your attention is one that favors deception.
184 * "It's targeted at crazy people" excuse suggests that it's OK to lie to people you've labeled as crazy. But Yudkowsky infamously says this about all Earthlings. What should we infer about whether we should trust him?
185 * "i presume ilani who notice the trope also then deduce the obvious consequence of the trope"
186 * If anything, "there's a deliberate conspiracy to mislead people about the difficulty of murder by stabbing" is the kind of thought that someone is _more likely_ to have as a paranoid delusion than otherwise! It's as if dath ilan is an exotic environment where mind-states that are tuned to hypothesize conspiracies everywhere are _adaptive_, in contrast to how on Earth, people who see conspiracies everywhere are wrong.
187 * this is correlated insanity that loves clever deceptions, not actually being smart and doing a neutral policy search
188 * A much larger cost is the frustration of ornery, nitpicky nerds who _want realistic fiction_, and don't want their favorite artform perverted to _manipulate extremely rare criminal-insanity cases_. Earth has Andy Weir fans and Greg Egan fans! When Andy Weir or Greg Egan sit down to write a novel or a screenplay, they are trying to paint a picture of _a world that makes sense_. They are not trying to manipulate rare social outcomes. Is that that kind of respect-for-reality just _not a thing_ in dath ilani art? Are they so uniformly, monomanically obsessed with trolling their perceived inferiors, that the voice of, "Knives Don't Actually Work That Way; Knives Don't Actually Work That Way _Even When Saying That Out Loud Increases Murders_; Knives Don't Actually Work That Way _Even When a Prediction Market Says That Saying That Out Loud Increases Murders_", has no power in the movie industry whatsoever?
191 ### Replies to Objections
193 This essay has had some negative things to say about dath ilani culture.
206 * Counterargument: dath ilan lets you immigrate to infohazardville. Reply: sure, I agree that a Society that marginalizes truth-tellers is better than one than executes them. It's still fundamentally fair game for me to point out that it's not consilient with the values of _Overcoming Bias_ readers in 2008.
207 * Counterargument: as a reductio, am I opposed to movie and homework exercise spoilers, too? Reply: No, I recognize spoiler protection as a legitimate class of infohazard. (And AGI/nuclear as an social exfohazard, and threats as a legitimate infohazard.) And the concealing eugenics predictions, self-fulfilling prophecy
208 * Counterargument: the text says that regular dath ilani do believe "that which can be destroyed" eventually, just not immediately. Reply: this would be more believable with a timetable; I can understand "I don't want to deal with this yet", but there's no indication that Merrin or Keltham are supposed to figure it out.
209 * Counterargument: but the gaslighting is supposed to inculcate distrust of authority. Reply: clearly it doesn't work, given how much everyone trusts the government that is lying to them all the time?
210 * Numendil's point: dath ilani should look spoiled to us, for the same reason we look spoiled to Golarionites
projects / Ultimately_Untrue_Thought.git / blob