Author: Zack M. Davis
Date: 2023-07-01 11:00
Category: commentary
-Tags: autogynephilia, bullet-biting, cathartic, Eliezer Yudkowsky, Scott Alexander, epistemic horror, my robot cult, personal, sex differences, two-type taxonomy, whale metaphors
+Tags: bullet-biting, cathartic, Eliezer Yudkowsky, epistemic horror, my robot cult, worldbuilding
Status: draft
-> When all its work is done, the lie shall rot;
-> The truth is great, and shall prevail,
-> When none cares whether it prevail or not.
+> All should be laid open to you without reserve, for there is not a truth existing which I fear, or would wish unknown to the whole world.
>
-> —[Coventry Patmore](https://www.lesswrong.com/posts/9M3ynNuBsRCDfmEsZ/less-wrong-poetry-corner-coventry-patmore-s-magna-est)
+> —Thomas Jefferson, earthling
-Perhaps for lack of any world-saving research to do, Yudkowsky started writing fiction again, largely in the form of Glowfic (a genre of collaborative storytelling pioneered by Alicorn) featuring the world of dath ilan (capitalization _sic_). Dath ilan had originally been introduced in a [2014 April Fool's Day post](https://yudkowsky.tumblr.com/post/81447230971/my-april-fools-day-confession), in which Yudkowsky "confessed" that the explanation for his seemingly implausible genius is that he's "actually" an ordinary person from a smarter, saner alternate version of Earth where the ideas he presented to this world as his own were common knowledge.
+Eliezer Yudkowsky's fiction about the world of dath ilan (capitalization _sic_) aims to portray a smarter, saner, better-coordinated alternate version of Earth. Dath ilan had originally been introduced in a [2014 April Fool's Day post](https://yudkowsky.tumblr.com/post/81447230971/my-april-fools-day-confession), in which Yudkowsky "confessed" that the explanation for his seemingly implausible genius is that he's "actually" an ordinary person from dath ilan, where the ideas he presented to this world as his own were common knowledge. (This likely inspired the trope of a [_medianworld_](https://www.glowfic.com/replies/1619639#reply-1619639), a setting where the average person is like the author along important dimensions.)[^medianworlds]
-The bulk of the dath ilan Glowfic canon was an epic titled [_Planecrash_](https://www.glowfic.com/boards/215)[^planecrash-title] coauthored with Lintamande, in which Keltham, an unusually selfish teenage boy from dath ilan, apparently dies in a freak aviation accident, and [wakes up in the world of](https://en.wikipedia.org/wiki/Isekai) Golarion, setting of the _Dungeons-&-Dragons_–alike _Pathfinder_ role-playing game. A [couple](https://www.glowfic.com/posts/4508) of [other](https://glowfic.com/posts/6263) Glowfic stories with different coauthors further flesh out the setting of dath ilan, which inspired a new worldbuilding trope, the [_medianworld_](https://www.glowfic.com/replies/1619639#reply-1619639), a setting where the average person is like the author along important dimensions.[^medianworlds]
+[^medianworlds]: You might think that the thought experiment of imagining what someone's medianworld is like would only be interesting for people who are "weird" in our own world, thinking that our world is a medianworld for people who are normal in our world. But [in high-dimensional spaces, _most_ of the probability-mass is concentrated in a "shell" some distance around the mode](/2021/May/sexual-dimorphism-in-the-sequences-in-relation-to-my-gender-problems/#typical-point), because even though the per-unit-hypervolume probability _density_ is greatest at the mode, there's vastly more hypervolume in the hyperspace around it. The upshot is that typical people are atypical along _some_ dimensions, so normies can play the medianworld game, too. (Or they could, if the normies of our world were into worldbuilding.)
-[^planecrash-title]: The title is a pun, referring to both the airplane crash leading to Keltham's death in dath ilan, and how his resurrection in Golarion collides dath ilan with [the "planes" of existence of the _Pathfinder_ universe](https://pathfinderwiki.com/wiki/Great_Beyond).
+Dath ilan's cognitive superiority to the real world is a recurring theme in modern Yudkowsky's work. In the fictional canon, it's a focus of the story ["But Hurting People is Wrong"](https://www.glowfic.com/posts/4508), but even when not discussing fiction, Yudkowsky often makes sneering comments about "Earth" or "Earth people", apparently meant to disparage all actually existing humans for not living up to his fiction.
-[^medianworlds]: You might think that the thought experiment of imagining what someone's medianworld is like would only be interesting for people who are "weird" in our own world, thinking that our world is a medianworld for people who are normal in our world. But [in high-dimensional spaces, _most_ of the probability-mass is concentrated in a "shell" some distance around the mode](/2021/May/sexual-dimorphism-in-the-sequences-in-relation-to-my-gender-problems/#typical-point), because even though the per-unit-hypervolume probability _density_ is greatest at the mode, there's vastly _more_ hypervolume in the hyperspace around it. The upshot is that typical people are atypical along _some_ dimensions, so normies can play the medianworld game, too.
+One is led to believe that people who were deeply inspired by Yudkowsky's [Sequences](https://www.readthesequences.com/) (a series of influential posts about rationality published on the _Overcoming Bias_ blog largely between 2007 and 2009) should regard dath ilan as a rationalist utopia. After all, on the terms of the 2014 April Fools' Day joke, that's where the knowledge came from.
-Everyone in dath ilan receives rationality training from childhood, but knowledge and training deemed psychologically hazardous to the general population is safeguarded by an order of [Keepers of Highly Unpleasant Things it is Sometimes Necessary to Know](https://www.glowfic.com/replies/1612937#reply-1612937). AGI research takes place in a secret underground city; some unspecified form of social engineering steers the _hoi polloi_ away from thinking about the possibility of AI.
+And yet, for such a supposed rationalist utopia, it's remarkable the extent to which dath ilan's Society is portrayed as being organized around conspiracies to lie or otherwise cover up the truth—not just when forced to by dire matters of planetary security (as when keeping nuclear or AGI secrets), but seemingly for any somewhat plausible excuse whatsoever, including protecting the feelings of people who would be happier if kept ignorant. Evidently, there are _many_ truths existing which dath ilan fears and would wish unknown to the whole world.
-Something that annoyed me about the portrayal of dath ilan was their incredibly casual attitude towards hiding information for some alleged greater good, seemingly without considering that [there are benefits and not just costs to people knowing things](http://benjaminrosshoffman.com/humility-argument-honesty/).
+The contrast to the [sense of life](http://aynrandlexicon.com/lexicon/sense_of_life.html) portrayed in the Sequences is striking. The Sequences emphasized that you—yes, you, the reader—had an interest in having an accurate world-model. On the subject of confronting unpleasant hypotheses, the Sequences [gave this advice](https://www.readthesequences.com/Avoiding-Your-Beliefs-Real-Weak-Points) (bolding mine):
-You can, of course, make up a sensible [Watsonian](https://tvtropes.org/pmwiki/pmwiki.php/Main/WatsonianVersusDoylist) rationale for this. A world with much smarter people is more "volatile"; with more ways for criminals and terrorists to convert knowledge into danger, maybe you _need_ more censorship just to prevent Society from blowing itself up.
+> When you're doubting one of your most cherished beliefs, close your eyes, empty your mind, grit your teeth, and **deliberately think about whatever hurts the most**. Don't rehearse standard objections whose standard counters would make you feel better. Ask yourself what _smart_ people who disagree would say to your first reply, and your second reply. Whenever you catch yourself flinching away from an objection you fleetingly thought of, drag it out into the forefront of your mind. **Punch yourself in the solar plexus. Stick a knife in your heart, and wiggle to widen the hole.** In the face of the pain, rehearse only this:
+>
+>> What is true is already so.
+>> Owning up to it doesn't make it worse.
+>> Not being open about it doesn't make it go away.
+>> And because it's true, it is what is there to be interacted with. Anything untrue isn't there to be lived.
+>> People can stand what is true, for they are already enduring it.
-I'm more preoccupied by a [Doylistic](https://tvtropes.org/pmwiki/pmwiki.php/Main/WatsonianVersusDoylist) interpretation—that dath ilan's obsessive secret-Keeping reflects something deep about how the Yudkowsky of the current year relates to speech and information, in contrast to the Yudkowsky who wrote the Sequences. The Sequences had encouraged you—yes, _you_, the reader—to be as rational as possible. In contrast, the dath ilan mythos seems to portray advanced rationality as dangerous knowledge that people need to be protected from. ["The universe is not so dark a place that everyone needs to become a Keeper to ensure the species's survival,"](https://glowfic.com/replies/1861879#reply-1861879) we're told. "Just dark enough that some people ought to."
+Meanwhile, the dath ilan mythos depicts rationality itself as hazardous knowledge that people need to be protected from. Dath ilani Civilization is steered by a secretive order of [Keepers of Highly Unpleasant Things it is Sometimes Necessary to Know](https://www.glowfic.com/replies/1612937#reply-1612937). ["The universe is not so dark a place that everyone needs to become a Keeper to ensure the species's survival,"](https://glowfic.com/replies/1861879#reply-1861879) we're told. "Just dark enough that some people ought to." Ordinary dath ilani do receive rationality training, but it's implied to be deliberately crippled, featuring ["signposts around the first steps [towards becoming a Keeper], placed to warn dath ilani off starting down that path unless they mean it."](https://www.glowfic.com/replies/1799590#reply-1799590) The maxim that "That which can be destroyed by the truth should be" is described as being ["remembered as much for how it's false, as for how it's true, because among the things that truths can destroy is people."](https://www.glowfic.com/replies/1687922#reply-1687922)
-[...]
+Clearly, this is not a culture that cares about ordinary people being well-informed. Apparently, they believe that owning up to it _does_ make it worse, that the untrue _is_ there to be lived (for non-Keepers).
-For example, we are told of an Ordinary Merrin Conspiracy centered around a famous medical technician with a psychological need to feel unimportant, of whom ["everybody in Civilization is coordinating to pretend around her"](https://www.glowfic.com/replies/1764946#reply-1764946) that her achievements are nothing special, which is deemed to be kindness to her. It's like a reverse [Emperor Norton](https://en.wikipedia.org/wiki/Emperor_Norton) situation. (Norton was ordinary, but everyone around him colluded to make him think he was special; Merrin is special, but everyone around her colludes to make her think she's ordinary.)
+We might say that the algorithm that designed dath ilan's Civilization can be seen as systematically preferring deception. When I speak of an algorithm preferring deception, [what I mean is](https://www.lesswrong.com/posts/fmA2GJwZzYtkrAKYJ/algorithms-of-deception) that given a social problem, candidate solutions that involve deceiving the populace seem to be higher in dath ilani Civilization's implicit search ordering than solutions that involve informing the populace. If a solution adequately achieves its purpose by means of deception or secrecy, Civilization isn't going to reject it on that basis and keep searching for a solution that involves telling the truth.
-But _as_ a rationalist, I condemn the Ordinary Merrin Conspiracy as _morally wrong_, for the same [reasons I condemn the Emperor Norton Conspiracy](/2018/Feb/the-categories-were-made-for-man-to-make-predictions/#emperor-norton). As [it was taught to me on _Overcoming Bias_ back in the 'aughts](https://www.lesswrong.com/posts/HYWhKXRsMAyvRKRYz/you-can-face-reality): what's true is already so. Denying it won't make it better. Acknowledging it won't make it worse. And _because_ it is true, it is what is there to be interacted with. Anything untrue isn't there to be lived. People can stand what is true, _because they are already doing so_.
+Crucially, these are [functionalist](https://www.lesswrong.com/posts/sXHQ9R5tahiaXEZhR/algorithmic-intent-a-hansonian-generalized-anti-zombie) [criteria](http://benjaminrosshoffman.com/bad-faith-behavior-not-feeling/) of "preference" and "deception". It's about how Civilization is structured in a way that systematically encourages divergences between popular belief and reality. I'm _not_ positing that Civilization's Keepers and Legislators and Chief Executive are laughing maniacally and telling each other out loud, "I personally love it when non-Keepers have false beliefs; we need to do as much of that as possible—as a [terminal value](https://www.readthesequences.com/Terminal-Values-And-Instrumental-Values)!"
-In ["For No Laid Course Prepare"](https://glowfic.com/posts/6263), the story about how Merrin came to the attention of dath ilan's bureau of Exception Handling, we see the thoughts of a Keeper, Rittaen, who talks to Merrin. We're told that the discipline of modeling people mechanistically rather than [through empathy](https://www.lesswrong.com/posts/NLMo5FZWFFq652MNe/sympathetic-minds) is restricted to Keepers to prevent the risk of ["turning into an exceptionally dangerous psychopath"](https://glowfic.com/replies/1862201#reply-1862201). Rittaen [uses his person-as-machine Sight](https://glowfic.com/replies/1862204#reply-1862204) to infer that Merrin was biologically predisposed to learn to be afraid of having too much status.
+Rather, I'm positing that they don't care about non-Keepers having false beliefs. (They might care about [not technically lying](https://www.lesswrong.com/posts/PrXR66hQcaJXsgWsa/not-technically-lying), but that [turns out to be a weak constraint](https://www.lesswrong.com/posts/MN4NRkMw7ggt9587K/firming-up-not-lying-around-its-edge-cases-is-less-broadly).)
-Notwithstanding that Rittaen can be Watsonianly assumed to have detailed neuroscience skills that the author Doylistically doesn't know how to write, I am entirely unimpressed by the assertion that this idea is somehow _dangerous_, a secret that only Keepers can bear, rather than something _Merrin herself should be clued into_. "It's not [Rittaen's] place to meddle just because he knows Merrin better than Merrin does," we're told.
+If you're in the business of coming up with clever plans to solve problems, and you don't care about people having false beliefs, you mostly end up with clever plans that work by means of giving people false beliefs that trick them into doing what you want them to do (perhaps without technically lying).
-In the same story, Merrin is dressed up as a member of a fictional alien species, the Sparashki, due to having been summoned to the hospital from a fan convention with no time to change outfits. An agent from Exception Handling [tells Merrin that the bureau's Fake Conspiracy section is running an operation to plant evidence that Sparashki are real](https://glowfic.com/replies/1860952#reply-1860952), and asks Merrin not to contradict this, and Merrin just ... goes along with it.
+Why wouldn't you? There are more false maps than true maps. If you don't specifically care about [affirmatively telling the truth](http://benjaminrosshoffman.com/honesty-and-perjury/#Intent_to_inform), you mostly end up supplying false maps in order to control people's behavior by means of controling their information, because if you told them the truth, they wouldn't behave the way you want them to. Instrumental convergence is a harsh mistress.
-It's in-character for Merrin to go along with it, because she's a pushover. My question is, why is it okay that Exception Handling explicitly has a Fake Conspiracies section (!), any more than it would have been if FTX or Enron explicitly had a Fake Accounting department?
+### Interlude: Methodology for Worldbuilding Criticism; Or, I Can't Argue With Authorial Fiat
-Is it okay because the idea of humanoid aliens walking around in real life is sufficiently implausible that no one is "really fooled"? But this seems to be contradicted by the text, in which [the narrator tells us that Exception Handling deliberately does things that wouldn't otherwise make sense in order to make it harder to understand what's actually going on in Civilization](https://glowfic.com/replies/1860955#reply-1860955) (presumably, in order to protect the secret underground AGI alignment project). Even if very few dath ilani take the "Sparashki are real" ruse literally, it's still portrayed as successfully _kicking up epistemic dust_. If it were really the case that no one was being fooled about anything, then Exception Handling wouldn't have a strategic reason to do it.
+At this point, some readers might object that this kind of "dark" interpretation of a fictional universe oversteps the authority of the literary critic. One imagines that Yudkowsky doesn't particularly think of dath ilan as a world governed by deception. What grounds could I possibly have to argue that it is, given that he's the author and I'm not? Isn't that just making up my own fictional world and substituting it for the "real" dath ilan defined by Yudkowsky's authorial intent?
-Is it okay because dath ilan are the [designated good guys](https://tvtropes.org/pmwiki/pmwiki.php/Main/DesignatedHero)? Well, [so was FTX](https://forum.effectivealtruism.org/posts/sdjcH7KAxgB328RAb/ftx-ea-fellowships).
+But the craft of literature isn't a matter of merely conveying a fictional reality that existed fully formed in the author's imagination in advance of writing it down. The craft is about producing text that readers can use to build up their own model of the fictional world. The exacting labor of [converting vague ideas into definite text](http://www.paulgraham.com/words.html) is the difference between writing and daydreaming. We can accept [Word of God](https://tvtropes.org/pmwiki/pmwiki.php/Main/WordOfGod) as supplementary material where the text of a story is ambiguous or silent on a point of interest, but some kind of [Death of the Author](https://tvtropes.org/pmwiki/pmwiki.php/Main/DeathOfTheAuthor) stance is ultimately necessary for making sense of literature in a world in which telepathy doesn't exist and authors do occasionally die. The text is not a mere pointer to the "real" work inside the author's head; the text _is_ the work. That's the only way the technology of writing can function.
+
+Moreover, a Death of the Author stance seems particularly important for evaluating medianworlds. What makes the exercise of constructing a medianworld interesting is the challenge of envisioning the details of a realistic Society that would result given a population with an alternative [distribution of cognitive repertoires](/2020/Apr/book-review-human-diversity/), but where the same generalizations of biology, sociology, and economics that govern our own world are presumed to hold. If the world portrayed by the text [doesn't seem to hold together](/2022/Jun/comment-on-a-scene-from-planecrash-crisis-of-faith/) or has [unfortunate implications](https://tvtropes.org/pmwiki/pmwiki.php/Main/UnfortunateImplications) that the author doesn't acknowledge, it's the solemn duty of literary critics to point that out to less discerning readers.
+
+In analogy, mathematicians, like authors, are also in the business of creating imaginary worlds, but mathematical objects, once defined, can be examined on their own terms. A mathematician encountering [a deeply unsatisfying theorem about their new definition](https://blog.plover.com/math/major-screwups-4.html) understands that there can be no recourse in protesting, "But that's not how _I_ imagined it working." That would be failing to engage with the real difficulties of mathematical research. If you wanted different behavior, you should have written a better definition!
+
+Similarly, an author who says, "In _my_ medianworld, fully automated luxury gay space communism with central planning just works, because the populace is so smart and nice, unlike _Earth people_, who are so mean and dumb that they have to use _markets_ to allocate scarce resources" is failing to engage with the real difficulties of the medianworld exercise. Readers would have a right to be skeptical.
+
+Authors, of course, have much more wiggle room than mathematicians to try to salvage their cherished ideas. Rather than being forced back to the drawing board by an unwanted implication, a fiction writer finds it all too easy to simply add another sentence denying it. But the author's total freedom to specify the text necessarily interacts with readers' attempts to imagine a self-consistent universe that "projects into" that text. Short of an explicitly omniscient narrator declaring "And then a miracle occured", discerning readers will tend to reach for interprerations of the text that make sense—even if making sense entails casting doubt on the narrator's spin on the described events.
+
+Yudkowsky's 2009 story ["The Sword of Good"](https://www.yudkowsky.net/other/fiction/the-sword-of-good) is an incisive commentary on how unwary readers' moral compasses can be hijacked by authorial editorializing. If the story depicts our heroes wantonly slaughtering orcs, readers tend not to worry about the ethics of warfare: if the [designated hero](https://tvtropes.org/pmwiki/pmwiki.php/Main/DesignatedHero) is doing it, it presumably isn't a problem. But living creatures that are depicted as speaking language, having organized armies with complex tools, _&c._ are presumably sapient for the same reasons humans are. It's fair game for "The Sword of Good" to point that out—at least, short of the narrator _explicitly_ declaring, "Despite appearances, the orcs are unconscious [philosophical zombies](https://www.lesswrong.com/posts/fdEWWr8St59bXLbQr/zombies-zombies); killing them has no moral significance." (But a story that did declare that would be highly unusual, and basically conceding the critic's point about stories that didn't!)
+
+All I'm doing in this essay is holding the dath ilan mythos to the same standard that "The Sword of Good" holds classic fantasy tropes to. Maybe it's good to run a global conspiracy to keep people ignorant in order to protect their emotional well-being, and maybe those orcs deserved to die. But given a text that _does_ literally describe massive coverups or killing of human-like creatures, it's fundamentally fair game for literary critics to point that out, and prompt readers to think carefully about whether they should accept "it's fine because the good guys are doing it" or "it's fine because it's dath ilan, which is not Earth" as an implicit excuse.
+
+### The History Screen
+
+One of the distinctive features of dath ilani Civilization is that they've deleted their history. No one except a few specially cleared specialists ["in their own causally isolated bunker"](https://glowfic.com/replies/1688794#reply-1688794) is allowed to learn history before a point in a relatively recent past. (["[D]ecades ago, not millennia ago"](https://glowfic.com/replies/1789110#reply-1789110) as of Keltham's time.)
+
+This is an incredibly drastic measure, a civilizational lobotomy. (It perhaps bears worth mentioning that the most famous literary depiction of a Society that deleted its history is George Orwell's _Nineteen-Eighty-Four_, in which a totalitarian state maintains a Society in which "[n]othing exists except an endless present in which the Party is always right.") The full costs may not be obvious to many readers: if the immediate needs of living in the present don't directly require thinking about the past, what's the harm of not knowing?
+
+The problem is that [facts are causally entangled with each other](https://www.readthesequences.com/Entangled-Truths-Contagious-Lies). Forgetting one fact entails not just having to deal with not knowing that particular thing, but also not knowing everything it implies about other things in the world—which might be of little consequence for any one trivium, but is surely enormous for _all of human history_.
+
+In at least one case, the text depicts dath ilan as suffering practical consequences from its ignorance: a discussion of the eliezera's low happiness levels is cut short with ["How'd it happen? Nobody knows, at this point, they screened their history."](https://glowfic.com/replies/1812613#reply-1812613) (Generally, knowing how something happened is useful for figuring out how to remediate it.) Realistically, we can only surmise that there are many other cases where knowing history would be useful that the narrator hasn't gotten around to telling us.
+
+Given the costs, what could possibly justify the history screen, in an advanced Society that otherwise seems to value knowledge? A fantasy author could easily an invent an answer: maybe a psychic plague that spread through memories such that the infected must be not just physically quarantined, but forgotten. Medianworld authors have a tougher time insofar as we construe their genre as hard science fiction. Psychic plagues aren't real. What could _realistically_ justify the history screen?
-As another notable example of dath ilan hiding information for the alleged greater good, in Golarion, Keltham discovers that he's a sexual sadist, and deduces that Civilization has deliberately prevented him from realizing this, because there aren't enough corresponding masochists to go around in dath ilan. Having concepts for "sadism" and "masochism" as variations in human psychology would make sadists like Keltham sad about the desirable sexual experiences they'll never get to have, so Civilization arranges for them to _not be exposed to knowledge that would make them sad, because it would make them sad_ (!!).
+The answer the text gives leaves something to be desired. In "But Hurting People Is Wrong", we learn of the birth of modern dath ilan, when the Keepers noticed the AGI alignment problem and propose strict policy measures aimed at surviving the challenge.
-It did not escape my notice that when "rationalist" authorities _in real life_ considered public knowledge of some paraphilia to be an infohazard (ostensibly for the benefit of people with that paraphilia), I _didn't take it lying down_.
+For one, dath ilan's eugenics program is to optimize for intelligence even more than it already does. That makes sense: smarter alignment researchers have a better shot at solving the problem.
-This parallel between dath ilan's sadism/masochism coverup and the autogynephilia coverup I had fought in real life, was something I was only intending to comment on in passing in the present memoir, rather than devoting any more detailed analysis to, but as I was having trouble focusing on my own writing in September 2022, I ended up posting some critical messages about dath ilan's censorship regime in the "Eliezerfic" Discord server for reader discussion of _Planecrash_, using the sadism/masochism coverup as my central example.
+For another, restrictions on the manufacture of computers are imposed. That makes sense: if computer hardware were too capable and easily available before alignment researchers had the [serial time](https://www.lesswrong.com/posts/FS6NCWzzP8DHp4aD4/do-earths-with-slower-economic-growth-have-a-better-chance) to think the problem through, that would increase the risk of accidentally unleashing an unaligned intelligence explosion.
-What happens, I asked, to the occasional dath ilani free speech activists, with their eloquent manifestos arguing that Civilization would be better off coordinating on maps that reflect the territory, rather than coordinating to be a Keeper-managed zoo? (They _had_ to exist: in a medianworld centered on Yudkowsky, there are going to be a few weirdos who are +2.5 standard deviations on "speak the truth, even if your voice trembles" and −2.5 standard deivations on love of clever plots; this seems less weird than negative utilitarians, who were [established to exist](https://www.glowfic.com/replies/1789623#reply-1789623).) I _assumed_ they get dealt with somehow in the end (exiled from most cities? ... involuntarily cryopreserved?), but there had to be an interesting story about someone who starts out whistleblowing small lies (which Exception Handling allows; they think it's cute, and it's "priced in" to the game they're playing), and then just keeps _escalating and escalating and escalating_ until Governance decides to unperson him.
+For another, digital surveilance is imposed. That makes sense: Governance would want to know about any rogue AGI projects, so they can be shut down by force.
-Although Yudkowsky participated in the server, I had reasoned that my participation didn't violate my previous intent not to bother him anymore, because it was a publicly-linked Discord server with hundreds of members. Me commenting on the story for the benefit of the _other_ 499 people in the chat room wouldn't generate a notification _for him_, the way it would if I sent him an email or replied to him on Twitter.
+[And then we get this](https://www.glowfic.com/replies/1612939#reply-1612939):
-In the #dath-ilan channel of the server, Yudkowsky elaborated on the reasoning for the masochism coverup:
+> And so long as they were doing all that anyways, they might as well also carry out the less important but still useful operation of putting all of Civilization's past behind the most complete possible causal screen. That part wasn't as important, but still legitimately helpful; and doing it would help to overshadow the other changes, and lead to less attention going to the more dangerous places.
-> altruistic sadists would if-counterfactually-fully-informed prefer not to know, because Civilization is capped on the number of happy sadists. even if you can afford a masochist, which requires being very rich, you're buying them away from the next sadist to whom masochists were previously just barely affordable
+"So long as they were doing all that anyways, they might as well"! That's it? The only concrete benefit of enforcing ignorance (of history) we're told about is ... more ignorance (of the existence of the AGI project).
-In response to a question about how frequent sadism is among Keepers, Yudkowsky wrote:
+Keeping _the project's_ existence secret likely makes sense, if the surveillance measures are deemed insufficient to prevent rogue AGI projects. But classifying _all of human history_ along with it should require some enormously compelling reason, and one just isn't suppplied.
-> I think they're unusually likely to be aware, nonpracticing potential sexual sadists. Noticing that sort of thing about yourself, and then not bidding against the next sadist over for the limited masochist supply, and instead just operating your brain so that it doesn't hurt much to know what you can't have, is exactly the kind of cost you're volunteering to take on when you say you wanna be a Keeper.
-> that's archetypally exactly The Sort Of Thing Keepers Do And Are
+Is it that the Singularity is so important, the stakes so massive, that [no cost is too small to pay](https://tvtropes.org/pmwiki/pmwiki.php/Main/UtopiaJustifiesTheMeans) to make sure it goes well? That would be understandable. (As the saying goes, ["Shut up and multiply."](https://www.lesswrong.com/posts/r5MSQ83gtbjWRBDWJ/the-intuitions-behind-utilitarianism))
-> They choose not to, not just out of consideration for the next person in line, but because not harming the next person in line is part of the explicit bargain of becoming a Keeper.
-> Like, this sort of thing is exactly what you're signing up for when you throw yourself on the bounded rationality grenade.
-> Let the truth destroy what it can—but in you, not in other people.
+But the word choices make it seem like that's not what's going on. "So long as they were doing all that anyways, they might as well" is not the language of a tough-minded utilitarian who understands the gravity of the crimes they're about to commit, but is determined to proceed [for the good of the many](https://tvtropes.org/pmwiki/pmwiki.php/Main/TheNeedsOfTheMany). "So long as they were doing all that anyways, they might as well" is the language of someone who's never considered that what they're about to do might be a crime, that [people knowing things as benefits as well as costs](http://benjaminrosshoffman.com/humility-argument-honesty/).
-I objected (to the room, I told myself, not technically violating my prior intent to not bother Yudkowsky himself anymore) that "Let the truth destroy what it can—in yourself, not in other people" is such an _incredibly_ infantilizing philosophy. It's a meme that optimizes for shaping people (I know, _other_ people) into becoming weak, stupid, and unreflective, like Thellim's impression of Jane Austen characters. I expect people on Earth—not even "rationalists", just ordinary adults—to be able to cope with ... learning facts about psychology that imply that there are desirable sexual experiences they won't get to have.
+It would be remiss to conclude the section without considering a [Doylist rather than Watsonian](https://tvtropes.org/pmwiki/pmwiki.php/Main/WatsonianVersusDoylist) interpretation of the history screen. Rumored Word of God[^wog-rumor] states that one reason for the history screen is that it makes a better story for dath ilani isekai protagonists to be more ignorant of other ways of life than the power of their civilization would otherwise imply; it's more interesting to watch Keltham encounter a medieval monarchy if he doesn't know what a queen is.
-A user called Numendil insightfully pointed out that dath ilani might be skeptical of an Earthling saying that an unpleasant aspect our of existence is actually fine, for the same reason we would be skeptical of a resident of Golarion saying that; it makes sense for people from richer civilizations to look "spoiled" to people from poorer ones.
+[^wog-rumor]: I don't have a link for this; I think I heard Yudkowsky say it at a party.
-Other replies were more disturbing. One participant wrote:
+The Doylist storytelling case for the history screen is compelling insofar as the function of the dath ilan mythos is strictly to tell entertaining stories. However, such a prominent feature of the Society existing for storytelling reasons undermines the function of dath ilan as a utopian contrast to the real world, as empitomized by the incessant in- and out-of-story sneers at "Earth" (_i.e._, real) people.
-> I think of "not in other people" not as "infantilizing", but as recognizing independent agency. You don't get to do harm to other people without their consent, whether that is physical or pychological.
+### The Ordinary Merrin Conspiracy
+
+The story of Merrin provides us with a case study on dath ilani culture's penchant for deception in a lower-stakes context. Merrin is a famous emergency medical technician of exceptional endurance, who obsessively improves her skills by training on simulated disasters when she's not on duty saving QALYs in real life. We are [told of an Ordinary Merrin Conspiracy](https://www.glowfic.com/replies/1764946#reply-1764946)
+
+> wherein Merrin has some weird psychological hangup about believing she is a totally normal and ordinary person or even something of a struggling low achiever, and everybody in Civilization is coordinating to pretend around her that ordinary normal people totally get their weird Exception Handling training scenarios televised to a million watchers on a weekly basis.
+
+It's like a reverse [Emperor Norton](https://en.wikipedia.org/wiki/Emperor_Norton) situation. (Norton was ordinary, but everyone around him colluded to make him think he was special; Merrin is special, but everyone around her colludes to make her think she's ordinary.)
+
+One is given the sense that from a dath ilani moral perspective, this is heartwarming. Merrin doesn't want to feel important, so everyone in her world colludes to conceal evidence from her than she's important. Isn't that thoughtful?
+
+From the moral perspective of an _Overcoming Bias_ reader in 2008 "on Earth" (_i.e._, in real life) who places value on authenticity and truth in human affairs, this is unsettling and immoral, for the same reason [the Emperor Norton Conspiracy was immoral](/2018/Feb/the-categories-were-made-for-man-to-make-predictions/#emperor-norton). Merrin is living a lie, and everyone she trusts is in on it!
+
+The deception by the people in Merrin's life is especially galling in light of the fact that the situation that Merrin needs to be protected from is one of their own making. EMTs can't become famous on accident. If Merrin doesn't like attention, she should not have a TV show with a million viewers! (We can assume that most unusually skilled workers aren't celebrities in dath ilan, as in real life, not because their work is particularly secret, but because recording and editing it for public consumption is a cost that no one has particular reason to pay.) One almost wonders whether the deception is the point—that people in dath ilan enjoy "coordinating to pretend around" someone, and Merrin's psychological hangup presented them with a golden opportunity to put someone in a [_The Truman Show_](https://en.wikipedia.org/wiki/The_Truman_Show)-like psuedo-reality.
+
+["For No Laid Course Prepare"](https://glowfic.com/posts/6263), the story about how Merrin came to the attention of dath ilan's bureau of Exception Handling, shows us more examples of dath ilani conspiracy culture. In the story, Merrin is dressed up as a member of a fictional alien species, the Sparashki, due to having been summoned to the hospital from a fan convention with no time to change outfits. Merrin's coworkers falsely maintain to outsiders that Merrin always cosplays as a Sparashki while on duty. ["This is not considered a lie, in that it would be universally understood and expected that no one in this social circumstance would tell the truth,"](https://www.glowfic.com/replies/1857346#reply-1857346) the narrator tells us. The language used here is strikingly similar to that of one of the corrupt executives in [_Moral Mazes_](https://thezvi.wordpress.com/2019/05/30/quotes-from-moral-mazes/): "We lie all the time, but if everyone knows that we're lying, is a lie really a lie?"
+
+But if [everyone really knew](https://thezvi.wordpress.com/2019/07/02/everybody-knows/), what would be function of saying the false thing? On dath ilan (if not in real-world boardrooms), one supposes the answer is "Because it's fun"? But that just prompts the followup question: but what is the function of the brain giving out a "fun" reward in this context? It seems like at some point, there has to be the expectation of some cognitive system (although possibly not an entire "person") taking the signals literally.
+
+Indeed, in the described circumstances, it's hard to see how receivers of the Merrin-always-cosplays falsehood are supposed to figure out that the claim is not true. If it's universally understood and expected that no dath ilani would pass up a good trolling opportunity, it does not therefore follow that the people being trolled know it. (That's exactly what makes a good troll!)
+
+It's tempting to speculate that the eliezera are high in an "enjoying social deception" trait. Not-actually-plausible conspiracies that most people are in on are a [superstimulus](https://www.lesswrong.com/posts/Jq73GozjsuhdwMLEG/superstimuli-and-the-collapse-of-western-civilization) like zero-calorie sweetener: engineered to let everyone enjoy the thrill of lying, without doing serious damage to shared maps.
+
+An agent from Exception Handling [tells Merrin that the bureau's Fake Conspiracy section is running an operation to plant evidence that Sparashki are real](https://glowfic.com/replies/1860952#reply-1860952), and asks Merrin not to contradict this.
+
+One immediately wonders: why does Exception Handling have a Fake Conspiracies department? Why is it okay for a government agency to be pulling hoaxes on its own citizens, any more than it would have been if FTX or Enron in real life had a Fake Accounting department?
+
+Is it okay because dath ilan are the [designated good guys](https://tvtropes.org/pmwiki/pmwiki.php/Main/DesignatedHero)? Well, [so was FTX](https://forum.effectivealtruism.org/posts/sdjcH7KAxgB328RAb/ftx-ea-fellowships).
-I pointed out that this obviously applies to, say, religion. Was it wrong to advocate for atheism in a religious Society, where robbing someone of their belief in God might be harming them?
+Is it okay because the idea of humanoid aliens walking around in real life is sufficiently implausible that no one is "really fooled"?
-"Every society strikes a balance between protectionism and liberty," someone said. "This isn't news."
+The narrator soon tells us that [Exception Handling deliberately does things that wouldn't otherwise make sense in order to make it harder to understand what's actually going on in Civilization](https://glowfic.com/replies/1860955#reply-1860955) (presumably, in order to protect the secret AGI alignment project). But this is in tension with the potential rationale that no one is really being fooled. Even if very few dath ilani take the "Sparashki are real" ruse literally, it's still portrayed as successfully "kicking up epistemic dust": if it were really the case that no one was being fooled about anything, then Exception Handling wouldn't have a strategic reason to do it.
-It's not news about _humans_, I conceded. It was just—I thought people who were fans of Yudkowsky's writing in 2008 had a reasonable expectation that the dominant messaging in the local subculture would continue in 2022 to be _in favor_ of telling the truth and _against_ benevolently intended Noble Lies. It ... would be interesting to know why that changed.
+Later in the story, we see the thoughts of a Keeper, Rittaen, who talks to Merrin. We're told that the discipline of modeling people mechanistically rather than [through empathy](https://www.lesswrong.com/posts/NLMo5FZWFFq652MNe/sympathetic-minds) is restricted to Keepers to prevent the risk of ["turning into an exceptionally dangerous psychopath"](https://glowfic.com/replies/1862201#reply-1862201). Rittaen [uses his person-as-machine Sight](https://glowfic.com/replies/1862204#reply-1862204) to infer that Merrin was biologically predisposed to learn to be afraid of having too much status.
-Someone else said:
+Notwithstanding that Rittaen can be Watsonianly assumed to have detailed neuroscience skills that the author Doylistically doesn't know how to write, the assertion that _knowing too much about psychology_ is dangerous says something about dath ilan. The explanation of Merrin's psychology that we do see isn't dangerous. (I read it, and I haven't become a psychopath.) I think a culture that shared the values of _Overcoming Bias_ readers in 2008 would consider that knowledge something _Merrin herself should be clued into_. Not dath ilan, though. "It's not [Rittaen's] place to meddle just because he knows Merrin better than Merrin does," we're told. But apparently it _is_ the place of everyone else who can see Merrin's shyness to form the Ordinary Merrin Conspiracy, to keep her from knowing herself?
-> dath ilan is essentially a paradise world. In a paradise world, people have the slack to make microoptimisations like that, to allow themselves Noble Lies and not fear for what could be hiding in the gaps. Telling the truth is a heuristic for this world where Noble Lies are often less Noble than expected and trust is harder to come by.
+That's not an accusation of hypocrisy. The dath ilani are being consistent. In a Society that prizes freedom-from-infohazards over freedom-of-speech, it makes sense that telling people things would be meddling, but colluding to not tell them, isn't.
-I said that I thought people were missing this idea that the reason "truth is better than lies; knowledge is better than ignorance" is such a well-performing [injunction](https://www.lesswrong.com/posts/dWTEtgBfFaz6vjwQf/ethical-injunctions) in the real world (despite the fact that there's no law of physics preventing lies and ignorance from having beneficial consequences), is because [it protects against unknown unknowns](https://www.lesswrong.com/posts/E7CKXxtGKPmdM9ZRc/of-lies-and-black-swan-blowups). Of course an author who wants to portray an ignorance-maintaining conspiracy as being for the greater good, can assert by authorial fiat whatever details are needed to make it all turn out for the greater good, but _that's not how anything works in real life_.
+### Keltham and the Sadism/Masochism Coverup
-I started a new thread to complain about the attitude I was seeing (Subject: "Noble Secrets; Or, Conflict Theory of Optimization on Shared Maps"). When fiction in this world, _where I live_, glorifies Noble Lies, that's a cultural force optimizing for making shared maps less accurate, I explained. As someone trying to make shared maps _more_ accurate, this force was hostile to me and mine. I understood that "secrets" and "lies" are not the same thing, but if you're a consequentialist thinking in terms of what kinds of optimization pressures are being applied to shared maps, [it's the same issue](https://www.lesswrong.com/posts/YptSN8riyXJjJ8Qp8/maybe-lying-can-t-exist): I'm trying to steer _towards_ states of the world where people know things, and the Keepers of Noble Secrets are trying to steer _away_ from states of the world where people know things. That's a conflict. I was happy to accept Pareto-improving deals to make the conflict less destructive, but I wasn't going to pretend the pro-ignorance forces were my friends just because they self-identified as "rationalists" or "EA"s. I was willing to accept secrets around nuclear or biological weapons, or AGI, on "better ignorant than dead" grounds, but the "protect sadists from being sad" thing wasn't a threat to anyone's life; it was _just_ coddling people who can't handle reality, which made _my_ life worse.
+The story of Keltham's sexuality provides another case study on dath ilani Civilization's predilection for deception. In Golarion, Keltham discovers that he's a sexual sadist, and infers that ["not many sadists in dath ilan know what they are and Civilization tries to prevent us from finding out, because dath ilan does _not_ have masochists."](https://glowfic.com/replies/1735044#reply-1735044) (The notion of an evolved organism preferring pain in any context strikes him as implausible.) Word of God clarifies that masochists do exist at low frequencies, but that Keltham's guess is otherwise correct; there is a deliberate effort (by the Keepers?—the exact agency responsible isn't clear) to prevent public knowledge that sadism and masochism even exist.
-I wasn't buying the excuse that secret-Keeping practices that wouldn't be okay on Earth were somehow okay on dath ilan, which was asserted by authorial fiat to be sane and smart and benevolent enough to make it work. Alternatively, if I couldn't argue with authorial fiat: the reasons why it would be bad on Earth (even if it wouldn't be bad in the author-assertion paradise of dath ilan) are reasons why _fiction about dath ilan is bad for Earth_.
+The naïve utilitarian case for the coverup is straightforward. There aren't enough masochists to go around. People with a latent inclination towards sadism are better off not being self-aware about it, because if they knew what they wanted, they would feel sad about not being able to have it.
-And just—back in the 'aughts, I said, Robin Hanson had this really great blog called _Overcoming Bias_. (You probably haven't heard of it.) I wanted that _vibe_ back, of Robin Hanson's blog in 2008—the will to _just get the right answer_, without all this galaxy-brained hand-wringing about who the right answer might hurt.
+The non-naïve utilitarian case is less straightforward, when you consider the second-order effects of censoring scientific information about psychology. The existence of sadism and masochism are facts about the world that you would expect to be mentioned in the sexuality section of psychology textbooks. If dath ilan is censoring real conditions from its psychology textbooks in order to avoid emotional harm to people with those conditions, that's a constraint on the extent to which their Society is in touch with reality.
-(_Overcoming Bias_ had actually been a group blog then, but I was enjoying the æsthetic of saying "Robin Hanson's blog" (when what I had actually loved about _Overcoming Bias_ was Yudkowsky's Sequences) as a way of signaling contempt for the Yudkowsky of the current year.)
+Again, [facts are connected to each other](https://www.lesswrong.com/posts/wyyfFfaRar2jEdeQK/entangled-truths-contagious-lies), which both
-I would have expected a subculture descended from the memetic legacy of Robin Hanson's blog in 2008 to respond to that tripe about protecting people from the truth being a form of "recognizing independent agency" with something like—
+[TODO—
+ * They do manage to censor sex information for children, but "age" is a Schelling cateogry in human Society; you can limit epistemic damage somewhat
-"Hi! You must be new here! Regarding your concern about truth doing harm to people, a standard reply is articulated in the post ["Doublethink (Choosing to be Biased)"](https://www.lesswrong.com/posts/Hs3ymqypvhgFMkgLb/doublethink-choosing-to-be-biased). Regarding your concern about recognizing independent agency, a standard reply is articulated in the post ["Your Rationality Is My Business"](https://www.lesswrong.com/posts/anCubLdggTWjnEvBS/your-rationality-is-my-business)."
+ * What about the costs of all the other recursive censorship you'd have to do to keep the secret? (If a biography mentioned masochism in passing along with many other traits of the subject, you'd need to either censor the paragraphs with that detail, or censor the whole book. Those are real costs, even under a soft-censorship regime where people can give special consent to access "Ill Advised" products.) Maybe latent sadists could console themselves with porn if they knew, or devote their careers to making better sex robots, just as people on Earth with non-satisfiable sexual desires manage to get by. (I _knew some things_ about this topic.) What about dath ilan's heritage optimization (read: eugenics) program? Are they going to try to breed more masochists, or fewer sadists, and who's authorized to know that? And so on.
-—or _something like that_. Not that the reply needed to use those particular Sequences links, or _any_ Sequences links; what's important is that someone needed to counter to this very obvious [anti-epistemology](https://www.lesswrong.com/posts/XTWkjCJScy2GFAgDt/dark-side-epistemology).
+ * transsexualism example
-And what we actually saw in response to the "You don't get to do harm to other people" message was ... it got 5 "+1" emoji-reactions.
+ * "They're better off not knowing about masochists that they can't have" disregards other ways of dealing with it: fantasy, porn, inventing in better sex robots, implications for eugenic policy ...
-Yudkowsky [chimed in to point out that](/images/yudkowsky-it_doesnt_say_tell_other_people.png) "Doublethink" was about _oneself_ not reasonably being in the epistemic position of knowing that one should lie to oneself. It wasn't about telling the truth to _other_ people.
+* Difficulties of recursive censorship; do the people who can afford it have to stay in the closet for the benefit of people like Keltham? You could claim that they're so good at coordinating that they negotiated this, but how does the negotiation work when the people the deal allegedly benefits don't know about it?
-On the one hand, fair enough. My generalization from "you shouldn't want to have false beliefs for your own benefit" to "you shouldn't want other people to have false beliefs for their own benefit" (and the further generalization to it being okay to intervene) was not in the text of the post itself. It made sense for Yudkowsky to refute my misinterpretation of the text he wrote.
+ * Word of God talks about "affording" a masochist—suggesting not just that classism and sex work are legal, but institutionalized??
-On the other hand—given that he was paying attention to this #overflow thread anyway, I might have naïvely hoped that he would appreciate what I was trying to do?—that, after the issue had been pointed out, he would decided that he _wanted_ his chatroom to be a place where we don't want other people to have false beliefs for their own benefit?—a place that approves of "meddling" in the form of _telling people things_.
+ * the empirical setting of probabilities and utilities just happening to settle such that ignorance is the best policy
-The other chatroom participants mostly weren't buying what I was selling.
+ * Keltham thought evolutionary psychology ruled out the possibility of masochists;
-A user called April wrote that "the standard dath ilani has internalized almost everything in the sequences": "it's not that the standards are being dropped[;] it's that there's an even higher standard far beyond what anyone on earth has accomplished". (This received a checkmark emoji-react from Yudkowsky, an indication of his agreement/endorsement.)
+ * It would be one thing if BDSM orientation was a self-fulfilling propehecy. You can't miss an addictive drug. The text seems to suggest that "obligate sadism" is a real thing that already exists. It's not analogous to a drug that you can't miss if no one ever invents it.
-Someone else said he was "pretty leery of 'ignore whether models are painful' as a principle, for Earth humans to try to adopt," and went on to offer some thoughts for Earth. I continued to maintain that it was ridiculous that we were talking of "Earth humans" as if there were any other kind—as if rationality in the Yudkowskian tradition wasn't something to aspire to in real life.
+Comparisons to other possible non-practicable minority sexualities may be instructive on the perils of deleting concepts. Imagine a world where homosexuality couldn't be safely practiced because [an engineered plague made it amount to certain death](https://www.gregegan.net/MISC/MORAL/Moral.html). Would we then approve of censoring the concept itself, forbidding the knowledge that same-sex attraction was ever a thing? And homosexuality has other correlates. [(The stereotypes of masculine lesbians and feminine gay men are empirically grounded.)](/papers/lippa-gender-related_traits_in_gays.pdf) Given the plague, would it be better not to notice the clusters—for gay people to possibly be left with some vague sense that there's something not normal about them, and not have a concept to name it?
-Dath ilan [is _fiction_](https://www.lesswrong.com/posts/rHBdcHGLJ7KvLJQPk/the-logical-fallacy-of-generalization-from-fictional), I pointed out. Dath ilan _does not exist_. I thought it was a horrible distraction to try to see our world through Thellim's eyes and feel contempt over how much better things must be on dath ilan (which, to be clear, again, _does not exist_), when one could be looking through the eyes of an ordinary reader of Robin Hanson's blog in 2008 (the _real_ 2008, which _actually happened_), and seeing everything we've lost.
+Or consider how our Society handles pedophilia. This is a sexuality that we do not want to be practiced, so we stigmatize it and criminalize its practice. We do not censor the concept. Given that the tendency exists (and is not merely a cultural self-fulfilling prophecy that wouldn't exist if it weren't named), it's better to have a concept about it that can be reasoned about, rather than a blank spot on our map. Imagine a young man noticing he's attracted to children—or for that matter, an abused child—having _no idea what was happening to them_, because Society's benevolent censors decided that it was better that people not have concepts for this aspect of reality. Is that going to end well? How do you get help for a problem you're not supposed to have a word for?
-[As it was taught to me then](https://www.lesswrong.com/posts/iiWiHgtQekWNnmE6Q/if-you-demand-magic-magic-won-t-help): if you demand Keepers, _Keepers won't help_. If I'm going to be happy anywhere, or achieve greatness anywhere, or learn true secrets anywhere, or save the world anywhere, or feel strongly anywhere, or help people anywhere—I may as well do it _on Earth_.
+]
-The thread died out soon enough. I had some more thoughts about dath ilan's predilection for deception, of which I typed up some notes for maybe adapting into a blog post later, but there was no point in wasting any more time on Discord.
+### Kitchen Knives
-On 29 November 2022 (four years and a day after the "hill of meaning in defense of validity" Twitter performance that had ignited my rationalist civil war), Yudkowsky remarked about the sadism coverup again:
+[TODO—
-> Keltham is a romantically obligate sadist. This is information that could've made him much happier if masochists had existed in sufficient supply; Civilization has no other obvious-to-me-or-Keltham reason to conceal it from him.
+ * Word of God says, "there's also a deliberate semiconspiracy whereby movies make standard kitchen knives look much deadlier and easier to wield than they actually are, so that people who suddenly go nuts won't improvise much more dangerous weapons than that"
+ * "too nuts to see past cached thoughts" presupposes a specific model of mental illness that I think is importantly backwards; while I was psychotic, my problem was the _opposite_ of not being able to "see past cached thoughts": on the contrary, I was having _lots_ of new ideas that I had never thought of before (<http://zackmdavis.net/blog/2017/02/cognitive-bayesian-therapy-i/>), and would never have thought of in my normal state—it's just that, after I got some sleep, I decided that all of my new ideas were clearly false, _which is why_ we call the psychotic state "crazy", and the normal state "sane".
+ * You can't leak to the general public without leaking to crazy people. Where do you think they come from?
+ * the mechanism by which the plan positively contributes to the goal, is deception: the plan works _because_ if people who want to commit violence have _less accurate beliefs_ about how to effectively commit violence, they will be less effective at their goal of committing violence.
+ * If you were just trying to reduce murders, is this hte plan you would pick? Obviously not. (Think about how you would solve the problem "on Earth"!) Making it easier to commit people is a much more direct mechanism. The algorithm that promoted the movie-misportrayal to your attention is one that favors deception.
+ * "It's targeted at crazy people" excuse suggests that it's OK to lie to people you've labeled as crazy. But Yudkowsky infamously says this about all Earthlings. What should we infer about whether we should trust him?
+ * "i presume ilani who notice the trope also then deduce the obvious consequence of the trope"
+ * If anything, "there's a deliberate conspiracy to mislead people about the difficulty of murder by stabbing" is the kind of thought that someone is _more likely_ to have as a paranoid delusion than otherwise! It's as if dath ilan is an exotic environment where mind-states that are tuned to hypothesize conspiracies everywhere are _adaptive_, in contrast to how on Earth, people who see conspiracies everywhere are wrong.
+ * this is correlated insanity that loves clever deceptions, not actually being smart and doing a neutral policy search
+ * A much larger cost is the frustration of ornery, nitpicky nerds who _want realistic fiction_, and don't want their favorite artform perverted to _manipulate extremely rare criminal-insanity cases_. Earth has Andy Weir fans and Greg Egan fans! When Andy Weir or Greg Egan sit down to write a novel or a screenplay, they are trying to paint a picture of _a world that makes sense_. They are not trying to manipulate rare social outcomes. Is that that kind of respect-for-reality just _not a thing_ in dath ilani art? Are they so uniformly, monomanically obsessed with trolling their perceived inferiors, that the voice of, "Knives Don't Actually Work That Way; Knives Don't Actually Work That Way _Even When Saying That Out Loud Increases Murders_; Knives Don't Actually Work That Way _Even When a Prediction Market Says That Saying That Out Loud Increases Murders_", has no power in the movie industry whatsoever?
+]
-Despite the fact that there was no point in wasting any more time on Discord, I decided not to resist the temptation to open up the thread again and dump some paragraphs from my notes on the conspiracies of dath ilan.
+### Replies to Objections
-If we believe that [IQ research validates the "Jews are clever" stereotype](https://web.mit.edu/fustflum/documents/papers/AshkenaziIQ.jbiosocsci.pdf), I wondered if there's a distinct (albeit probably correlated) "enjoying deception" trait that validates the "Jews are sneaky" stereotype? If dath ilan is very high in this "sneakiness" trait (relative to Earth Jews), that would help explain all the conspiracies![^edgy-anti-semitism]
+This essay has had some negative things to say about dath ilani culture.
-[^edgy-anti-semitism]: It probably would have been possible to bring up the sneakiness-trait hypothesis in a less edgy way, but I didn't care to.
-Not-actually-plausible conspiracies that everyone is in on (like "Sparashki are real") are a [superstimulus](https://www.lesswrong.com/posts/Jq73GozjsuhdwMLEG/superstimuli-and-the-collapse-of-western-civilization) like zero-calorie sweetener: engineered to let everyone enjoy the thrill of lying, without doing any real damage to shared maps.
-In "For No Laid Course Prepare", Merrin's coworkers falsely maintain to outsiders that Merrin always cosplays as a Sparashki while on duty. ["This is not considered a lie, in that it would be universally understood and expected that no one in this social circumstance would tell the truth,"](https://www.glowfic.com/replies/1857346#reply-1857346) the narrator tells us. The language used here is strikingly similar to that of one of the corrupt executives in [_Moral Mazes_](https://thezvi.wordpress.com/2019/05/30/quotes-from-moral-mazes/): "We lie all the time, but if everyone knows that we're lying, is a lie really a lie?"
+#### Infohazardville
-But if [everyone really knew](https://thezvi.wordpress.com/2019/07/02/everybody-knows/), what would be _function_ of saying the false thing? On dath ilan (if not in Earth boardrooms), one supposes the answer is "Because it's fun"? But that just prompts the followup question: but what is the function of the brain giving out a "fun" reward in this context? It seems like at _some_ point, there has to be the expectation of _some_ cognitive system (although possibly not an entire "person") taking the signals literally.[^funny-or-powerful-falsehood]
-[^funny-or-powerful-falsehood]: This is why, when I notice myself misrepresenting my actual beliefs or motivations because I think it's funny or rhetorically powerful, I often take care to disclaim it immediately, precisely because I _don't_ think that "everybody knows"; I'm not going to give up on humor or powerful rhetoric, but I'm also not going to delude myself into thinking it's "zero-calorie": people who don't "get the joke" _are_ going to be misled, and I don't think it's unambigously "their fault" for not being able to read my "intent" to arbitrary precision. But maybe dath ilan is (by authorial fiat) sufficiently good at achieving common knowledge in large groups that they _can_ pull off a zero-calorie "everyone knows" conspiracy without damaging shared maps?
-The existence of such a widespread sneakiness/"taste for deception" trait among the eliezera, in conjunction with their culture just not particularly valuing public knowledge (because they assume everything important is being handled by the Keepers), explains the recurring conspiracies and coverups, like the Ordinary Merrin Conspiracy, Exception Handling's fabrication of evidence for Sparashki being real, the sadism/masochism coverup, and [the village that deliberately teaches anti-redhead bigotry to children in order to test the robustness of dath ilan's general humanism indoctrination](https://www.lesswrong.com/posts/uyBeAN5jPEATMqKkX/lies-told-to-children-1).
-I stress (at least now, even if I didn't do a good job of explaining it at the time) that this hypothesis _doesn't_ require dath ilani to be cartoon villains who hate knowledge and want people to be ignorant. Just that, as a result of the widespread sneakiness trait and their outsourcing information-process to the Keepers, in the course of trying to accomplish other things, plans-that-involve-conspiracies are often higher in their search ordering than plans-that-involve-keeping-people-informed.
+####
-I claimed that there was a hidden-core-of-rationality thing about a culture that values living in truth, that the dath ilani didn't have. In previous discussion of the Sparashki example, a user called lc had written, "If you see someone wearing an elf costume at work and conclude elves are real and make disastrous decisions based on that conclusion you are mentally deranged". And indeed, you would be mentally deranged if you did that _on Earth_, because we don't have an elves-are-real conspiracy on Earth.
-In elves-are-real conspiracy-world, you (Whistleblower) see someone (Conspirator) wearing an elf costume at work and say, "Nice costume." They say, "What costume?" You say, "I see that you're dressed like an elf, but elves aren't real." They say, "What do you mean? Of course elves are real. I'm right here." You say, "You know exactly what I mean."
+[TODO—
+ * Counterargument: dath ilan lets you immigrate to infohazardville. Reply: sure, I agree that a Society that marginalizes truth-tellers is better than one than executes them. It's still fundamentally fair game for me to point out that it's not consilient with the values of _Overcoming Bias_ readers in 2008.
+ * Counterargument: as a reductio, am I opposed to movie and homework exercise spoilers, too? Reply: No, I recognize spoiler protection as a legitimate class of infohazard. (And AGI/nuclear as an social exfohazard, and threats as a legitimate infohazard.) And the concealing eugenics predictions, self-fulfilling prophecy
+ * Counterargument: the text says that regular dath ilani do believe "that which can be destroyed" eventually, just not immediately. Reply: this would be more believable with a timetable; I can understand "I don't want to deal with this yet", but there's no indication that Merrin or Keltham are supposed to figure it out.
+ * Counterargument: but the gaslighting is supposed to inculcate distrust of authority. Reply: clearly it doesn't work, given how much everyone trusts the government that is lying to them all the time?
+ * Numendil's point: dath ilani should look spoiled to us, for the same reason we look spoiled to Golarionites
+]
-It would appear that there's a conflict between Conspirator (who wants to maintain a social reality in which they're an elf, because it's fun, and the conspiracy is sufficiently outlandish that it's assumed that no one is "really" being deceived) and Whistleblower (who wants default social reality to map to actual reality; make-believe is fine at a designated fandom convention which has designated boundaries, but let's be serious at work, where your coworkers are trying to make a living and haven't opted-in to this false social reality).
+### Conclusion
-I was skeptical that a culture where people collude to maintain a fake social reality at their job in a hospital, and everyone else is expected to play along because it's fun, really has this living-in-truth thing. People play those social-reality games on Earth, too, and when _they_ say no one is being deceived, they're _definitely_ lying about that, and I doubted that the eliezera were actually built that differently.
+> That's the point where it starts to look like somebody is trying to hide something, or fool future generations, and not just respond to a strange threat in a way where they're happy to let you take all the precautions required to make sure nobody's fooling anybody.
+https://www.glowfic.com/replies/1813073#reply-1813073
projects / Ultimately_Untrue_Thought.git / blobdiff