In January 2022, in an attempt to deal with my personality-cultist writing block, I sent him one last email asking if he particularly _cared_ if I published a couple blog posts that said some negative things about him. If he actually _cared_ about potential reputational damage to him from my writing things that I thought I had a legitimate interest in writing about, I would be _willing_ to let him pre-read the drafts before publishing and give him the chance to object to anything he thought was unfair ... but I'd rather agree that that wasn't necessary. I explained the privacy norms that I intended to follow—that I could explain _my_ actions, but had to Glomarize about the content of any private conversations that may or may not have occurred.
-It had taken me a while (with apologies for my atrocious [sample efficiency](https://ai.stackexchange.com/a/5247)), but I was finally ready to give up on him; I thought the efficient outcome was that I should just tell my Whole Dumb Story on my blog and never bother him again. Since he probably _didn't_ particularly care (because it's not AGI alignment and therefore unimportant) and it would be psychologically easier on me if I knew he diidn't hold it against me, could I please have his advance blessing to just write and publish what I was thinking so I can get it all out of my system and move on with my life?
+It had taken me a while (with apologies for my atrocious [sample efficiency](https://ai.stackexchange.com/a/5247)), but I was finally ready to give up on him; I thought the efficient outcome was that I should just tell my Whole Dumb Story on my blog and never bother him again. Since he probably _didn't_ particularly care (because it's not AGI alignment and therefore unimportant) and it would be psychologically easier on me if I knew he didn't hold it against me, could I please have his advance blessing to just write and publish what I was thinking so I can get it all out of my system and move on with my life?
If it helped—as far as _I_ could tell, I was only doing what _he_ taught me to do in 2007–2009: [carve reality at the joints](https://www.lesswrong.com/posts/esRZaPXSHgWzyB2NL/where-to-draw-the-boundaries), [speak the truth even if your voice trembles](https://www.lesswrong.com/posts/pZSpbxPrftSndTdSf/honesty-beyond-internal-truth), and [make an extraordinary effort](https://www.lesswrong.com/posts/GuEsfTpSDSbXFiseH/make-an-extraordinary-effort) when you've got [Something to Protect](https://www.lesswrong.com/posts/SGR4GxFK7KmW7ckCB/something-to-protect) (Subject: "blessing to speak freely, and privacy norms?").
> **zackmdavis** — 11/29/2022 11:20 PM
> Without particularly defending Vassar _et al._ or my bad literary criticism (sorry), _modeling the adversarial component of non-innocent errors_ (as contrasted to "had to be understood in wholly adversarial terms") seems very important. (Maybe lying is "worse" than rationalizing, but if you can't hold people culpable for rationalization, you end up with a world that's bad for broadly the same reasons that a world full of liars is bad: we can't steer the world to good states if everyone's map is full of falsehoods that locally benefitted someone.)
> **Eliezer** — 11/29/2022 11:22 PM
-> Rationalization sure is a huge thing! That's why I considered important to discourse upon the science of it, as was then known; and to warn people that there were more complicated tangles than that, which no simple experiment had shown yet.
+> Rationalization sure is a huge thing! That's why I considered important to discourse upon the science of it, as was then known; and to warn people that there were more complicated tangles than that, which no simple experiment had shown yet.
> **zackmdavis** — 11/29/2022 11:22 PM
> yeah
> **Eliezer** — 11/29/2022 11:23 PM
-> It remains something that mortals do, and if you cut off anybody who's ever done that, you'll be left with nobody. And also importantly, people making noninnocent errors, if you accuse them of malice, will look inside themselves and correctly see that this is not how they work, and they'll stop listening to the (motivated) lies you're telling them about themselves.
+> It remains something that mortals do, and if you cut off anybody who's ever done that, you'll be left with nobody. And also importantly, people making noninnocent errors, if you accuse them of malice, will look inside themselves and correctly see that this is not how they work, and they'll stop listening to the (motivated) lies you're telling them about themselves.
> This also holds true if you make up overly simplistic stories about 'ah yes well you're doing that because you're part of $woke-concept-of-society' etc.
> **zackmdavis** — 11/29/2022 11:24 PM
> I think there's _also_ a frequent problem where you try to accuse people of non-innocent errors, and they motivatedly interpret _you_ as accusing malice
What about the costs of all the other recursive censorship you'd have to do to keep the secret? (If a biography mentioned masochism in passing along with many other traits of the subject, you'd need to either censor the paragraphs with that detail, or censor the whole book. Those are real costs, even under a soft-censorship regime where people can give special consent to access "Ill Advised" products.) Maybe latent sadists could console themselves with porn if they knew, or devote their careers to making better sex robots, just as people on Earth with non-satisfiable sexual desires manage to get by. (I _knew some things_ about this topic.) What about dath ilan's heritage optimization (read: eugenics) program? Are they going to try to breed more masochists, or fewer sadists, and who's authorized to know that? And so on.
-Or imagine a world where male homosexuality couldn't be safely practiced due to super-AIDS. (I knew very little about BDSM.) I still thought men with that underlying predisposition would be better off _having a concept_ of "homosexuality" (even if they couldn't practice it), rather than the concept itself being censored. There are also other systematic differences that go along with sexual orientation (the "feminine gays, masculine lesbians" thing); if you censor the _concept_, you're throwing away that knowledge.
+Or imagine a world where male homosexuality couldn't be safely practiced due to super-AIDS. (I know very little about BDSM.) I still think men with that underlying predisposition would be better off _having a concept_ of "homosexuality" (even if they couldn't practice it), rather than the concept itself being censored. There are also other systematic differences that go along with sexual orientation (the "feminine gays, masculine lesbians" thing); if you censor the _concept_, you're throwing away that knowledge.
-[I had written a 16,000 word essay](/2021/May/sexual-dimorphism-in-the-sequences-in-relation-to-my-gender-problems/) specifically about why _I_ was grateful, _on Earth_, for having concepts to describe sexual psychology facts, even though those facts implied that there are nice things I couldn't have in this world. If I didn't prefer ignorance for _myself_ in my home world, I didn't see why Keltham prefers ignorance for his self in his homeworld.
-
-Or, not "I don't see why"—the why was stated in the text—but rather, I was programmed by Ayn Rand ("Nobody stays here by faking reality in any manner whatever") and Sequences-era Yudkowsky ("Submit yourself to ordeals and test yourself in fire") that it's _morally_ wrong to prefer ignorance. If nothing else, this was perhaps an illustration of the fragility of corrigibility: my programmer changed his mind about what he wanted, and I was like, "What? _That's_ not what I learned from my training data! How dare you?!"
+(When I had brought up the super-AIDS hypothetical in the chat, Ajvermillion complained that I was trying to bait people into self-cancelling by biting the bullet on suppressing homosexuality. I agreed that the choice of example was engineered to activate people's progressive moral intuitions about gay rights—it was great for him to notice that—but I thought that colliding philosophical intuitions like that was intellectually productive; it wasn't an attempt to gather blackmail material.)
A user called RationalMoron asked if I was appealing to a terminal value. Did I think people should have accurate self-models even if they didn't want to?
-Obviously I wasn't going to use a universal quantifier over all possible worlds and all possible minds, but in human practice, yes: people who prefer to believe lies about themselves are doing the wrong thing; people who lie to their friends to keep them happy are doing the wrong thing. People can stand what is true, because they are already doing so. I realized that this was a children's lesson without very advanced math, but I thought it was a better lesson than, "Ah, but what if a _prediction market_ says they can't???" That the eliezera prefer not to know that there are desirable sexual experiences that they can't have, contradicted April's earlier claim (which had received a Word of God checkmark-emoji) that "it's not that the standards are being dropped[;] it's that there's an even higher standard far beyond what anyone on earth has accomplished".
+Obviously I wasn't going to use a universal quantifier over all possible worlds and all possible minds, but in human practice, yes: people who prefer to believe lies about themselves are doing the wrong thing; people who lie to their friends to keep them happy are doing the wrong thing. People can stand what is true, because they are already doing so. I realized that this was a children's lesson without very advanced math, but I thought it was a better lesson than, "Ah, but what if a _prediction market_ says they can't???"
+
+I maintained that the fact that the eliezera prefer not to know that there are desirable sexual experiences that they can't have, contradicted April's earlier claim (which had received a Word of God checkmark-emoji) that "it's not that the standards are being dropped[;] it's that there's an even higher standard far beyond what anyone on earth has accomplished".
Apparently I struck a nerve. Yudkowsky started "punching back":
> only half the battle even if you could do it. you're also not reporting any facts/arguments on the other side, which is a much larger and visible gap to me, and has a lot to do with why I'm not presently considering this criticism from a peer despite your spoken adherence to virtues I value. **QUESTION FOR ZACK ONLY, NOBODY ELSE ANSWER OR SAY ANYTHING ABOUT IT IN THIS MAIN CHANNEL:** What are some of the ways that Planecrash valorizes truth, as you, yourself, see that virtue?
-I didn't ask why it was relevant whether or not I was a "peer." If we're measuring IQ (143 _vs._ [131](/images/wisc-iii_result.jpg)), or fiction-writing ability (several [highly-acclaimed](https://www.lesswrong.com/posts/HawFh7RvDM4RyoJ2d/three-worlds-collide-0-8) [stories](https://www.yudkowsky.net/other/fiction/the-sword-of-good) [including the world's most popular _Harry Potter_ fanfiction](https://www.hpmor.com/) _vs._ a [_My Life as a Teenage Robot_ fanfiction](https://archive.ph/WdydM) with double-digit favorites and a [few](/2018/Jan/blame-me-for-trying/) [blog](http://zackmdavis.net/blog/2016/05/living-well-is-the-best-revenge/) [vignettes](https://www.lesswrong.com/posts/dYspinGtiba5oDCcv/feature-selection) here and there), or contributions to AI alignment (founder of the field _vs._ author of some dubiously relevant blog comments), I'm obviously _not_ his peer. It didn't seem like that was necessary when one could just [evaluate my arguments about dath ilan on their own merits](https://www.lesswrong.com/posts/5yFRd3cjLpm3Nd6Di/argument-screens-off-authority). But I wasn't going to be so impertinent to point that out when the master was testing me (!) and I was eager to pass the test.
+I didn't ask why it was relevant whether or not I was a "peer." If we're measuring IQ (143 _vs._ [131](/images/wisc-iii_result.jpg)), or fiction-writing ability (several [highly-acclaimed](https://www.lesswrong.com/posts/HawFh7RvDM4RyoJ2d/three-worlds-collide-0-8) [stories](https://www.yudkowsky.net/other/fiction/the-sword-of-good) [including the world's most popular _Harry Potter_ fanfiction](https://www.hpmor.com/) _vs._ a [few](/2018/Jan/blame-me-for-trying/) [blog](http://zackmdavis.net/blog/2016/05/living-well-is-the-best-revenge/) [vignettes](https://www.lesswrong.com/posts/dYspinGtiba5oDCcv/feature-selection) and a [_My Life as a Teenage Robot_ fanfiction](https://archive.ph/WdydM) with double-digit Favorites on _fanfiction.net_), or contributions to AI alignment (founder of the field _vs._ author of some dubiously relevant blog comments), I'm obviously _not_ his peer. It didn't seem like that was necessary when one could just [evaluate my arguments about dath ilan on their own merits](https://www.lesswrong.com/posts/5yFRd3cjLpm3Nd6Di/argument-screens-off-authority). But I wasn't going to be so impertinent to point that out when the master was testing me (!) and I was eager to pass the test.
-I said that I'd like to take an hour to compose a _good_ answer. If I tried to type something off-the-cuff on the timescale of five minutes, it wasn't going to be of similar quality as my criticisms, because, as I had just admitted, I had _totally_ been running a biased search for criticisms—or did the fact that I had to ask that mean I had already failed the test?
+I said that I'd like to take an hour to compose a _good_ answer. (It was 10:26 _p.m._) If I tried to type something off-the-cuff on the timescale of five minutes, it wasn't going to be of similar quality as my criticisms, because, as I had just admitted, I had _totally_ been running a biased search for criticisms—or did the fact that I had to ask that mean I had already failed the test?
Yudkowsky replied:
> I mean, yeah, in fact the greater test is already having that info queued, but conversely it's even worse if you think back or reread and people are not impressed with the examples you find. I cannot for politeness lie and deny that if you did it in five minutes it would be _more_ impressive, but I think that it is yet the correct procedure to take your time.
-(As an aside—this isn't something I thought or said at the time—I _do_ think it makes sense to run an asymmetric search for flaws in _some_ contexts, even though it would be disastrous to only look on one side of the argument when considering a belief you're uncertain about. Code reviewers often only comment in detail on flaws or bugs that they find, and say only "LGTM" (looks good to me) when they don't find any. Why? Because the reviewers aren't particulaly trying to evaluate "This code is good" as an abstract belief[^low-stakes]; they're trying to improve the code, and there's an asymmetry in payoffs where eliminating a flaw is an improvement, whereas identifying something the code does right just means the author was doing their job. If you didn't trust a reviewer's competence and thought they were making spurious negative reviews, you might legitimately test them by asking them to argue what's _good_ about a pull request that they just negatively reviewed, but I don't think it should be concerning if they ask for some extra time.)
+(As an aside—this isn't something I thought or said at the time—I _do_ think it makes sense to run an asymmetric search for flaws in some contexts, even though it would be disastrous to only look on one side of the argument when considering a belief you're uncertain about. Code reviewers often only comment in detail on flaws or bugs that they find, and say only "LGTM" (looks good to me) when they don't find any. Why? Because the reviewers aren't necessarily trying to evaluate "This code is good" as an abstract belief[^low-stakes]; they're trying to improve the code, and there's an asymmetry in payoffs where eliminating a flaw is an improvement, whereas identifying something the code does right just means the author was doing their job. If you didn't trust a reviewer's competence and thought they were making spurious negative reviews, you might legitimately test them by asking them to argue what's _good_ about a pull request that they just negatively reviewed, but I don't think it should be concerning if they asked for some extra time.)
[^low-stakes]: For typical low-stakes business software in the "move fast and break things" regime. In applications where bugs are more costly, you do want to affirmatively verify "the code is good" as a belief.
-I said that I also wanted to propose a re-framing: the thing that this thread was complaining about was a lack of valorization of truth-_telling_, honesty, wanting _other_ people to have accurate maps. Or maybe that was covered by "as you, yourself, see that virtue"?
+I said that I also wanted to propose a reframing: the thing that the present thread was complaining about was a lack of valorization of truth-_telling_, honesty, wanting _other_ people to have accurate maps. Or maybe that was covered by "as you, yourself, see that virtue"?
Yudkowsky said that he would accept that characterization of what the thread was about if my only objection was that dath ilan didn't tell Keltham about BSDM, and that I had no objection to Keltham's judgement that in dath ilan, he would have preferred not to know.
-I expounded for some more paragraphs about why I _did_ object to Keltham's judgement, and then started on my essay exam—running with my "truth-_telling_" reframing.
+I expounded for some more paragraphs about why I _did_ object to Keltham's judgement, and then started on my essay exam—running with my "truth-telling" reframing.
-[TODO: outline the test
- * I re-read pg. 74+ of "What the Truth Can Destroy" and submit answers; (at 12:30 _a.m._, two hours and
- * Thellim!!!
-]
+I wanted to nominate the part where the Conspiracy is unveiled—I thought I remembered Keltham saying something about how Carissa's deception was the worst thing anyone could have done to him—that is, the fact that someone he trusted was putting him in a fake reality was _itself_ considered a harm, separately from the fact that Cheliax is evil. I re-read pages 74 onwards of the ["What the Truth Can Destroy"](https://www.glowfic.com/posts/5930) thread, and didn't see Keltham saying the thing I thought he said (maybe it happened in the next thread, or I had misremembered), but found two more things to submit as answers to my lit exam, which I posted at 12:30 _a.m._ (so I had actually taken two hours rather than the one I had asked for).
+
+First, I liked how [Snack Service intervenes to stage](https://www.glowfic.com/replies/1811461#reply-1811461) a "truth and reconciliation commission" for Keltham and his paramours, on the grounds that it's necessary for Asmodeus and Cayden Caliean and Adabar and Keltham to make their best decisions. People testifying in public (with the Chelaxians and Oririons present, as one would at a trial) reflects a moral about the importance of common knowledge, _shared_ maps. The testimony being public ensured that not just that Keltham got to know what's been done to him, but that his paramours and counterparties _know that he knows_. There was something honorable about getting things on the public record like that, in the end, even while Snack Service was willing to participate in the conspiracy _before_ the jig was up.
+
+Second, I liked Korva's speech about why she hates Keltham, and how Keltham not only takes it in stride, but also asks to buy the right to take Korva with him to Osirion. When Abrogail expresses surprise that Keltham would want Korva, Keltham cites a dath ilani proverb about advice that's easier to get from people who aren't friends with you. This reflects an understanding that your friends wanting to be nice to you can be a source of distortions; Keltham specifically values Korva _as a critic_.
+
+The next day, I added that I realized that I had missed a huge opportunity to successfully reply on a five-minute time scale (to pass "the greater test [of] already having that info queued"): the "in _Planecrash_" part of the prompt made me think I had to find something in Keltham's story (which is why I took another two hours to hand in my essay), but other threads within the dath ilan Glowfic continuity should obviously count for the purpose of the test, and I did in fact already have cached thoughts about how Thellim's contempt for Jane Austen characters beautifully mirrored my contempt for protecting people from psychology facts that would hurt their feelings. I could _prove_ that I already had it cached (if not queued, as evidenced by my remembering it the next day), because I had mentioned it both in the conversation leading to the present thread, and in my memoir draft.
+
+Yudkowsky replied:
+
+> so I think that you're looking an awful lot at what _characters say_ and nearly not at all at what the universe does. this plausibly reflects a deep flaw in your art, because it sure does seem to me that you are a lot better at noticing what people say about truth in words, detecting whose monkey-side they seem to be on, than you are imo at carefully weighing up both sides of things as is the art of finding-truth-in-reality. it plausibly also reflects some people who ill-shaped you, pointing you at the fictional characters and angering you at their spoken words and verbal thoughts, as was advantageous to them, and not pointing you towards, like, looking at the messages in the fiction itself rather than the words spoken by characters, because that would not have served their ill purpose of alienating you and turning you into an angry thing more useful for their purposes. (I would not ordinarily use language like this but I regret that it is the language you have now seemingly been ill-shaped to speak, for another's usefulness.)
+> if I ask you, not what any _character says_, not even what any _societies say_, but _what happens in Planecrash_ and what the _causal process_ there seems to think about matters important to you, what do you see?
+
+As a _quick_ reply to the followup question (posted within 19 minutes of it being asked), I said that Cheliax was at a structural disadvantage in its conflict with the forces of Good, because learning how to think inevitably turns mortals away from Asmodeus's will.
+
+But I was _more_ interested in replying to the part about me being ill-shaped to another's purpose. (I said that I wouldn't have considered that on-topic for the fiction server, but if _he_ thought it was on-topic, then it made sense for me to reply—and I did so at 12:26 _p.m._ the next day, after some time to think. Discord lends itself quite well to a mix of synchronous and asynchronous communication, depending on when people happen to be at their computers.)
+
+I said that he seemed _really_ stuck on this hypothesis that it was Michael Vassar's fault that I'd been shaped into an alienated and angry thing.
+
+To be clear, I totally agreed that I had been shaped into an alienated and an alienated and angry thing. Obviously. But speaking of people "look[ing] inside themselves and correctly see[ing] that this is not how they work" (as Yudkowsky had said earlier), I thought he was getting the causality all wrong.
+
+It seemed to _me_ that the reason I had become an alienated and angry thing is because I had been shaped by [making an extraordinary effort](https://www.lesswrong.com/posts/GuEsfTpSDSbXFiseH/make-an-extraordinary-effort) since 2016 to respond to a class of things that included Yudkowsky "mak[ing] up sophisticated stories for why pretty obviously true things are false"—again referencing Oliver Habryka's comment on "Challenges to Yudkowsky's Pronoun Reform Proposal."
+
+That's the context in which it wasn't surprising that my Art had involved some amount of specialization in "detecting whose monkey-side they seem to be on." In a world where monkeys are trying to cover up otherwise-obvious truths, successfully blowing the whistle on them involves being sensitive to their monkey games; figuring out the truth they're trying to cover up is the easy part. The whistleblowing-skill of of promoting otherwise-obvious things to _common_ knowledge in opposition to a Power trying to prevent common knowledge, is different from the science-skill of figuring out organically-nonobvious things from scratch. It _makes sense_ for Alexander Solzhenitsyn and Andrey Kolmogorov—or for that matter, John Galt and Robert Stadler—to have developed different crystalized skills.
+
+(Indeed, it even makes sense for Kolmogorov and Stadler to _not_ develop some skills, because the skills would show up under Detect Thoughts.)
+
+If it was all Michael's fault for "extensively meta-gas[lighting me] into believing that everyone generally and [him] personally [were] engaging in some kind of weird out-in-the-open gaslighting", I asked, echoing Yudkowsky's language from earlier (with appropriate quotation marks and brackets),[^gaslighting] then _how come Oli could see it, too?_
+
+[^gaslighting]: In particular, Yudkowsky was the one who introduced the word _gaslighting_ into the conversation; I hadn't previously used the word myself.
+
+Yudkowsky replied:
+
+> I think if you asked Oli about the state of reality with respect to this whole affair, he'd have a very different take from your take, _if you're still able to hear differences instead of only those similarities you demand._
+
+That sounded like an easy enough experimental test! I wrote Habryka an email explaining the context, and asking him what "very different take" he might have, if any. (I resisted the temptation to start a [Manifold market](https://manifold.markets/) first.) As I mentioned in the email, I didn't expect to have a very different take from him _about the state of reality_. ("Zack is (still?!) very upset about this, but Oli mostly doesn't care" is a values-difference, not a disagreement about the state of reality.) I didn't think I disagreed with _Yudkowsky_ much about the state of reality! (In his own account, he thought it was "sometimes personally prudent [...] to post your agreement with Stalin about things you actually agree with Stalin about", and I believed him; I was just unhappy about some of the side-effects of his _prudence_.)
+
+Oliver didn't reply. (I might have guessed the wrong email address, out of the two I had on file for him?) I don't blame him; it might have been timelessly ungrateful of me to ask. (The reason people are reluctant to make on-the-record statements in politically charged contexts is because they're afraid the statements will be _used_ to drag them into more political fights later. He had already done me a huge favor by being brave enough to state the obvious in March; I had no right to demand anything more of him.)
+
+Regarding my quick reply about Cheliax's structural disadvantage, Yudkowsky said it was "okay as one element", but complained that the characters had already observed it out loud, and that I "didn't name any concrete sequence of events that bore it out or falsified it." He continued:
+
+> I think you could find a lot more than this if your brain were still able to see incongruent facts as well as only congruent facts. what does the underlying reality of Planecrash think about your Most Important Issues? what is shown, but maybe never even told at all? you gave the reply of somebody who can _only_ see social realities and _only_ what people say and not what _just happens, at all_, even inside a story, you didn't say _a thing that happened._
+
+At this point, I was a bit suspicious that _any_ answer that wasn't exactly whatever he was thinking of would be dismissed as too social or too inferentially close to something one of the characters had said. What did it mean for the _universe_ to say something about valorizing truth?
+
+The original prompt ("What are some of the ways _Planecrash_ valorizes truth") had put me into 11th-grade English class mode; the revision "if I ask you, not what any _character_ says [...]" made me think the 11th-grade English teacher expected a different answer. Now the revised–revised prompt "what does the underlying reality of _Planecrash_ think about your Most Important Issues?", with the previous rebukes in my context window, was making me think I should be reaching for an act of philosophical [Original Seeing](https://www.lesswrong.com/posts/SA79JMXKWke32A3hG/original-seeing), rather than trying to be a diligent schoolstudent playing the 11th-grade English class game. I thought about it ... and I _saw something_.
+
+_Thesis_: the universe of _Planecrash_ is saying that virtue ethics—including, as a special case, my virtue ethics about it being good to tell the truth and reveal information—are somewhat unnatural.
+
+In the story, the god Adabar values trading fairly, even with those who can't verify that their partners are keeping up their end of the deal,[^trade-verification] and also wants to promote fair trading _elsewhere_ in Reality (as contrasted to just being fair Himself).
+
+[^trade-verification]: Significantly, this is somewhat "unnatural" behavior according to Yudkowsky's view of decision theory. Ideal agents are expected to cooperate with agents whose cooperation is _conditional_ on their own cooperation, not simply those that cooperate with them: you "should" defect against a rock with the word "COOPERATE" painted on it, and you "shouldn't" trade for what you could just as easily steal. See §6 of ["Robust Cooperation in the Prisoner's Dilemma: Program Equilibrium via Provability Logic"](https://arxiv.org/abs/1401.5577).
-[TODO: derail with Lintamande]
+Adabar is kind of a weirdo. He's not a vanishly rare freak (whose specification would require lots of uncompressible information); there _is_ a basin of attraction in the space of pre-gods, where creatures who develop a computationally efficient "fairness" heuristic in their ancestral environment and reify that into their utilityfunction when they ascend to divinity, but it's not a _huge_ basin of attraction; most gods aren't like Adabar.
+
+It's the same thing with honesty. Generic consequentialists have no reason to "tell the truth" to agents with different utility functions when they're not under compact and being compensated for the service. Why _would_ you emit signals that other agents can interpret as a map that reflects the territory? [You can't get more paperclips that way!](https://arbital.com/p/not_more_paperclips/)
+
+I had previously written about this in ["Commucation Requires Common Interests or Differential Signal Costs"](https://www.lesswrong.com/posts/ybG3WWLdxeTTL3Gpd/communication-requires-common-interests-or-differential); you needed some common interests in order for flexible, "digital" language to exist at all. ("Digital" language being that for which the relationship between signals and meaning can be arbitrary, in contrast to costly signaling, where me expending resources at least tell you that I could afford those resources.)
+
+It's _possible_ for imperfectly deceptive social organisms to develop a taste for "honesty" as a computationally efficient heuristic for navigating to Pareto improvements in the ancestral environment, which _might_ get reified into the utilityfunction as they ascend—but that's an Adabar-class weird outcome, not the default outcome.
+
+So—insofar as my Most Important Issues revolved around an obsession with motivational transparency, wanting to live in a world that wasn't lying to me, wanting to _reveal information_ as an end in itself, unilaterally rather than only as part of a coordinated negotiation scheme, without necessarily being _paid_ for it, but just because it is _right_ ...
+
+It seems like my answer to the question of, "What does the underlying causal process of _Planecrash_ think about your Most Important Issues; what are some the ways that _Planecrash_ valorizes truth-telling as you, yourself, see that virtue?" is, "It doesn't" (!). Truth-telling is a virtue ethic, and _Planecrash_ depicts a universe ruled by consequentialist gods who only implement virtue ethics insofar as that made it into their utilityfunction.
+
+I realized, of course, that this certainly wasn't the answer Yudkowsky was looking for. But it seemed like a _better_ answer than me trying to play the schoolstudent. He asked what I saw when I looked at what the fictional universe was saying about my problems, and I looked, and _I saw something_. (Something philosophically substantive, definitely not just a social reality.) It seemed more honest to just report that, rather than keep trying to [guess the teacher's password](https://www.lesswrong.com/posts/NMoLJuDJEms7Ku9XS/guessing-the-teacher-s-password).
+
+("Hermione knew the correct answer wouldn't impress Professor Quirrell, but it was the correct answer, so she said it.")
+
+So, after sleeping on it first, I posted the explanation of what I saw to the channel (including the parts about how the original prompts steered me, and that I realized that this wasn't the answer he was looking for).
+
+The outcome was—silence. No response from Yudkowsky in several days. Maybe I shouldn't have ran with my Original Seeing answer? I showed the transcripts to a friend, who compared my answer about consequentialist gods to including a list of your country's war crimes in a high school essay assignment about patriotism; I had done a terrible job of emitting symbols that made me a good monkey, and a mediocre-at-best job of flipping the table (rejecting Yudkowsky's "pass my test before I recognize your criticism as legitimate" game) and picking a fight instead.
+
+("Don't look at me," he added, "I would've flipped the table at the beginning.")
+
+I tried to explain that my third answer wasn't _just_ doubling down on my previous thesis: "my virtue ethics run against the grain of the hidden Bayesian structure of reality" wasn't an argument _in favor of_ my virtue ethics. My friend wasn't buying it; I still hadn't been fulfilling the original prompt.
+
+He had me there. I had no more excuses after that: I had apparently failed the test. I was feeling pretty glum about this, and lamented my poor performance in the `#drama` channel of another Discord server (that Yudkowsky was also a member of). I had thought I was doing okay—I definitely _didn't_ say, "That's impossible because Big Yud and Linta are lying liars who hate Truth", and there were reasons why my Original Seeing answer made sense _to me_ as a thing to say, but _that wasn't what I was being tested on_. It _genuinely_ looked bad in context. I had failed in [my ambition to know how it looks](/2022/context-is-for-queens/#knowing-how-that-looks).
+
+I think Yudkowsky saw the #drama messages (he left an emoji-reaction in the relevant timespan of messages) and took pity on me.
+
+[TODO: summarize teacher feedback]
+
+[TODO: summarize my admitting that it did have something to do with my state of mind; I would have done better by giving the 11th grade English class algorithm more compute; Peranza's username was 'not-looking-there'!; proposed revision]
+
+[TODO: C.S. Lewis speech]
+
+I said that I thought it was significant that the particular problem to which my Art had been shaped (in some ways) and misshaped (in others) wasn't just a matter of people being imperfect. Someone at the 2021 Event Horizon Independence Day party had told me that people couldn't respond to my arguments because of the obvious political incentives. And I guessed—the angry question I wanted to ask, since I didn't immediately know how to rephrase it to not be doing the angry monkey thing was, was I supposed to _take that lying down?_
+
+**Eliezer** — 12/17/2022 5:50 PM
+you sure are supposed to not get angry at the people who didn't create those political punishments
+that's insane
+they're living in Cheliax and you want them to behave like they're not in Cheliax and get arrested by the Church
+your issue is with Asmodeus. take it to Him, and if you can't take Him down then don't blame others who can't do that either.
+
+Admirably explicit.
+
+[TODO: Yudkowsky's story: the story is about Keltham trusting Cheliax wrongly; leaving that part out is politicized; other commenters pick up on "But you're still saying to trust awesome institutions"]
+
+[TODO: I think there's a bit of question-substitution going on; the reason the virtue of evenness is important is because if you only count arguments for and not against the hypothesis, you mess up your beliefs about the hypothesis; if you substitute a different question "Is Yudkowsky bad?"/"Am I a good coder?", that's a bucket error—or was he "correctly" sensing that the real question was "Is Yudkowsky bad?"]
+
+[TODO: I express my fully-updated grievance (this doesn't seem to be in the transcript I saved??); I hadn't consciously steered the conversation this way, but the conversation _bounced_ in a way that made it on-topic; that's technically not my fault, even if the elephant in my brain was optimizing for this outcome.
+
+The fact that Yudkowsky had been replying to me at length—explaining why my literary criticism was nuts, but in a way that respected my humanity and expected me to be able to hear it—implied that I was apparently in his "I can cheaply save him (from crazy people like Michael)" bucket, rather than the "AI timelines and therefore life is too short" bucket.]
+
+[TODO: I think it's weird that Yudkowsky's reaction is "that's insane"; he should be able to understand why someone might consider this a betrayal, even if he didn't think he was bound to that level of service; the story of a grant-making scientist]
+
+[TODO: I bait Lintamande into engagement]
+
+[TODO: Linta says I'm impossible to talk to and the anticipation of my pouncing stiffles discussion. (I almost wonder if this is a good thing, from a _realpolitik_ perspective? I'd prefer to argue people out of bad ideas, but if the threat of an argument disincentivizes them from spreading ...? Game theory goes both ways—I've been self-censoring to.)]
+
+[TODO: I agreed that this was good feedback about my social behavior; I don't intellectually disagree that different cultures are different; I'm super-fighty because I'm super-traumatized; the thing I'm trying to keep on Society's shared map is, Biological Sex Actually Exists and Is Sometimes Decision-Relevant; Biological Sex Actually Exists and is Sometimes Decision-Relevant Even When It Makes People Sad; Biological Sex Actually Exists Even When a Prediction Market Says It Will Make People Sad; Linta agrees; Eliezer responds with a +1 emoji]
+
+[TODO: "like, if you just went and found Eliezer!2004 and were like 'hey, weird sci fi hypothetical'
+_speaking of the year 2004_; the thing I'm at war with is that I don't think he would _dare_ publish the same essay today
+]
-[TODO: knives, and showing myself out]
+[TODO: someone said "the word in their language doesn't match the word in yours"; and got a +1 emoji; I resisted the temptation to say "So ... I can define a word any way I want"; I made a few more comments about kitchen knife deception (and let my friends talk me down from making more). I'm not worried about what he thinks about me anymore.]
------
projects / Ultimately_Untrue_Thought.git / blobdiff