But if you think the only hope for there _being_ a future flows through maintaining influence over what large tech companies are doing as they build transformative AI, declining to contradict the state religion makes more sense—if you don't have _time_ to win a culture war, because you need to grab hold of the Singularity (or perform a [pivotal act](https://arbital.com/p/pivotal/) to prevent it) _now_. If the progressive machine marks you as a transphobic bigot, the machine's functionaries at OpenAI or Meta AI Research are less likely to listen to you when you explain why [their safety plan](https://openai.com/blog/our-approach-to-alignment-research/) won't work, or why they should have a safety plan at all.
-(I remarked to "Wilhelm" in June 2022 that DeepMind [changing its Twitter avatar to a rainbow variant of their logo for Pride month](https://web.archive.org/web/20220607123748/https://twitter.com/DeepMind) was a bad sign.)
+(I remarked to "Wilhelm" in mid-2022 that DeepMind [changing its Twitter avatar to a rainbow variant of their logo for Pride month](https://web.archive.org/web/20220607123748/https://twitter.com/DeepMind) was a bad sign.)
So isn't there a story here where I'm the villain, willfully damaging humanity's chances of survival by picking unimportant culture-war fights in the xrisk-reduction social sphere, when _I know_ that the sphere needs to keep its nose clean in the eyes of the progressive egregore? _That's_ why Yudkowsky said the arguably-technically-misleading things he said about my Something to Protect: he _had_ to, to keep our collective nose clean. The people paying attention to contemporary politics don't know what I know, and can't usefully be told. Isn't it better for humanity if my meager talents are allocated to making AI go well? Don't I have a responsibility to fall in line and take one for the team—if the world is at stake?
That is, there's _no story_ under which misleading people about trans issues is on Yudkowsky's critical path for shaping the intelligence explosion. _I'd_ prefer him to have free speech, but if _he_ thinks he can't afford to be honest about things he [_already_ got right in 2009](https://www.lesswrong.com/posts/QZs4vkC7cbyjL9XA9/changing-emotions), he could just—not issue pronouncements on topics where he intends to _ignore counterarguments on political grounds!_
-In [a Twitter discussion about why not to trust organizations that refuse to explain their reasoning, Yudkowsky wrote](https://twitter.com/esyudkowsky/status/1374161729073020937):
+In [a March 2021 Twitter discussion about why not to trust organizations that refuse to explain their reasoning, Yudkowsky wrote](https://twitter.com/esyudkowsky/status/1374161729073020937):
> Having some things you say "no comment" to, is not at _all_ the same phenomenon as being an organization that issues Pronouncements. There are a _lot_ of good reasons to have "no comments" about things. Anybody who tells you otherwise has no life experience, or is lying.
I can think of two reasons why the naïve objection might fail. (And who can say but that a neutral expert witness on decision theory wouldn't think of more?)
-First, the true decision theory is subtler than "defy anything that you can commonsensically pattern-match as looking like 'extortion'"; the case for resisting extortion specifically rests on there existing a subjunctive dependence between your decision and the extortionist's decision (they threaten _because_ you'll give in, or don't bother _because_ you won't), and the relevant subjunctive dependence doesn't obviously pertain in the real-life science intellectual _vs._ social justice mob match-up. If the mob has been trained from past experience to predict that their targets will give in, should you defy them now in order to somehow make your current situation "less real"? Depending on the correct theory of logical counterfactuals, the correct stance might be ["We don't negotiate with terrorists, but we do appease bears"](/2019/Dec/political-science-epigrams/) (because the bear's response isn't calculated based on our response), and the progressive egregore might be relevantly bear-like.
+First, the true decision theory is subtler than "defy anything that you can commonsensically pattern-match as looking like 'extortion'"; the case for resisting extortion specifically rests on there existing a subjunctive dependence between your decision and the extortionist's decision (they threaten _because_ you'll give in, or don't bother _because_ you won't), and the relevant subjunctive dependence doesn't obviously pertain in the real-life science intellectual _vs._ social justice mob match-up. If the mob has been trained from past experience to predict that their targets will give in, should you defy them now in order to somehow make your current situation "less real"?[^emerson] Depending on the correct theory of logical counterfactuals, the right stance might be ["We don't negotiate with terrorists, but we do appease bears"](/2019/Dec/political-science-epigrams/) (because the bear's response isn't calculated based on our response), and the forces of political orthodoxy might be relevantly bear-like.
-On the other hand, the relevant subjunctive dependence doesn't obviously _not_ pertain, either!
+[^emerson]: I remember back in 'aught-nine, Tyler Emerson was caught embezzling funds from the Singularity Institute, and SingInst made a point of prosecuting him on decision-theoretic grounds, when a lot of other nonprofits would have quietly covered it up to spare themselves the embarrassment.
+
+On the other hand, the relevant subjunctive dependence doesn't obviously _not_ pertain, either! Parsing social justice as an agentic "threat" rather than a non-agentic obstacle like an avalanche, does seem to line up with the fact that people punish heretics (who dissent from an ideological group) more than infidels (who were never part of the group to begin with), _because_ heretics are more extortable—more vulnerable to social punishment from the original group.
+
+Which brings me to the second reason the naïve anti-extortion argument might fail: [what counts as "extortion" depends on the relevant "property rights", what the "default" action is](https://www.lesswrong.com/posts/Qjaaux3XnLBwomuNK/countess-and-baron-attempt-to-define-blackmail-fail). If having free speech is the default, being excluded from the coalition for defying the orthodoxy could be construed as extortion. But if _being excluded from the coalition_ is the default, maybe toeing the line of orthodoxy is the price you need to pay in order to be included.
[TODO: defying threats, cont'd—
- * Yudkowsky does seemingly back commonsensical interpretations, re voting, or how, back in 'aught-nine, SingInst had made a point of prosecuting Tyler Emerson, citing decision theory
+ * Yudkowsky has an algorithm for bargaining between agents with different notions of "fairness": you'd prefer a fair split on the Pareto boundary, but you should be willing to except an unfair split, as long as the other guy also does worse—all the way to the Nash equilibrium https://www.lesswrong.com/posts/z2YwmzuT7nWx62Kfh/cooperating-with-agents-with-different-ideas-of-fairness
+ * How does this map on to the present situation, though? Does he think he's playing Nash, or does he think he's getting gains-from-trade? (Either figure this out, or write some smart sentences about my confusion)
- * But the parsing of social justice as an agentic "threat" to be avoided rather than a rock to be dodged does seem to line up with the fact that people punish heretics more than infidels.
+ * I asked him why he changed his mind about voting
+ * "Vote when you're part of a decision-theoretic logical cohort large enough to change things, or when you're worried about your reputation and want to be honest about whether you voted."
+ * So maybe he doesn't think he's part of a decision-theoretic logical cohort large enough to resist the egregore, and he's also not worried about his reputation for resisting the egregore
+ * If his reptuation in the eyes of people like me just isn't that valuable, I guess I can't argue with that
- * But it matters where you draw the zero point: is being excluded from the coalition a "punishment" to threaten you out of bad behavior, or is being included a "reward" for good behavior?
+Curtis Yarvin [likes to compare](/2020/Aug/yarvin-on-less-wrong/) Yudkowsky to [Sabbatai Zevi](https://en.wikipedia.org/wiki/Sabbatai_Zevi#Conversion_to_Islam), the Jewish religious leader who was purported to be the Messiah, who converted to Islam under coercion from the Ottomans. "I know, without a shadow of a doubt, that in the same position, Eliezer Yudkowsky would also convert to Islam," said Yarvin.
- * Curtis Yarvin has compared Yudkowsky to Sabbatai Zevi (/2020/Aug/yarvin-on-less-wrong/), and I've got to say the comparison is dead-on. Sabbatai Zevi was facing much harsher coercion: his choices were to convert to Islam or be impaled https://en.wikipedia.org/wiki/Sabbatai_Zevi#Conversion_to_Islam
+ * But this isn't necessarily crazy. Zevi was facing some very harsh coercion: convert or be impaled.
+ * My real question is, in the same position, would Sabbatai Zevi declare that 30% of the ones with penises are actually women?
]
* But he does, in fact, seem to actively encourage this conflation (contrast to how the Sequences had a [Litany Against Gurus](https://www.lesswrong.com/posts/t6Fe2PsEwb3HhcBEr/the-litany-against-gurus) these days, with the way he sneers as Earthlings and post-rats)
- * "I may as well do it on Earth"
-
* a specific example that made me very angry in September 2021—
https://twitter.com/ESYudkowsky/status/1434906470248636419
------
-In October 2021, Jessica [published a post about her experiences at MIRI](https://www.lesswrong.com/posts/MnFqyPLqbiKL8nSR7/my-experience-at-and-around-miri-and-cfar-inspired-by-zoe), making analogies between sketchy social pressures she had experienced in the core rationalist community (around short AI timelines, secrecy, deference to community leaders, _&c._) and those reported in [Zoe Cramer's recent account of her time at Leverage Research](https://medium.com/@zoecurzi/my-experience-with-leverage-research-17e96a8e540b).
+In October 2021, Jessica Taylor [published a post about her experiences at MIRI](https://www.lesswrong.com/posts/MnFqyPLqbiKL8nSR7/my-experience-at-and-around-miri-and-cfar-inspired-by-zoe), making analogies between sketchy social pressures she had experienced in the core rationalist community (around short AI timelines, secrecy, deference to community leaders, _&c._) and those reported in [Zoe Cramer's recent account of her time at Leverage Research](https://medium.com/@zoecurzi/my-experience-with-leverage-research-17e96a8e540b).
Scott Alexander posted a comment claiming to add important context, essentially blaming Jessica's problems on her association with Michael Vassar, to the point of describing her psychotic episode as a "Vassar-related phenomenon" (!). Alexander accused Vassar of trying "jailbreak" people from normal social reality, which "involve[d] making them paranoid about MIRI/CFAR and convincing them to take lots of drugs". Yudkowsky posted [a comment that uncritically validated Scott's reliability as a narrator](https://www.lesswrong.com/posts/MnFqyPLqbiKL8nSR7/my-experience-at-and-around-miri-and-cfar-inspired-by-zoe?commentId=x5ajGhggHky9Moyr8).
-To me, this looked like raw factional conflict: Jessica had some negative-valence things to say about the Caliphate, so Caliphate leaders moved in to discredit her by association. (Quite effectively, as it turned out: the karma score on Jessica's post dropped by more than half, while Alexander's comment got voted up to more than 380 karma.)
+To me, this looked like raw factional conflict: Jessica had some negative-valence things to say about the Caliphate, so Caliphate leaders moved in to discredit her by association. Quite effectively, as it turned out: the karma score on Jessica's post dropped by more than half, while Alexander's comment got voted up to more than 380 karma. (The fact that Scott said ["it's fair for the community to try to defend itself"](https://www.lesswrong.com/posts/MnFqyPLqbiKL8nSR7/my-experience-at-and-around-miri-and-cfar-inspired-by-zoe?commentId=qsEMmdo6DKscvBvDr) in ensuing back-and-forth suggests that he also saw the conversation as an adversarial one, even if he thought Jessica shot first.)
I explained [why I thought Scott was being unfair](https://www.lesswrong.com/posts/MnFqyPLqbiKL8nSR7/my-experience-at-and-around-miri-and-cfar-inspired-by-zoe?commentId=GzqsWxEp8uLcZinTy) (and [offered textual evidence](https://www.lesswrong.com/posts/MnFqyPLqbiKL8nSR7/my-experience-at-and-around-miri-and-cfar-inspired-by-zoe?commentId=yKo2uuCcwJxbwwyBw) against the silly claim that Michael was _trying_ to drive Jessica crazy).
[^vassarite-scare-quotes]: Scare quotes because "Vassarite" seems to be Alexander's coinage; we didn't call ourselves that.
-But I will _absolutely_ bite the bullet on it being correct to freak out about learning your entire Society is corrupt and gaslighting (as I explained to Scott on Discord a few days later).
+But I will _absolutely_ bite the bullet on it being correct to freak out about learning your entire Society is corrupt and gaslighting (as I explained to Scott in an asynchronous 22–27 October conversation on Discord).
-Imagine living in the Society of Alexander's ["Kolmogorov Complicity and the Parable of Lightning"](https://slatestarcodex.com/2017/10/23/kolmogorov-complicity-and-the-parable-of-lightning/) (which I keep linking) in the brief period when the lightening taboo is being established, trying to make sense of everyone you know, suddenly deciding, seemingly in lockstep, that thunder comes before lightning. (When you try to point out that this isn't true and no one believed it five years ago, they point out that it depends on what you mean by the word 'before'.)
+Imagine living in the Society of Alexander's ["Kolmogorov Complicity and the Parable of Lightning"](https://slatestarcodex.com/2017/10/23/kolmogorov-complicity-and-the-parable-of-lightning/) (which I keep linking) in the brief period when the lightening taboo is being established, trying to make sense of everyone you know suddenly deciding, seemingly in lockstep, that thunder comes before lightning. (When you try to point out that this isn't true and no one believed it five years ago, they point out that it depends on what you mean by the word 'before'.)
Eventually, you would get used to it, but at first, I think this would be legitimately pretty upsetting! If you were already an emotionally fragile person, it might even escalate to a psychiatric emergency through the specific mechanism "everyone I trust is inexplicably lying about lightning → stress → sleep deprivation → temporary psychosis". (That is, it's not that Society being corrupt directly causes mental ilness—that would be silly—but confronting a corrupt Society is very stressful, and that can [snowball into](https://lorienpsych.com/2020/11/11/ontology-of-psychiatric-conditions-dynamic-systems/) things like lost sleep, and sleep is [really](https://www.jneurosci.org/content/34/27/9134.short) [biologically important](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6048360/).)
I had my answer. But I wasn't sure that Scott would understand.
-To assess whether joining the "Vassarites" was harmful to me, one would need to answer: as compared to what? In the counterfactual where Michael vanished from the world in 2016, I think I would have been just as upset about the same things for the same reasons, but with fewer allies and fewer ideas to make sense of what was going on in my social environment.
+To assess whether joining the "Vassarites" had been harmful to me, one would need to answer: as compared to what? In the counterfactual where Michael vanished from the world in 2016, I think I would have been just as upset about the same things for the same reasons, but with fewer allies and fewer ideas to make sense of what was going on in my social environment.
-Additionally, it's really obnoxious when people have tried to use my association with Michael to try to discredit the content of what I was saying—interpreting me as Michael's pawn. Gwen, one of the "Zizians", in a blog post about her grievances against CfAR, has [a section on "Attempting to erase the agency of everyone who agrees with our position"](https://everythingtosaveit.how/case-study-cfar/#attempting-to-erase-the-agency-of-everyone-who-agrees-with-our-position), complaining about how people try to cast her and Somni and Emma as Ziz's minions, rather than acknowledging that they're separate people with their own ideas who had good reasons to work together. I empathized a lot with this. My thing, and separately Ben Hoffman's [thing about Effective Altruism](http://benjaminrosshoffman.com/drowning-children-rare/), and separately Jessica's thing in the OP, don't really have a whole lot to do with each other, except as symptoms of "the so-called 'rationalist' community is not doing what it says on the tin" (which itself isn't a very specific diagnosis). But insofar as our separate problems did have a hypothesized common root cause, it made sense for us to talk to each other and to Michael about them.
+Additionally, it was really obnoxious when people had tried to use my association with Michael to try to discredit the content of what I was saying—interpreting me as Michael's pawn. Gwen, one of the "Zizians", in a blog post about her grievances against CfAR, has [a section on "Attempting to erase the agency of everyone who agrees with our position"](https://everythingtosaveit.how/case-study-cfar/#attempting-to-erase-the-agency-of-everyone-who-agrees-with-our-position), complaining about how people try to cast her and Somni and Emma as Ziz's minions, rather than acknowledging that they're separate people with their own ideas who had good reasons to work together. I empathized a lot with this. My thing, and separately Ben Hoffman's [thing about Effective Altruism](http://benjaminrosshoffman.com/drowning-children-rare/), and separately Jessica's thing in the OP, didn't really have a whole lot to do with each other, except as symptoms of "the so-called 'rationalist' community is not doing what it says on the tin" (which itself isn't a very specific diagnosis). But insofar as our separate problems did have a hypothesized common root cause, it made sense for us to talk to each other and to Michael about them.
Was Michael using me, at various times? I mean, probably. But just as much, _I was using him_. Particularly with the November 2018–April 2019 thing (where I and the "Vassarite" posse kept repeatedly pestering Scott and Eliezer to clarify that categories aren't arbitrary): that was the "Vassarites" doing an _enormous_ favor for _me_ and _my_ agenda. (If Michael and crew hadn't had my back, I wouldn't have been anti-social enough to keep escalating.) And here Scott was trying to get away with claiming that _they_ were making my situation worse? That's _absurd_. Had he no shame?
-I _did_, I admitted, have some specific, nuanced concerns—especially since the December 2020 psychiatric disaster, with some nagging doubts beforehand—about ways in which being an inner-circle "Vassarite" might be bad for someone, but at the moment, I was focused on rebutting Scott's story, which was silly. A defense lawyer has an easier job than a rationalist—if the prosecution makes a terrible case, you can just destroy it, without it being your job to worry about whether your client is separately guilty of vaguely similar crimes that the incompetent prosecution can't prove.
-
-[TODO—
+I _did_, I admitted, have some specific, nuanced concerns—especially since the December 2020 psychiatric disaster, with some nagging doubts beforehand—about ways in which being an inner-circle "Vassarite" might be bad for someone, but at the moment, I was focused on rebutting Scott's story, which was _silly_. A defense lawyer has an easier job than a rationalist—if the prosecution makes a terrible case, you can just destroy it, without it being your job to worry about whether your client is separately guilty of vaguely similar crimes that the incompetent prosecution can't prove.
-"what you think is support" is actually a different thing
+When Scott expressed concern about the group-yelling behavior that [Ziz had described in a blog comment](https://sinceriously.fyi/punching-evil/#comment-2345) and [Yudkowsky had described on Twitter](https://twitter.com/ESYudkowsky/status/1356494768960798720), I clarified that that thing was very different from what it was like to actually be friends with them. The everyone-yelling operation seemed like a new innovation (that I didn't like) that they wield as a psychological weapon only against people who they think are operating in bad faith? In the present conversation with Scott, I had been focusing on rebutting the claim that my February–April 2017 (major) and March 2019 (minor) psych problems were caused by the "Vassarites", because with regard to those _specific_ incidents, the charge was absurd and false. But, well ... my January 2021 (minor) psych problems actually _were_ the result of being on the receiving end of the everyone-yelling thing. I briefly described the December 2020 "Lenore" disaster, and in particular the part where Michael/Jessica/Jack yelled at me.
-(In December, Jessica published [a followup post explaining the circumstances of her psychotic episode in more detail](https://www.lesswrong.com/posts/pQGFeKvjydztpgnsY/occupational-infohazards).
-
-Scott concedes: https://www.lesswrong.com/posts/MnFqyPLqbiKL8nSR7/my-experience-at-and-around-miri-and-cfar-inspired-by-zoe?commentId=RGKkmyvyoeWe2LB7d
-
-]
+[TODO: Scott post-Jessica Discord parley, cont'd]
------
After a month of dawdling, I eventually decided to pull the trigger on publishing "Challenges", without the extended political coda.[^coda] The post was a little bit mean to Yudkowsky, but not so mean that I was scared of the social consequences of pulling the trigger. (Yudkowsky had been mean to Christiano and Richard Ngo and Rohin Shah in [the recent MIRI dialogues](https://www.lesswrong.com/s/n945eovrA3oDueqtq); I didn't think this was worse than that.)
-[^coda]: The text from the draft coda would later be incorporated into the present post.
+[^coda]: The text from the draft coda would later be incorporated into the present memoir.
I cut the words "in this domain" from the go-for-the-throat concluding sentence that I had been worried about. "I'm better off because I don't trust Eliezer Yudkowsky to tell the truth," full stop.
/2017/Jan/from-what-ive-tasted-of-desire/
]
-Meanwhile, Yudkowsky started writing fiction again, largely in the form of Glowfic (a genre of collaborative storytelling pioneered by Alicorn) featuring the world of dath ilan (capitalization _sic_). Dath ilan had originally been introduced in a [2014 April Fool's Day post](https://yudkowsky.tumblr.com/post/81447230971/my-april-fools-day-confession), in which Yudkowsky "confessed" that the explanation for his seemingly implausible genius is that he's "actually" an ordinary person from a smarter, saner alternate version of Earth in which the ideas Yudkowsky presented to this world as his own, were commonplace.
+Meanwhile, Yudkowsky started writing fiction again, largely in the form of Glowfic (a genre of collaborative storytelling pioneered by Alicorn) featuring the world of dath ilan (capitalization _sic_). Dath ilan had originally been introduced in a [2014 April Fool's Day post](https://yudkowsky.tumblr.com/post/81447230971/my-april-fools-day-confession), in which Yudkowsky "confessed" that the explanation for his seemingly implausible genius is that he's "actually" an ordinary person from a smarter, saner alternate version of Earth where the ideas he presented to this world as his own were common knowledge.
The bulk of the dath ilan Glowfic canon was an epic titled [_Planecrash_](https://www.glowfic.com/boards/215)[^planecrash-title] coauthored with Lintamande, in which Keltham, an unusually selfish teenage boy from dath ilan, apparently dies in a freak aviation accident, and [wakes up in the world of](https://en.wikipedia.org/wiki/Isekai) Golarion, setting of the _Dungeons-&-Dragons_–alike _Pathfinder_ role-playing game. A [couple](https://www.glowfic.com/posts/4508) of [other](https://glowfic.com/posts/6263) Glowfic stories with different coauthors further flesh out the setting of dath ilan, which inspired a new worldbuilding trope, the [_medianworld_](https://www.glowfic.com/replies/1619639#reply-1619639), a setting where the average person is like the author along important dimensions.[^medianworlds]
Notwithstanding that Rittaen can be Watsonianly assumed to have detailed neuroscience skills that the author Doylistically doesn't know how to write, I am entirely unimpressed by the assertion that this idea is somehow _dangerous_, a secret that only Keepers can bear, rather than something _Merrin herself should be clued into_. "It's not [Rittaen's] place to meddle just because he knows Merrin better than Merrin does," we're told.
-In the same story, an agent from Exception Handling [tells Merrin that the bureau's Fake Conspiracy section is running an operation to plant evidence that Sparashki (the fictional alien Merrin happens to be dressed up as) are real](https://glowfic.com/replies/1860952#reply-1860952), and asks Merrin not to contradict this, and Merrin just ... goes along with it. (Elsewhere in the text, we're told that claiming to be a Sparashki isn't "lying", because no one would _expect_ someone to tell the truth in that situation.) It's in-character for Merrin to go along with it, because she's a pushover. My question is, why is it okay that Exception Handling has a Fake Conspiracies section, any more than it would have been if FTX or Enron explicitly had a Fake Accounting department? (Because dath ilan are the designated good guys? Well, so was FTX.)
+In the same story, an agent from Exception Handling [tells Merrin that the bureau's Fake Conspiracy section is running an operation to plant evidence that Sparashki (the fictional alien Merrin happens to be dressed up as) are real](https://glowfic.com/replies/1860952#reply-1860952), and asks Merrin not to contradict this, and Merrin just ... goes along with it. It's in-character for Merrin to go along with it, because she's a pushover. My question is, why is it okay that Exception Handling has a Fake Conspiracies section, any more than it would have been if FTX or Enron explicitly had a Fake Accounting department? (Because dath ilan are the designated good guys? Well, so was FTX.)
As another notable example of dath ilan hiding information for the alleged greater good, in Golarion, Keltham discovers that he's a sexual sadist, and deduces that Civilization has deliberately prevented him from realizing this, because there aren't enough corresponding masochists to go around in dath ilan. Having concepts for "sadism" and "masochism" as variations in human psychology would make sadists like Keltham sad about the desirable sexual experiences they'll never get to have, so Civilization arranges for them to _not be exposed to knowledge that would make them sad, because it would make them sad_ (!!).
projects / Ultimately_Untrue_Thought.git / blobdiff