-[^second-half]: In an unfinished slice-of-life short story I started writing _circa_ 2010, my protagonist (a supermarket employee resenting his job while thinking high-minded thoughts about rationality and the universe) speculates about "a threshold of economic efficiency beyond which nothing human could survive" being a tighter bound on future history than physical limits (like the heat death of the universe), and comments that "it imposes a sense of urgency to suddenly be faced with the fabric of your existence coming apart in ninety years rather than 10<sup>90</sup>."
-
- But if ninety years is urgent, what about ... nine? Looking at what deep learning can do in 2023, the idea of Singularity 2032 doesn't seem self-evidently _absurd_ in the way that Singularity 2019 seemed absurd in 2010 (correctly, as it turned out).
-
-My AlphaGo moment was 5 January 2021, when OpenAI released [DALL-E](https://openai.com/blog/dall-e/) (by far the most significant news story of [that week in January 2021](https://en.wikipedia.org/wiki/January_6_United_States_Capitol_attack)). Previous AI milestones, like GANs for a _fixed_ image class, were easier to dismiss as clever statistical tricks. If you have thousands of photographs of people's faces, I didn't feel surprised that some clever algorithm could "learn the distribution" and spit out another sample; I don't know the _details_, but it doesn't seem like scary "understanding." DALL-E's ability to _combine_ concepts—responding to "an armchair in the shape of an avacado" as a novel text prompt, rather than already having thousands of examples of avacado-chairs and just spitting out another one of those—viscerally seemed more like "real" creativity to me, something qualitatively new and scary.[^qualitatively-new]
-
-[^qualitatively-new]: By mid-2022, DALL-E 2 and Midjourney and Stable Diffusion were generating much better pictures, but that wasn't surprising. Seeing AI being able to do a thing at all is the model update; AI being able to do the thing much better 18 months later feels "priced in."
-
-[As recently as 2020, I had been daydreaming about](/2020/Aug/memento-mori/#if-we-even-have-enough-time) working at an embryo selection company (if they needed programmers—but everyone needs programmers, these days), and having that be my altruistic[^eugenics-altruism] contribution to the great common task. Existing companies working on embryo selection [boringly](https://archive.is/tXNbU) [market](https://archive.is/HwokV) their services as being about promoting health, but [polygenic scores should work as well for maximizing IQ as they do for minimizing cancer risk](https://www.gwern.net/Embryo-selection).[^polygenic-score] Making smarter people would be a transhumanist good in its own right, and [having smarter biological humans around at the time of our civilization's AI transition](https://www.lesswrong.com/posts/2KNN9WPcyto7QH9pi/this-failing-earth) would give us a better shot at having it go well.[^ai-transition-go-well]
-
-[^eugenics-altruism]: If it seems odd to frame _eugenics_ as "altruistic", translate it as a term of art referring to the component of my actions dedicating to optimizing the world at large, as contrasted to "selfishly" optimizing my own experiences.
-
-[^polygenic-score]: Better, actually: [the heritability of IQ is around 0.65](https://en.wikipedia.org/wiki/Heritability_of_IQ), as contrasted to [about 0.33 for cancer risk](https://pubmed.ncbi.nlm.nih.gov/26746459/).
-
-[^ai-transition-go-well]: Natural selection eventually developed intelligent creatures, but evolution didn't know what it was doing and was not foresightfully steering the outcome in any particular direction. The more humans know what we're doing, the more our will determines the fate of the cosmos; the less we know what we're doing, the more our civilization is just another primordial soup for the next evolutionary transition.
-
-But pushing on embryo selection only makes sense as an intervention for optimizing the future if AI timelines are sufficiently long, and the breathtaking pace (or too-fast-to-even-take-a-breath pace) of the deep learning revolution is so much faster than the pace of human generations, that it's starting to look unlikely that we'll get that much time. If our genetically uplifted children would need at least twenty years to grow up to be productive alignment researchers, but unaligned AI is [on track to end the world in twenty years](https://www.lesswrong.com/posts/AfH2oPHCApdKicM4m/two-year-update-on-my-personal-ai-timelines), we would need to start having those children _now_ in order for them to make any difference at all.
-
-[It's ironic that "longtermism" got traction as the word for the "EA" cause area of benefitting the far future](https://applieddivinitystudies.com/longtermism-irony/), because the decision-relevant beliefs of most of the people who think about the far future, end up working out to extreme _short_-termism. Common-sense longtermism—a longtermism that assumed there's still going to be a recognizable world of humans in 2123—_would_ care about eugenics, and would be willing to absorb political costs today in order to fight for a saner future. The story of humanity would not have gone _better_ if Galileo had declined to publish for pre-emptive fear of the Inquisition.
-
-But if you think the only hope for there _being_ a future flows through maintaining influence over what large tech companies are doing as they build transformative AI, declining to contradict the state religion makes more sense—if you don't have _time_ to win a culture war, because you need to grab hold of the Singularity (or perform a [pivotal act](https://arbital.com/p/pivotal/) to prevent it) _now_. If the progressive machine marks you as a transphobic bigot, the machine's functionaries at OpenAI or Meta AI Research are less likely to listen to you when you explain why their safety plan won't work (or why they should have a safety plan at all).
-
-(I remarked to "Wilhelm" in June 2022 that DeepMind [changing its Twitter avatar to a rainbow variant of their logo for Pride month](https://web.archive.org/web/20220607123748/https://twitter.com/DeepMind) was a bad sign.)
-
-So isn't there a story here where I'm the villain, willfully damaging humanity's chances of survival by picking unimportant culture-war fights in the xrisk-reduction social sphere, when _I know_ that the sphere needs to keep its nose clean in the eyes of the progressive egregore? _That's_ why Yudkowsky said the arguably-technically-misleading things he said about my Something to Protect: he _had_ to, to keep our nose clean. The people paying attention to contemporary politics don't know what I know, and can't usefully be told. Isn't it better for humanity if my meager talents are allocated to making AI go well? Don't I have a responsibility to fall in line and take one for the team? If the world is at stake.
-
-As usual, the Yudkowsky of 2009 has me covered. In his short story ["The Sword of Good"](https://www.yudkowsky.net/other/fiction/the-sword-of-good), our protagonist Hirou wonders why the powerful wizard Dolf lets other party members risk themselves fighting, when Dolf could have protected them:
-
-> _Because Dolf was more important, and if he exposed himself to all the risk every time, he might eventually be injured_, Hirou's logical mind completed the thought. _Lower risk, but higher stakes. Cold but necessary–_
->
-> _But would you_, said another part of his mind, _would you, Hirou, let your friends walk before you and fight, and occasionally die, if you_ knew _that you yourself were stronger and able to protect them? Would you be able to stop yourself from stepping in front?_
->
-> _Perhaps_, replied the cold logic. _If the world were at stake._
->
-> _Perhaps_, echoed the other part of himself, _but that is not what was actually happening._
-
-That is, there's _no story_ under which misleading people about trans issues is on Yudkowsky's critical path for shaping the intelligence explosion. _I'd_ prefer him to have free speech, but if _he_ thinks he can't afford to be honest about things he [_already_ got right in 2009](https://www.lesswrong.com/posts/QZs4vkC7cbyjL9XA9/changing-emotions), he could just—not issue pronouncements on topics where he intends to _ignore counterarguments on political grounds!_
-
-In [a Twitter discussion about why not to trust organizations that refuse to explain their reasoning, Yudkowsky wrote](https://twitter.com/esyudkowsky/status/1374161729073020937):
-
-> Having some things you say "no comment" to, is not at _all_ the same phenomenon as being an organization that issues Pronouncements. There are a _lot_ of good reasons to have "no comments" about things. Anybody who tells you otherwise has no life experience, or is lying.
-
-Sure. But if that's your story, I think you need to _actually not comment_. ["[A]t least 20% of the ones with penises are actually women"](https://www.facebook.com/yudkowsky/posts/10154078468809228) is _not "no comment"._ ["[Y]ou're not standing in defense of truth if you insist on a word, brought explicitly into question, being used with some particular meaning"](https://twitter.com/ESYudkowsky/status/1067198993485058048) is _not "no comment"_. We did get a clarification on that one—but then, within a matter of months, he turned around and came back with his "simplest and best proposal" about how the "important things [...] would be all the things [he's] read [...] from human beings who are people—describing reasons someone does not like to be tossed into a Male Bucket or Female Bucket, as it would be assigned by their birth certificate", _which is also not "no comment."_
-
-[TODO: defying threats, cont'd—
-
- * I don't pick fights with Paul Christiano, because Paul Christiano doesn't take a shit on my Something to Protect, because Paul Christiano isn't trying to be a religious leader. If he has opinions about transgenderism, we don't know about them.
-
- * The cowardice is particularly puzzling in light of his timeless decision theory, which says to defy extortion.
-
- * Of course, there's a lot of naive misinterpretations of TDT that don't understand counterfactual dependence. There's a perspective that says, "We don't negotiate with terrorists, but we do appease bears", because the bear's response isn't calculated based on our response. /2019/Dec/political-science-epigrams/
-
- * You could imagine him mocking me for trying to reason this out, instead of just using honor. "That's right, I'm appealing to your honor, goddamn it!"
-
- * back in 'aught-nine, SingInst had made a point of prosecuting Tyler Emerson, citing decision theory
-
- * But the parsing of social justice as an agentic "threat" to be avoided rather than a rock to be dodged does seem to line up with the fact that people punish heretics more than infidels.
-
- * But it matters where you draw the zero point: is being excluded from the coalition a "punishment" to threaten you out of bad behavior, or is being included a "reward" for good behavior?
-
- * Curtis Yarvin has compared Yudkowsky to Sabbatai Zevi (/2020/Aug/yarvin-on-less-wrong/), and I've got to say the comparison is dead-on. Sabbatai Zevi was facing much harsher coercion: his choices were to convert to Islam or be impaled https://en.wikipedia.org/wiki/Sabbatai_Zevi#Conversion_to_Islam
-
-]
-
-I like to imagine that they have a saying out of dath ilan: once is happenstance; twice is coincidence; _three times is hostile optimization_.
-
-I could forgive him for taking a shit on d4 of my chessboard (["at least 20% of the ones with penises are actually women"](https://www.facebook.com/yudkowsky/posts/10154078468809228)).
-
-I could even forgive him for subsequently taking a shit on e4 of my chessboard (["you're not standing in defense of truth if you insist on a word [...]"](https://twitter.com/ESYudkowsky/status/1067198993485058048)) as long as he wiped most of the shit off afterwards (["you are being the bad guy if you try to shut down that conversation by saying that 'I can define the word "woman" any way I want'"](https://www.facebook.com/yudkowsky/posts/10158853851009228)), even though, really, I would have expected someone so smart to take a hint after the incident on d4.
-
-But if he's _then_ going to take a shit on c3 of my chessboard (["important things [...] would be all the things I've read [...] from human beings who are people—describing reasons someone does not like to be tossed into a Male Bucket or Female Bucket, as it would be assigned by their birth certificate", "the simplest and best protocol is, '"He" refers to the set of people who have asked us to use "he"'"](https://www.facebook.com/yudkowsky/posts/10159421750419228)), the "playing on a different chessboard, no harm intended" excuse loses its credibility. The turd on c3 is a pretty big likelihood ratio! (That is, I'm more likely to observe a turd on c3 in worlds where Yudkowsky _is_ playing my chessboard and wants me to lose, than in world where he's playing on a different chessboard and just _happened_ to take a shit there, by coincidence.)
-
------
-
-In June 2021, MIRI Executive Director Nate Soares [wrote a Twitter thread aruging that](https://twitter.com/So8res/status/1401670792409014273) "[t]he definitional gynmastics required to believe that dolphins aren't fish are staggering", which [Yudkowsky retweeted](https://archive.is/Ecsca).[^not-endorsements]
-
-[^not-endorsements]: In general, retweets are not necessarily endorsements—sometimes people just want to draw attention to some content without further comment or implied approval—but I was inclined to read this instance as implying approval, partially because this doesn't seem like the kind of thing someone would retweet for attention-without-approval, and partially because of the working relationship between Soares and Yudkowsky.
-
-Soares's points seemed cribbed from part I of Scott Alexander's ["... Not Man for the Categories"](https://slatestarcodex.com/2014/11/21/the-categories-were-made-for-man-not-man-for-the-categories/), which post I had just dedicated _more than three years of my life_ to rebutting in [increasing](/2018/Feb/the-categories-were-made-for-man-to-make-predictions/) [technical](https://www.lesswrong.com/posts/esRZaPXSHgWzyB2NL/where-to-draw-the-boundaries) [detail](https://www.lesswrong.com/posts/onwgTH6n8wxRSo2BJ/unnatural-categories-are-optimized-for-deception), _specifically using dolphins as my central example_—which Soares didn't necessarily have any reason to have known about, but Yudkowsky (who retweeted Soares) definitely did. (Soares's [specific reference to the Book of Jonah](https://twitter.com/So8res/status/1401670796997660675) made it seem particularly unlikely that he had invented the argument independently from Alexander.) [One of the replies (which Soares Liked) pointed out the similar _Slate Star Codex_ article](https://twitter.com/max_sixty/status/1401688892940509185), [as did](https://twitter.com/NisanVile/status/1401684128450367489) [a couple of](https://twitter.com/roblogic_/status/1401699930293432321) quote-Tweet discussions.
-
-The elephant in my brain took this as another occasion to _flip out_. I didn't _immediately_ see anything for me to overtly object to in the thread itself—[I readily conceded that](https://twitter.com/zackmdavis/status/1402073131276066821) there was nothing necessarily wrong with wanting to use the symbol "fish" to refer to the cluster of similarities induced by convergent evolution to the acquatic habitat rather than the cluster of similarities induced by phylogenetic relatedness—but in the context of our subculture's history, I read this as Soares and Yudkowsky implicitly lending more legitimacy to "... Not Man for the Categories", which was _hostile to my interests_. Was I paranoid to read this as a potential [dogwhistle](https://en.wikipedia.org/wiki/Dog_whistle_(politics))? It just seemed _implausible_ that Soares would be Tweeting that dolphins are fish in the counterfactual in which "... Not Man for the Categories" had never been published.
-
-After a little more thought, I decided the thread _was_ overtly objectionable, and [quickly wrote up a reply on _Less Wrong_](https://www.lesswrong.com/posts/aJnaMv8pFQAfi9jBm/reply-to-nate-soares-on-dolphins): Soares wasn't merely advocating for a "swimmy animals" sense of the word _fish_ to become more accepted usage, but specifically deriding phylogenetic definitions as unmotivated for everyday use ("definitional gynmastics [_sic_]"!), and _that_ was wrong. It's true that most language users don't directly care about evolutionary relatedness, but [words aren't identical with their definitions](https://www.lesswrong.com/posts/i2dfY65JciebF3CAo/empty-labels). Genetics is at the root of the causal graph underlying all other features of an organism; creatures that are more closely evolutionarily related are more similar _in general_. Classifying things by evolutionary lineage isn't an arbitrary æsthetic whim by people who care about geneology for no reason. We need the natural category of "mammals (including marine mammals)" to make sense of how dolphins are warm-blooded, breathe air, and nurse their live-born young, and the natural category of "finned cold-blooded vertebrate gill-breathing swimmy animals (which excludes marine mammals)" is also something that it's reasonable to have a word for.
-
-(Somehow, it felt appropriate to use a quote from Arthur Jensen's ["How Much Can We Boost IQ and Scholastic Achievement?"](https://en.wikipedia.org/wiki/How_Much_Can_We_Boost_IQ_and_Scholastic_Achievement%3F) as an epigraph.)
-
-[TODO: dolphin war con'td
-
- * Nate conceded all of my points (https://twitter.com/So8res/status/1402888263593959433), said the thread was in jest ("shitposting"), and said he was open to arguments that he was making a mistake (https://twitter.com/So8res/status/1402889976438611968), but still seemed to think his shitposting was based
-
- * I got frustrated and lashed out; "open to arguments that he was making a mistake" felt fake to me; rats are good at paying lip service to humility, but I'd lost faith in getting them to change their behavior, like not sending PageRank to "... Not Man for the Categories"
-
- * Nate wrote a longer reply on Less Wrong the next morning
-
- * I pointed out that his followup thread lamented that people hadn't read "A Human's Guide to Words", but that Sequence _specifically_ used the example of dolphins. What changed?!?
-
- * [Summarize Nate's account of his story], phylogeny not having the courage of its convictions
-
- * Twitter exchange where he said he wasn't sure I would count his self-report as evidnece, I said it totally counts
-
- * I overheated. This was an objectively dumb play. (If I had cooled down and just written up my reply, I might have gotten real engagement and a resolution, but I blew it.) I apologized a few days later.
-
- * Nate's reaction to me blowing up said it looked like I was expecting deference. I deny this; I wouldn't expect people to defer to me—what I did expect was a fair hearing, and at this point, I had lost faith that I would get one. (Could you blame me, when Yudkowsky says a fair hearing is less important than agreeing with Stalin?)
-
- * My theory of what's going on: I totally believe Nate's self report that he wasn't thinking about gender. (As Nate pointed out, you could give the thread an anti-trans interpretation, too.) Nevertheless, it remains the case that Nate's thinking is causally downstream of Scott's arguments in "... Not Man for the Categories." Where did Scott get it from? I think he pulled it out of his ass because it was politically convenient.
-
- * This is like radiocontrast dye for dark side epistemology: we can see Scott sneezing his bad epistemology onto everyone else because he's such a popular writer. No one can think fast enough to think their own thoughts, but you would hope for an intellectual community that can do error-correction, rather than copying smart people's views including mistakes.
-
- * I look up the relevant phylogenetics definitions, and write "Blood Is Thicker Than Water"
-
-]
-
-
-[TODO:
-
- * depressed after talking to him at Independence Day party 2021 (I can mention that, because it was outdoors and probably lots of other people saw us, even if I can't talk about content)
-
- * It wouldn't be so bad if he weren't trying to sell himself as a religious leader, and profiting from the conflation of rationalist-someone-who-cares-about-reasoning, and rationalist-member-of-robot-cult
-
- * But he does, in fact, seem to actively encourage this conflation (contrast to how the Sequences had a [Litany Against Gurus](https://www.lesswrong.com/posts/t6Fe2PsEwb3HhcBEr/the-litany-against-gurus) these days, with the way he sneers as Earthlings and post-rats)
-
- * "I may as well do it on Earth"
-
- * a specific example that made me very angry in September 2021—
-
-https://twitter.com/ESYudkowsky/status/1434906470248636419
-> Anyways, Scott, this is just the usual division of labor in our caliphate: we're both always right, but you cater to the crowd that wants to hear it from somebody too modest to admit that, and I cater to the crowd that wants somebody out of that closet.
-
-Okay, I get that it was meant as humorous exaggeration. But I think it still has the effect of discouraging people from criticizing Scott or Eliezer because they're the leaders of the Caliphate. I spent three and a half years of my life explaining in exhaustive, exhaustive detail, with math, how Scott was wrong about something, no one serious actually disagrees, and Eliezer is still using his social power to boost Scott's right-about-everything (!!) reputation. That seems really unfair, in a way that isn't dulled by "it was just a joke."
-
-Or [as Yudkowsky put it](https://www.facebook.com/yudkowsky/posts/10154981483669228)—
-
-> I know that it's a bad sign to worry about which jokes other people find funny. But you can laugh at jokes about Jews arguing with each other, and laugh at jokes about Jews secretly being in charge of the world, and not laugh at jokes about Jews cheating their customers. Jokes do reveal conceptual links and some conceptual links are more problematic than others.
-
-It's totally understandable to not want to get involved in a political scuffle because xrisk reduction is astronomically more important! But I don't see any plausible case that metaphorically sucking Scott's dick in public reduces xrisk. It would be so easy to just not engage in this kind of cartel behavior!
-
-An analogy: racist jokes are also just jokes. Alice says, "What's the difference between a black dad and a boomerang? A boomerang comes back." Bob says, "That's super racist! Tons of African-American fathers are devoted parents!!" Alice says, "Chill out, it was just a joke." In a way, Alice is right. It was just a joke; no sane person could think that Alice was literally claiming that all black men are deadbeat dads. But, the joke only makes sense in the first place in context of a culture where the black-father-abandonment stereotype is operative. If you thought the stereotype was false, or if you were worried about it being a self-fulfilling prophecy, you would find it tempting to be a humorless scold and get angry at the joke-teller.
-
-Similarly, the "Caliphate" humor _only makes sense in the first place_ in the context of a celebrity culture where deferring to Yudkowsky and Alexander is expected behavior. (In a way that deferring to Julia Galef or John S. Wentworth is not expected behavior, even if Galef and Wentworth also have a track record as good thinkers.) I think this culture is bad. _Nullius in verba_.
-
- * the fact that David Xu interpreted criticism of the robot cult as me going "full post-rat" suggests that Yudkowsky's framing had spilled onto others. (The framing is optimized to delegitimize dissent. Motte: someone who's critical of central rationalists; bailey: someone who's moved beyond reason.)
-
-sneering at post-rats; David Xu interprets criticism of Eliezer as me going "full post-rat"?! 6 September 2021
-
-> Also: speaking as someone who's read and enjoyed your LW content, I do hope this isn't a sign that you're going full post-rat. It was bad enough when QC did it (though to his credit QC still has pretty decent Twitter takes, unlike most post-rats).
-
-https://twitter.com/davidxu90/status/1435106339550740482
-
-https://twitter.com/zackmdavis/status/1435856644076830721
-> The error in "Not Man for the Categories" is not subtle! After the issue had been brought to your attention, I think you should have been able to condemn it: "Scott's wrong; you can't redefine concepts in order to make people happy; that's retarded." It really is that simple! 4/6
-
-I once wrote [a post whimsically suggesting that trans women should owe cis women royalties](/2019/Dec/comp/) for copying the female form (as "intellectual property"). In response to a reader who got offended, I [ended up adding](/source?p=Ultimately_Untrue_Thought.git;a=commitdiff;h=03468d274f5) an "epistemic status" line to clarify that it was not a serious proposal.
-
-But if knowing it was a joke partially mollifies the offended reader who thought I might have been serious, I don't think they should be _completely_ mollified, because the joke (while a joke) reflects something about my thinking when I'm being serious: I don't think sex-based collective rights are inherently a suspect idea; I think _something of value has been lost_ when women who want female-only spaces can't have them, and the joke reflects the conceptual link between the idea that something of value has been lost, and the idea that people who have lost something of value are entitled to compensation.
-
-At Valinor's 2022 [Smallpox Eradication Day](https://twitter.com/KelseyTuoc/status/1391248651167494146) party, I remember overhearing[^overhearing] Yudkowsky saying that OpenAI should have used GPT-3 to mass-promote the Moderna COVID-19 vaccine to Republicans and the Pfizer vaccine to Democrats (or vice versa), thereby harnessing the forces of tribalism in the service of public health.
-
-[^overhearing]: I claim that conversations at a party with lots of people are not protected by privacy norms; if I heard it, several other people heard it; no one had a reasonable expectation that I shouldn't blog about it.