But if ninety years is urgent, what about ... nine? Looking at what deep learning can do in 2023, the idea of Singularity 2032 doesn't seem self-evidently _absurd_ in the way that Singularity 2019 seemed absurd in 2010 (correctly, as it turned out).
-My AlphaGo moment was 5 January 2021, when OpenAI released [DALL-E](https://openai.com/blog/dall-e/) (by far the most significant news story of that week in January 2021). Previous AI milestones, like GANs for a _fixed_ image class, were easier to dismiss as clever statistical tricks. If you have thousands of photographs of people's faces, I didn't feel surprised that some clever algorithm could "learn the distribution" and spit out another sample; I don't know the _details_, but it doesn't seem like scary "understanding." DALL-E's ability to _combine_ concepts—responding to "an armchair in the shape of an avacado" as a novel text prompt, rather than already having thousands of examples of avacado-chairs and just spitting out another one of those—viscerally seemed more like "real" creativity to me, something qualitatively new and scary.[^qualitatively-new]
+My AlphaGo moment was 5 January 2021, when OpenAI released [DALL-E](https://openai.com/blog/dall-e/) (by far the most significant news story of [that week in January 2021](https://en.wikipedia.org/wiki/January_6_United_States_Capitol_attack)). Previous AI milestones, like GANs for a _fixed_ image class, were easier to dismiss as clever statistical tricks. If you have thousands of photographs of people's faces, I didn't feel surprised that some clever algorithm could "learn the distribution" and spit out another sample; I don't know the _details_, but it doesn't seem like scary "understanding." DALL-E's ability to _combine_ concepts—responding to "an armchair in the shape of an avacado" as a novel text prompt, rather than already having thousands of examples of avacado-chairs and just spitting out another one of those—viscerally seemed more like "real" creativity to me, something qualitatively new and scary.[^qualitatively-new]
[^qualitatively-new]: By mid-2022, DALL-E 2 and Midjourney and Stable Diffusion were generating much better pictures, but that wasn't surprising. Seeing AI being able to do a thing at all is the model update; AI being able to do the thing much better 18 months later feels "priced in."
But if you think the only hope for there _being_ a future flows through maintaining influence over what large tech companies are doing as they build transformative AI, declining to contradict the state religion makes more sense—if you don't have _time_ to win a culture war, because you need to grab hold of the Singularity (or perform a [pivotal act](https://arbital.com/p/pivotal/) to prevent it) _now_. If the progressive machine marks you as a transphobic bigot, the machine's functionaries at OpenAI or Meta AI Research are less likely to listen to you when you explain why their safety plan won't work (or why they should have a safety plan at all).
-(I remarked to "Wilhelm" in June 2022 that DeepMind changing its Twitter avatar to a rainbow variant of their logo for Pride month was a bad sign.)
+(I remarked to "Wilhelm" in June 2022 that DeepMind [changing its Twitter avatar to a rainbow variant of their logo for Pride month](https://web.archive.org/web/20220607123748/https://twitter.com/DeepMind) was a bad sign.)
So isn't there a story here where I'm the villain, willfully damaging humanity's chances of survival by picking unimportant culture-war fights in the xrisk-reduction social sphere, when _I know_ that the sphere needs to keep its nose clean in the eyes of the progressive egregore? _That's_ why Yudkowsky said the arguably-technically-misleading things he said about my Something to Protect: he _had_ to, to keep our nose clean. The people paying attention to contemporary politics don't know what I know, and can't usefully be told. Isn't it better for humanity if my meager talents are allocated to making AI go well? Don't I have a responsibility to fall in line and take one for the team? If the world is at stake.
What's _not_ fine is selective argumentation while claiming "confidence in [your] own ability to independently invent everything important that would be on the other side of the filter and check it [yourself] before speaking" when you _very obviously have done no such thing_.
-[TODO: https://www.lesswrong.com/posts/MnFqyPLqbiKL8nSR7/my-experience-at-and-around-miri-and-cfar-inspired-by-zoe October 2021
-https://www.lesswrong.com/posts/pQGFeKvjydztpgnsY/occupational-infohazards December 2021
- * when Jessica published her story, the karma took a nosedive when Scott commented blaming all of Jessica's problems on Michael, and Yudkowsky backed up Scott; to me, this looks like raw factional conflict: Jessica had some negative-valence things to say about the Calilphate, so Caliphate leaders move in to discredit her by association.
- * (extract points from my conversation about Michael with Scott)
+------
+
+In October 2021, Jessica Taylor [published a memoir about her experiences at MIRI](https://www.lesswrong.com/posts/MnFqyPLqbiKL8nSR7/my-experience-at-and-around-miri-and-cfar-inspired-by-zoe), making analogies between sketchy social pressures she had experienced in the core rationalist community (around short AI timelines, secrecy, deference to community leaders, _&c._) and those reported in [Zoe Cramer's recent account of Leverage Research](https://medium.com/@zoecurzi/my-experience-with-leverage-research-17e96a8e540b).
+
+Scott Alexander posted a comment "add[ing] some context [he thought was] important to this", essentially blaming Jessica's problems on her association with Michael Vassar, describing her psychotic episode as a "Vassar-related phenomenon" (!).
+
+I thought this was unfair, and [said so](https://www.lesswrong.com/posts/MnFqyPLqbiKL8nSR7/my-experience-at-and-around-miri-and-cfar-inspired-by-zoe?commentId=GzqsWxEp8uLcZinTy) (and [offered textual evidence](https://www.lesswrong.com/posts/MnFqyPLqbiKL8nSR7/my-experience-at-and-around-miri-and-cfar-inspired-by-zoe?commentId=yKo2uuCcwJxbwwyBw) against the claim that Michael was _trying_ to drive Jessica crazy).
+
+To me, Scott's behavior looked like raw factional conflict: Jessica had some negative-valence things to say about the Caliphate, so Caliphate leaders move in to discredit her by association.
+
+It was effective, though. After Alexander's comment (and [a comment from Yudkowsky](https://www.lesswrong.com/posts/MnFqyPLqbiKL8nSR7/my-experience-at-and-around-miri-and-cfar-inspired-by-zoe?commentId=x5ajGhggHky9Moyr8) uncritically accepting Alexander's charge of Vassar "causing psychotic breaks in people"), the karma score on Jessica's post dropped by more than half, while Alexander's comment got voted up to more than 380 karma.
+
+[TODO my conversation with Scott—
+
+> when you had some more minor issues in 2019 I was more in the loop and I ended out emailing the Vassarites (deliberately excluding you from the email, a decision I will defend in private if you ask me) accusing them of making your situation worse and asking them to maybe lay off you until you were maybe feeling slightly better, and obviously they just responded with their "it's correct to be freaking about learning your entire society is corrupt and gaslighting" shtick.
+
+ * Scott interviewed me
+ * I said
]
+In December, Jessica published [a followup post explaining the circumstances of her psychotic episode in more detail](https://www.lesswrong.com/posts/pQGFeKvjydztpgnsY/occupational-infohazards).
+
+[TODO: Scott concedes: https://www.lesswrong.com/posts/MnFqyPLqbiKL8nSR7/my-experience-at-and-around-miri-and-cfar-inspired-by-zoe?commentId=RGKkmyvyoeWe2LB7d ]
+
+------
+
[TODO:
Is this the hill _he_ wants to die on? If the world is ending either way, wouldn't it be more dignified for him to die _without_ Stalin's dick in his mouth?
* Although, as I struggled to write, I noticed I was wasting cycles worrying about what he'd think of me
* January 2022, I wrote to him asking if he cared if I said negative things about him, that it would be easier if he wouldn't hold it against me, and explained my understanding of the privacy norm (Subject: "blessing to speak freely, and privacy norms?")
* in retrospect, I was wrong to ask that. I _do_ hold it against him. And if I'm entitled to my feelings, isn't he entitled to his?
+ * what is the exact scope of not bothering him? I actually had left a Facebook comment shortly after blocking him on Twitter, and his reply seemed to imply that I did have commenting privileges (yudkowsky-twitter_is_worse_for_you.png)
]
In February 2022, I finally managed to finish a draft of ["Challenges to Yudkowsky's Pronoun Reform Proposal"](/2022/Mar/challenges-to-yudkowskys-pronoun-reform-proposal/) (A year after the post it replies to! I did other things that year, probably.) It's long (12,000 words), because I wanted to be thorough and cover all the angles. (To paraphrase Ralph Waldo Emerson, when you strike at Eliezer Yudkowsky, _you must kill him._)
I wanted to believe that if I wrote all the words dotting every possible _i_ and crossing every possible _t_ at all three levels of meta, then that would make it [a description and not an attack](http://benjaminrosshoffman.com/can-crimes-be-discussed-literally/)—that I could have it both ways if I explained the lower level of organization beneath the high-level abstractions of "betraying his principles and being dishonest." If that didn't work because [I only had five words](https://www.lesswrong.com/posts/4ZvJab25tDebB8FGE/you-have-about-five-words), then—I didn't know what I'd do. I'd think about it.
-After a month of dawdling, I eventually decided to pull the trigger on publishing "Challenges", without the extended political coda.[^coda] The post was a little bit mean to Yudkowsky, but not so mean that I was scared of the social consequences of pulling the trigger. (Yudkowsky had been mean to Christiano and Richard Ngo and Rohin Shah in the recent MIRI dialogues; I didn't think this was worse than that.)
+After a month of dawdling, I eventually decided to pull the trigger on publishing "Challenges", without the extended political coda.[^coda] The post was a little bit mean to Yudkowsky, but not so mean that I was scared of the social consequences of pulling the trigger. (Yudkowsky had been mean to Christiano and Richard Ngo and Rohin Shah in [the recent MIRI dialogues](https://www.lesswrong.com/s/n945eovrA3oDueqtq); I didn't think this was worse than that.)
[^coda]: The text from the draft coda would later be incorporated into the present post.
https://www.lesswrong.com/posts/nCvvhFBaayaXyuBiD/shut-up-and-do-the-impossible
- * swimming to shore analogy
+ * swimming to shore analogy https://www.lesswrong.com/posts/j9Q8bRmwCgXRYAgcJ/miri-announces-new-death-with-dignity-strategy?commentId=R59aLxyj3rvjBLbHg
+
+> your plane crashed in the ocean. To survive, you must swim to shore. You know that the shore is west, but you don't know how far. The optimist thinks the shore is just over the horizon; we only need to swim a few miles and we'll almost certainly make it. The pessimist thinks the shore is a thousand miles away and we will surely die. But the optimist and pessimist can both agree on how far we've swum up to this point, and that the most dignified course of action is "Swim west as far as you can."
* I've believed since Kurzweil that technology will remake the world sometime in the 21th century; it's just "the machines won't replace us, because we'll be them" doesn't seem credible
Meanwhile, Yudkowsky started writing fiction again, largely in the form of Glowfic (a genre of collaborative storytelling pioneered by Alicorn) featuring the world of dath ilan (capitalization _sic_). Dath ilan had originally been introduced in a [2014 April Fool's Day post](https://yudkowsky.tumblr.com/post/81447230971/my-april-fools-day-confession), in which Yudkowsky "confessed" that the explanation for his seemingly implausible genius is that he's "actually" an ordinary person from a smarter, saner alternate version of Earth in which the ideas Yudkowsky presented to this world as his own, were commonplace.
-The bulk of the dath ilan Glowfic canon was an epic titled _Planecrash_,[^planecrash-title] in which Keltham, an unusually selfish teenage boy in dath ilan, apparently dies in a freak aviation accident, and wakes up in the world of Golarion, setting of the _Dungeons-&-Dragons_–alike _Pathfinder_ role-playing game.
+The bulk of the dath ilan Glowfic canon was an epic titled [_Planecrash_](https://www.glowfic.com/boards/215)[^planecrash-title] coauthored with Lintamande, in which Keltham, an unusually selfish teenage boy from dath ilan, apparently dies in a freak aviation accident, and [wakes up in the world of](https://en.wikipedia.org/wiki/Isekai) Golarion, setting of the _Dungeons-&-Dragons_–alike _Pathfinder_ role-playing game. A [couple](https://www.glowfic.com/posts/4508) of [other](https://glowfic.com/posts/6263) Glowfic stories with different coauthors further flesh out the worldbuilding of dath ilan, which inspired a new worldbuilding trope, the [_medianworld_](https://www.glowfic.com/replies/1619639#reply-1619639), a setting where the average person is like the author along important dimensions.
[^planecrash-title]: The title is a pun, referring to both the airplane crash leading to Keltham's death in dath ilan, and how his resurrection in Golarion collides dath ilan with [the "planes" of existence of the _Pathfinder_ universe](https://pathfinderwiki.com/wiki/Great_Beyond).
-[TODO background on Planecrash &c. cont'd—
- * later retconned to be a Yudkowsky medianworld (use my notes from "Another Me I haven't Met" to explain this), the Word of God calls them the "eliezera"
- * dath ilan's Society is steered by Keepers of Unpleasant Truths That Sometimes Important to Know; everyone in this Society receives rationality training, but Keepers receive special training that would damage most people's humanity or happiness
- * Keltham discovers that he's an obligate sexual sadist, but dath ilan has hidden this from him for his own happiness
+[^medianworlds]: You might think that the thought experiment of imagining what someone's medianworld is like would only be interesting for people who are "weird" in our own world, thinking that our world is a medianworld for people who are normal in our world. But [in high-dimensional spaces, _most_ of the probability-mass is concentrated in a "shell" some distance around the mode](/2021/May/sexual-dimorphism-in-the-sequences-in-relation-to-my-gender-problems/#typical-point), because even though the per-unit-hypervolume probability _density_ is greatest at the mode, there's vastly _more_ hypervolume in the space around it. The upshot is that typical people are atypical along _some_ dimensions, so normies can play the medianworld game, too.
+
+(I asked Anna how Yudkowsky could stand the Glowfic people. She said she thought Eliezer could barely stand anyone. That makes sense, I said.)
-(The title is a pun, referring to both the airplane crash leading to Keltham's death in dath ilan, and how his resurrection in Golarion collides dath ilan with [the "planes" of existence of the _Pathfinder_ universe](https://pathfinderwiki.com/wiki/Great_Beyond).)
+Everyone in dath ilan receives rationality training from childhood, but knowledge and training deemed psychologically hazardous to the general population is safeguarded by an order of [Keepers of Highly Unpleasant Things it is Sometimes Necessary to Know](https://www.glowfic.com/replies/1612937#reply-1612937). AGI research takes place in a secret underground city; the culture of the masses is carefully steered away from ordinary people thinking about AI.
-(I asked Anna how Eliezer could stand the Glowfic people. She said she thought Eliezer could barely stand anyone. That makes sense, I said.)
+Something that annoyed me about the portrayal of dath ilan was their incredibly casual attitude towards hiding information for some alleged greater good, seemingly without considering that [there are benefits and not just costs to the public knowing things](http://benjaminrosshoffman.com/humility-argument-honesty/).
+You can, of course, make up any number of sensible [Watsonian](https://tvtropes.org/pmwiki/pmwiki.php/Main/WatsonianVersusDoylist) rationales for this. (For example, a world with much smarter people is more "volatile"; with more ways to convert knowledge into danger, maybe you _need_ more censorship just to hold Society together.)
+
+I'm more preoccupied by a [Doylistic](https://tvtropes.org/pmwiki/pmwiki.php/Main/WatsonianVersusDoylist) interpretation—that dath ilan's obsessive secret-keeping reflects something deep about how the Yudkowsky of the current year relates to information, in contrast to the Yudkowsky who wrote the Sequences. The Sequences had encouraged you—yes, _you_, the reader—to be as rational as possible. In contrast, the dath ilan mythos seems to portray advanced rationality as dangerous knowledge that people need to be protected from.
+
+Someone at the 2021 Event Horizon Independence Day party had told me that I had been misinterpreting the "Speak the truth, even if your voice trembles" slogan from the Sequences. I had interpreted the slogan as suggesting the importance of speaking the truth _to other people_ (which I think is what "speaking" is usually about), but my interlocutor said it was about, for example, being able to speak the truth aloud in your own bedroom, to yourself. I think some textual evidence for my interpretation can be found in Daria's ending to ["A Fable of Science and Politics"](https://www.lesswrong.com/posts/6hfGNLf4Hg5DXqJCF/a-fable-of-science-and-politics):
+
+> Daria, once Green, tried to breathe amid the ashes of her world. _I will not flinch_, Daria told herself, _I will not look away_. She had been Green all her life, and now she must be Blue. Her friends, her family, would turn from her. _Speak the truth, even if your voice trembles_, her father had told her; but her father was dead now, and her mother would never understand. Daria stared down the calm blue gaze of the sky, trying to accept it, and finally her breathing quietened. _I was wrong_, she said to herself mournfully; _it's not so complicated, after all_. She would find new friends, and perhaps her family would forgive her ... or, she wondered with a tinge of hope, rise to this same test, standing underneath this same sky? "The sky is blue," Daria said experimentally, and nothing dire happened to her; but she couldn't bring herself to smile. Daria the Blue exhaled sadly, and went back into the world, wondering what she would say.
+
+Daria takes it as a given that she needs to be open about her new blue-sky belief, even though it's socially costly to herself and to her loved ones; the rationalist wisdom from her late father did _not_ say to go consult a Keeper to check whether telling everyone about the blue sky is a good idea.[^other-endings] I think this reflects the culture of the _Overcoming Bias_ in 2006 valuing the existence of a shared social reality that reflects actual reality: the conviction that it's both possible and desirable for people to rise to the same test, standing underneath the same sky.
+
+[^other-endings]: Even Eddin's ending, which portrays Eddin as more concerned with consequences than honesty, has him "trying to think of a way to prevent this information from blowing up the world", rather than trying to think of a way to suppress the information, in contrast to how Charles, in his ending, immediately comes up with the idea to block off the passageway. Daria and Eddin are clearly written as "rationalists"; the deceptive strategy only comes naturally to the non-rationalist Charles.
+
+In contrast, the culture of dath ilan does not seem to particularly value people _standing underneath the same sky_.
+
+For example, we are told of an Ordinary Merrin Conspiracy, centered around a famous medical technician with a psychological need to feel unimportant, of whom ["everybody in Civilization is coordinating to pretend around her"](https://www.glowfic.com/replies/1764946#reply-1764946) that her achievements are nothing special, which is deemed to be kindness to her. It's like a reverse [Emperor Norton](https://en.wikipedia.org/wiki/Emperor_Norton) situation. (Norton was ordinary, but everyone around him colluded to make him think he was special; Merrin is special, but everyone around her colludes to make her think she's ordinary.)
+
+And _as_ a rationalist, I condemn the Ordinary Merrin Conspiracy as _morally wrong_, for the same [reasons I condemn the Emperor Norton Conspiracy](/2018/Feb/the-categories-were-made-for-man-to-make-predictions/#emperor-norton). As [it was taught to me on _Overcoming Bias_ back in the 'aughts](https://www.lesswrong.com/posts/HYWhKXRsMAyvRKRYz/you-can-face-reality): what's true is already so. Denying it won't make it better. Acknowledging it won't make it worse. And _because_ it is true, it is what is there to be interacted with. Anything untrue isn't there to be lived. People can stand what is true, _because they are already doing so_.
+
+In [the story about how Merrin came to the attention of Exception Handling](https://glowfic.com/posts/6263), we see the thoughts of a Keeper, Rittaen, who talks to Merrin. We're told that the discipline of modeling people mechanistically rather than through sympathy is restricted to Keepers because it runs the risk of ["turning into an exceptionally dangerous psychopath"](https://glowfic.com/replies/1862201#reply-1862201). Rittaen [uses his person-as-machine Sight](https://glowfic.com/replies/1862204#reply-1862204) to infer that Merrin was biologically predisposed to learn to be afraid of having too much status.
+
+Notwithstanding that Rittaen can be Watsonianly assumed to have detailed neurology skills that the author Doylistically doesn't know how to write, I am entirely unimpressed by the assertion that this idea is somehow _dangerous_, a secret that only Keepers can bear, rather than something _Merrin herself should be clued into_. We're told that "[i]t's not [Rittaen's] place to meddle just because he knows Merrin better than Merrin does"—reflecting an attitude that revealing information is _meddling_, rather than talking being a free action.
+
+[ TODO—
+ * I think there's even a part (find it) where the Keeper says it's nice that they live in a world where not everyone has to be a Keeper just for the world to survive, portraying rationality, knowledge as a burden
]
+As another notable example of dath ilan hiding information for the alleged greater good, in Golarion, Keltham discovers that he's a sexual sadist, and deduces that Civilization has deliberately prevented him from realizing this, because there aren't enough corresponding masochists to go around. Having concepts for "sadism" and "masochism" as variations in human psychology would make sadists like Keltham sad about the desirable sexual experiences they'll never get to have, so Civilization arranges for them to _not be exposed to knowledge that would make them sad_.
+
It did not escape my notice that when "rationalist" authorities _in real life_ considered public knowledge of some paraphilia to be an infohazard (ostensibly for the benefit of people with that paraphilia), I _didn't take it lying down_.
-I ended up writing a few critical messages on dath ilan's censorship regime in the "Eliezerfic" Discord server for reader discussion of _Planecrash_ and related works.
+I had meant to only use this parallel between dath ilan's masochism coverup and the autogynephilia coverup I had fought in real life as an amusing comment-in-passing in the present memoir, but as I was having trouble focusing on writing in late 2022, I ended up writing a few critical messages on dath ilan's censorship regime in the "Eliezerfic" Discord server for reader discussion of _Planecrash_, using the masochism coverup as my central example.
+
+Although Yudkowsky participated in the server, I had reasoned that my participation didn't violate my previous intent not to bother him anymore, because it was a publicly-linked Discord server with hundreds of members. Me criticizing the story for the _other_ 499 people in the chat room wouldn't generate a notification _for him_, the way it would if I sent him an email or replied to him on Twitter.
-(Although Yudkowsky participated in the server, I had reasoned that my participation didn't violate my previous intent not to bother him anymore, because it was a publicly-linked Discord server with hundreds of members. Me criticizing the story for the _other_ 499 people in the chat room wouldn't generate a notification _for him_, the way it would if I sent him an email or replied to him on Twitter.)
+[TODO: Eliezerfic fight]
[TODO: regrets and wasted time
* Do I have regrets about this Whole Dumb Story? A lot, surely—it's been a lot of wasted time. But it's also hard to say what I should have done differently; I could have listened to Ben more and lost faith Yudkowsky earlier, but he had earned a lot of benefit of the doubt?
projects / Ultimately_Untrue_Thought.git / blobdiff