Now it looks like we have—less time? Not just tautologically because time has passed (the 21st century is one-fifth over—closer to a quarter over), but because of new information from the visible results of the deep learning revolution.[^second-half] Yudkowsky seemed particularly [spooked by AlphaGo](https://www.lesswrong.com/posts/7MCqRnZzvszsxgtJi/christiano-cotra-and-yudkowsky-on-ai-progress?commentId=gQzA8a989ZyGvhWv2) [and AlphaZero](https://intelligence.org/2017/10/20/alphago/) in 2016–2017, not because superhuman board game players were dangerous, but because of what it implied about the universe of algorithms.
-There had been a post in the Sequences that made fun of "the people who just want to build a really big neural net." These days, it's increasingly looking like just building a really big neural net ... [actually works](https://www.gwern.net/Scaling-hypothesis)?—which seems like bad news; if it's "easy" for non-scientific-genius engineering talent to shovel large amounts of compute into the birth of powerful minds that we don't understand and don't know how to control, then it would seem that the world is soon to pass outside of our understanding and control.
+In part of the Sequences, Yudkowsky had been [dismissive of people who aspired to build AI without understanding how intelligence works](https://www.lesswrong.com/posts/fKofLyepu446zRgPP/artificial-mysterious-intelligence)—for example, by being overly impressed by the [surface analogy](https://www.lesswrong.com/posts/6ByPxcGDhmx74gPSm/surface-analogies-and-deep-causes) between artificial neural networks and the brain. He conceded the possibility of brute-forcing AI (if natural selection had eventually gotten there with no deeper insight, so could we) but didn't consider it a default and especially not a desirable path. (["If you don't know how your AI works, that is not good. It is bad."](https://www.lesswrong.com/posts/fKofLyepu446zRgPP/artificial-mysterious-intelligence))
+
+These days, it's increasingly looking like making really large neural nets ... [actually works](https://www.gwern.net/Scaling-hypothesis)?—which seems like bad news; if it's "easy" for non-scientific-genius engineering talent to shovel large amounts of compute into the birth of powerful minds that we don't understand and don't know how to control, then it would seem that the world is soon to pass outside of our understanding and control.
[^second-half]: In an unfinished slice-of-life short story I started writing _circa_ 2010, my protagonist (a supermarket employee resenting his job while thinking high-minded thoughts about rationality and the universe) speculates about "a threshold of economic efficiency beyond which nothing human could survive" being a tighter bound on future history than physical limits (like the heat death of the universe), and comments that "it imposes a sense of urgency to suddenly be faced with the fabric of your existence coming apart in ninety years rather than 10<sup>90</sup>."
I pointed out that that's exactly what one would expect if the Vassar/breakdown correlation was mostly a selection effect rather than causal—that is, if the causal graph was the fork "prone-to-psychosis ← underlying-bipolar-ish-condition → gets-along-with-Michael".
-I had also had a sleep-deprivation-induced-psychotic-break-with-hospitalization in February 2013, and shortly thereafter, I remember Anna remarking that I was sounding a lot like Michael. But I hadn't been talking to Michael at all beforehand! (My previous email conversation with him had been in 2010.) So what could Anna's brain have been picking up on, when she said that? My guess: there was some underlying dimension of psychological variation (psychoticism? bipolar?—you tell me; this is supposed to be Scott's professional specialty) where Michael and I were already weird/crazy in similar ways, and sufficiently bad stressors could push me further along that dimension (enough for Anna to notice). Was Scott also going to blame Yudkowsky for making people autistic?
+I had also had a sleep-deprivation-induced-psychotic-break-with-hospitalization in February 2013, and shortly thereafter, I remember Anna remarking that I was sounding a lot like Michael. But I hadn't been talking to Michael at all beforehand! (My previous email conversation with him had been in 2010.) So what could Anna's brain have been picking up on, when she said that? My guess: there was some underlying dimension of psychological variation (psychoticism? bipolar?—you tell me; this is supposed to be Scott's professional specialty) where Michael and I were already weird/crazy in similar ways, and sufficiently bad stressors could push me further along that dimension (enough for Anna to notice). Was Scott also going to blame Yudkowsky for making people [autistic](https://twitter.com/ESYudkowsky/status/1633396201427984384)?
Concerning the lightning parable, Scott said that from his perspective, the point of "Kolmogorov Complicity" was that, yes, people can be crazy, but that we have to live in Society without spending all our time freaking out about it. If, back in the days of my ideological anti-sexism, the first ten Yudkowsky posts I had read had said that men and women are psychologically different for biological reasons and that anyone who denies this is a mind-killed idiot—which Scott assumed Yudkowsky did think—he could imagine me being turned off. It was probably good for me and the world that that wasn't my first ten experiences of the rationalist community.
I thought the "you could not have generated the answer I just told you" gambit was a pretty dirty argumentative trick on Yudkowsky's part. (Given that I could, how would I be able to prove it?—this was itself a good use-case for spoilers.)
-As it happened, however, I _had_ already considered the case of spoilers as a class of legitimate infohazards, and was prepared to testify that I had already thought of it, and explain why hiding spoilers were relevantly morally different from coverups in my view. The previous night, 7 December 2022, I had had a phone call with Anna Salamon,[^evidence-of-independent-generation] in which (I pretty distinctly remember) citing dath ilan's practice of letting children figure out heliocentrism for themselves as not being objectionable in the way the sadism/masochism coverup was.
+As it happened, however, I _had_ already considered the case of spoilers as a class of legitimate infohazards, and was prepared to testify that I had already thought of it, and explain why hiding spoilers were relevantly morally different from coverups in my view. The previous night, 7 December 2022, I had had a phone call with Anna Salamon,[^evidence-of-independent-generation] in which (I pretty distinctly remember) citing dath ilan's [practice of letting children figure out heliocentrism for themselves](https://www.glowfic.com/replies/1777588#reply-1777588) as not being objectionable in the way the sadism/masochism coverup was.
[^evidence-of-independent-generation]: I was lucky to be able to point to Anna as a potential witness to defend myself against the "could not have generated" trick—as a matter of principle, not because I seriously expected anyone to go ask Anna if she remembered the conversation the same way.
As for the claim that my thinking was distorted and I was arguing instead of seeing, it was definitely true that I was _motivated to look for_ criticisms of Yudkowsky and dath ilan, for personal reasons outside the scope of the server, and I thought it was great for people to notice this and take it into account. I hoped to nevertheless be competent to only report real criticisms and not fake criticisms. (Whether I succeeded, of course, was up to the reader to decide.)
-[TODO: Yudkowsky tests me]
+[TODO: Yudkowsky tests me: (despite "Do not let the argument wander and become about something else, such as someone's virtue as a rationalist")]
[TODO: derail with Lintamande]
projects / Ultimately_Untrue_Thought.git / blobdiff