[^egan-paraphrasing]: The original quote says "one hundred thousand straights" ... "gay community" ... "gay and lesbian" ... "franchise rights on homosexuality" ... "unauthorized queer."
-Recapping our Whole Dumb Story so far: in a previous post, "Sexual Dimorphism in Yudkowsky's Sequences, in Relation to My Gender Problems", I told you about how I've always (since puberty) had this obsessive erotic fantasy about being magically transformed into a woman and how I used to think it was immoral to believe in psychological sex differences, until I read these really great Sequences of blog posts by Eliezer Yudkowsky which incidentally pointed out how absurdly impossible my obsessive fantasy was—
+Recapping our Whole Dumb Story so far: in a previous post, "Sexual Dimorphism in Yudkowsky's Sequences, in Relation to My Gender Problems", I told you about how I've always (since puberty) had this obsessive erotic fantasy about being magically transformed into a woman and how I used to think it was immoral to believe in psychological sex differences, until I read these really great Sequences of blog posts by Eliezer Yudkowsky which incidentally pointed out how absurdly impossible my obsessive fantasy was ...
-—none of which gooey private psychological minutiæ would be at all in the public interest to blog about _except that_, as I explained in a subsequent post, "Blanchard's Dangerous Idea and the Plight of the Lucid Crossdreamer", in 2016, everyone in the community that formed around the Sequences suddenly decided for political reasons that guys like me might actually be women in some unspecified metaphysical sense, and the cognitive dissonance of having to rebut all this nonsense coming from everyone I used to trust, drove me temporarily insane from stress and sleep deprivation—
+—none of which gooey private psychological minutiæ would be at all in the public interest to blog about _except that_, as I explained in a subsequent post, "Blanchard's Dangerous Idea and the Plight of the Lucid Crossdreamer", around 2016ish, everyone in the community that formed around the Sequences suddenly decided for political reasons that guys like me might actually be women in some unspecified metaphysical sense, and the cognitive dissonance of having to rebut all this nonsense coming from everyone I used to trust, drove me temporarily insane from stress and sleep deprivation ...
—which would have been the end of the story, _except that_, as I explained in a subsequent–subsequent post, "A Hill of Validity in Defense of Meaning", in late 2018, Eliezer Yudkowsky prevaricated about his own philosophy of language for the same political reasons, and my unsuccessful attempts to get him to clarify led me and allies to conclude that Yudkowsky and his "rationalists" were corrupt.
I believed that there _was_ a real problem, but didn't feel like I had a good grasp on what it was specifically. Cultural critique is a fraught endeavor: if someone tells an outright lie, you can, maybe, with a lot of effort, prove that to other people, and get a correction on that specific point. (Actually, as we had just discovered, even that might be too much to hope for.) But _culture_ is the sum of lots and lots of little micro-actions by lots and lots of people. If your _entire culture_ has visibly departed from the Way that was taught to you in the late 'aughts, how do you demonstrate that to people who, to all appearances, are acting like they don't remember the old Way, or that they don't think anything has changed, or that they notice some changes but think the new way is better? It's not as simple as shouting, "Hey guys, Truth matters!"—any ideologue or religious person would agree with _that_. It's not feasible to litigate every petty epistemic crime in something someone said, and if you tried, someone who thought the culture was basically on track could accuse you of cherry-picking. If "culture" is a real thing at all—and it certainly seems to be—we are condemned to grasp it unclearly, relying on the brain's pattern-matching faculties to sum over thousands of little micro-actions as a [_gestalt_](https://en.wiktionary.org/wiki/gestalt), rather than having the kind of robust, precise representation a well-designed AI could compute plans with.
-Ben called the _gestalt_ he saw the Blight, after the rogue superintelligence in Vernor Vinge's _A Fire Upon the Deep_: the problem wasn't that people were getting dumber; it's that there was locally coherent coordination away from clarity and truth and towards coalition-building, which was validated by the official narrative in ways that gave it a huge tactical advantage; people were increasingly making decisions that were better explained by their political incentives rather than acting on coherent beliefs about the world—using and construing claims about facts as moves in a power game, albeit sometimes subject to genre constraints under which only true facts were admissible moves in the game.
+Ben called the _gestalt_ he saw the Blight, after the rogue superintelligence in Vernor Vinge's _A Fire Upon the Deep_: the problem wasn't that people were getting dumber; it was that there was locally coherent coordination away from clarity and truth and towards coalition-building, which was validated by the official narrative in ways that gave it a huge tactical advantage; people were increasingly making decisions that were better explained by their political incentives rather than acting on coherent beliefs about the world—using and construing claims about facts as moves in a power game, albeit sometimes subject to genre constraints under which only true facts were admissible moves in the game.
-When I asked him for specific examples of MIRI or CfAR leaders behaving badly, he gave the example of [MIRI executive director Nate Soares posting that he was "excited to see OpenAI joining the space"](https://intelligence.org/2015/12/11/openai-and-other-news/), despite the fact that [_no one_ who had been following the AI risk discourse](https://slatestarcodex.com/2015/12/17/should-ai-be-open/) [thought that OpenAI as originally announced was a good idea](http://benjaminrosshoffman.com/openai-makes-humanity-less-safe/). Nate had privately clarified to Ben that the word "excited" wasn't necessarily meant positively, and in this case meant something more like "terrified."
+When I asked him for specific examples of MIRI or CfAR leaders behaving badly, he gave the example of [MIRI executive director Nate Soares posting that he was "excited to see OpenAI joining the space"](https://intelligence.org/2015/12/11/openai-and-other-news/), despite the fact that [_no one_ who had been following the AI risk discourse](https://slatestarcodex.com/2015/12/17/should-ai-be-open/) [thought that OpenAI as originally announced was a good idea](http://benjaminrosshoffman.com/openai-makes-humanity-less-safe/). Nate [had privately clarified that](https://twitter.com/jessi_cata/status/1462454555925434375) the word "excited" wasn't necessarily meant positively, and in this case meant something more like "terrified."
This seemed to me like the sort of thing where a particularly principled (naïve?) person might say, "That's _lying for political reasons!_ That's _contrary to the moral law!_" and most ordinary grown-ups would say, "Why are you so upset about this? That sort of strategic phrasing in press releases is just how the world works, and things could not possibly be otherwise."
[^soldiers]: At least, not blameworthy _in the same way_ as someone who committed the same violence as an individual.
-I wrote to Anna:
+I wrote to Anna (Subject: "Re: the end of the Category War (we lost?!?!?!)"):
-> To: Anna Salamon <[redacted]>
-> Date: 20 April 2019 11:08 _p.m._
-> Subject: Re: the end of the Category War (we lost?!?!?!)
->
> I was _just_ trying to publicly settle a _very straightforward_ philosophy thing that seemed _really solid_ to me
>
> if, in the process, I accidentally ended up being an unusually useful pawn in Michael Vassar's deranged four-dimensional hyperchess political scheming
And as it happened, on 4 May 2019, Yudkowsky [re-Tweeted Colin Wright on the "univariate fallacy"](https://twitter.com/ESYudkowsky/status/1124751630937681922)—the point that group differences aren't a matter of any single variable—which was _sort of_ like the clarification I had been asking for. (Empirically, it made me feel a lot less personally aggrieved.) Was I wrong to interpet this as another "concession" to me? (Again, notwithstanding that the whole mindset of extracting "concessions" was corrupt and not what our posse was trying to do.)
-Separately, I visited some friends' house on 30 April 2019 saying, essentially (and sincerely), "[Oh man oh jeez](https://www.youtube.com/watch?v=NivwAQ8sUYQ), Ben and Michael want me to join in a rationalist civil war against the corrupt mainstream-rationality establishment, and I'd really rather not, and I don't like how they keep using scary hyperbolic words like 'cult' and 'war' and 'criminal', but on the other hand, they're _the only ones backing me up_ on this _incredibly basic philosophy thing_ and I don't feel like I have anywhere else to _go_." The ensuing group conversation made some progress, but was mostly pretty horrifying.
+Separately, I visited some friends' house on 30 April 2019 saying, essentially (and sincerely), "Oh man oh jeez, Ben and Michael want me to join in a rationalist civil war against the corrupt mainstream-rationality establishment, and I'd really rather not, and I don't like how they keep using scary hyperbolic words like 'cult' and 'war' and 'criminal', but on the other hand, they're _the only ones backing me up_ on this _incredibly basic philosophy thing_ and I don't feel like I have anywhere else to _go_." The ensuing group conversation made some progress, but was mostly pretty horrifying.
In an adorable twist, my friends' two-year-old son was reportedly saying the next day that Kelsey doesn't like his daddy, which was confusing until it was figured out he had heard Kelsey talking about why she doesn't like Michael _Vassar_.
-And as it happened, on 7 May 2019, Kelsey wrote [a Facebook comment displaying evidence of understanding my point](https://www.facebook.com/julia.galef/posts/pfbid0QjdD8kWAZJMiczeLdMioqmPkRhewcmGtQpXRBu2ruXq8SkKvw5yvvSH2cWVDghWRl?comment_id=10104430041947222&reply_comment_id=10104430059182682).
+And as it happened, on 7 May 2019, Kelsey wrote [a Facebook comment displaying evidence of understanding my point](/images/piper-spending_social_capital_on_talking_about_trans_issues.png).
These two datapoints led me to a psychological hypothesis (which was maybe "obvious", but I hadn't thought about it before): when people see someone wavering between their coalition and a rival coalition, they're motivated to offer a few concessions to keep the wavering person on their side. Kelsey could _afford_ (_pace_ [Upton Sinclair](https://www.goodreads.com/quotes/21810-it-is-difficult-to-get-a-man-to-understand-something)) to not understand the thing about sex being a natural category ("I don't think 'people who'd get surgery to have the ideal female body' cuts anything at the joints"!!) when it was just me freaking out alone, but "got it" almost as soon as I could credibly threaten to _walk_ (defect to a coalition of people she dislikes) ... and maybe my "closing thoughts" email had a similar effect on Yudkowsky (assuming he otherwise wouldn't have spontaneously tweeted something about the univariate fallacy two weeks later)?? This probably wouldn't work if you repeated it (or tried to do it consciously)?
I decided to take a break from the religious civil war [and from this blog](/2019/May/hiatus/), and [declared May 2019 as Math and Wellness Month](http://zackmdavis.net/blog/2019/05/may-is-math-and-wellness-month/).
-My dayjob performance had been suffering terribly for months. The psychology of the workplace is ... subtle. There's a phenomenon where some people are _way_ more productive than others and everyone knows it, but no one is cruel enough [to make it _common_ knowledge](https://slatestarcodex.com/2015/10/15/it-was-you-who-made-my-blue-eyes-blue/), which is awkward for people who simultaneously benefit from the culture of common-knowledge-prevention allowing them to collect the status and money rents of being a $150K/yr software engineer without actually [performing at that level](http://zackmdavis.net/blog/2013/12/fortune/), while also having [read enough Ayn Rand as a teenager](/2017/Sep/neither-as-plea-nor-as-despair/) to be ideologically opposed to subsisting on unjustly-acquired rents rather than value creation. The "everyone knows I feel guilty about underperforming, so they don't punish me because I'm already doing enough internalized domination to punish myself" dynamic would be unsustainable if it were to evolve into a loop of "feeling gulit _in exchange for_ not doing work" rather than the intended "feeling guilt in order to successfully incentivize work". I didn't think they would actually fire me, but I was worried that they _should_. I asked my boss to temporarily take on some easier tasks, that I could make steady progress on even while being psychologically impaired from a religious war. (We had a lot of LaTeX templating of insurance policy amendments that needed to get done.) If I was going to be psychologically impaired _anyway_, it was better to be upfront about how I could best serve the company given that impairment, rather than hoping that the boss wouldn't notice.
+My dayjob performance had been suffering terribly for months. The psychology of the workplace is ... subtle. There's a phenomenon where some people are _way_ more productive than others and everyone knows it, but no one is cruel enough [to make it _common_ knowledge](https://slatestarcodex.com/2015/10/15/it-was-you-who-made-my-blue-eyes-blue/), which is awkward for people who simultaneously benefit from the culture of common-knowledge-prevention allowing them to collect the status and money rents of being a $150K/yr software engineer without actually [performing at that level](http://zackmdavis.net/blog/2013/12/fortune/), while also having [read enough Ayn Rand as a teenager](/2017/Sep/neither-as-plea-nor-as-despair/) to be ideologically opposed to subsisting on unjustly-acquired rents rather than value creation. The "everyone knows I feel guilty about underperforming, so they don't punish me because I'm already doing enough internalized domination to punish myself" dynamic would be unsustainable if it were to evolve into a loop of "feeling gulit _in exchange for_ not doing work" rather than the intended "feeling guilt in order to successfully incentivize work". I didn't think they would actually fire me, but I was worried that they _should_.
+
+I asked my boss to temporarily take on some easier tasks, that I could make steady progress on even while being psychologically impaired from a religious war. (We had a lot of LaTeX templating of insurance policy amendments that needed to get done.) If I was going to be psychologically impaired _anyway_, it was better to be upfront about how I could best serve the company given that impairment, rather than hoping that the boss wouldn't notice.
My "intent" to take a break from the religious war didn't take. I met with Anna on the UC Berkeley campus, and read her excerpts from some of Ben's and Jessica's emails. (She had not acquiesced to my request for a comment on "... Boundaries?", including in the form of two paper postcards that I stayed up until 2 _a.m._ on 14 April 2019 writing; I had figured that spamming people with hysterical and somewhat demanding physical postcards was more polite (and funnier) than my usual habit of spamming people with hysterical and somewhat demanding emails.) While we (my posse) were aghast at Yudkowsky's behavior, she was aghast at ours: reaching out to try to have a conversation with Yudkowsky, and then concluding he was a fraud because we weren't satisfied with the outcome was like hiding soldiers in an ambulance, introducing a threat against Yudkowsky in context where he had a right to be safe.
I complained that I had _actually believed_ our own marketing material about the "rationalists" remaking the world by wielding a hidden Bayesian structure of Science and Reason that applies [outside the laboratory](https://www.lesswrong.com/posts/N2pENnTPB75sfc9kb/outside-the-laboratory). Was that all a lie? Were we not trying to do the thing anymore? Anna was dismissive: she thought that the idea I had gotten about what "the thing" was, was never actually part of the original vision. She kept repeating that she had _tried_ to warn me in previous years that public reason didn't work, and I didn't listen. (Back in the late 'aughts, she had often recommended Paul Graham's essay ["What You Can't Say"](http://paulgraham.com/say.html) to people, summarizing Graham's moral that you should figure out the things you can't say in your culture, and then don't say them.)
-It was true that she had tried to warn me for years, and (not yet having gotten over [my teenage ideological fever dream](/2021/May/sexual-dimorphism-in-the-sequences-in-relation-to-my-gender-problems/#antisexism)), I hadn't known how to listen. But this seemed really fundamentally unresponsive to how _I_ kept repeating that I only expected consensus on the basic philosophy-of-language stuff (not my object-level special interest). Why was it so unrealistic to imagine that the actually-smart people could [enforce standards](https://srconstantin.github.io/2018/12/24/contrite-strategies-and-the-need-for-standards/) in our own tiny little bubble of the world?
+It was true that she had tried to warn me for years, and (not yet having gotten over [my teenage ideological fever dream](/2021/May/sexual-dimorphism-in-the-sequences-in-relation-to-my-gender-problems/#antisexism)), I hadn't known how to listen. But this seemed really fundamentally unresponsive to how _I_ kept repeating that I only expected consensus on the basic philosophy-of-language stuff (not my object-level special interest). Why was it so unrealistic to imagine that the actually-smart people could [enforce standards](https://srconstantin.github.io/2018/12/24/contrite-strategies.html) in our own tiny little bubble of the world?
My frustration bubbled out into follow-up emails:
Anna said she didn't want to receive monetary offers from me anymore; previously, she had regarded my custom of throwing money at people to get what I wanted as good-faith libertarianism between consenting adults, but now she was afraid that if she accepted, it would be portrayed in some future Ben Hoffman essay as an instance of her _using_ me. She agreed that someone could have gotten the ideals I had gotten out of "A Sense That More Is Possible", "Raising the Sanity Waterline", _&c._, but there was also evidence from that time pointing the other way (_e.g._, ["Politics Is the Mind-Killer"](https://www.lesswrong.com/posts/9weLK2AJ9JEt2Tt8f/politics-is-the-mind-killer)), that it shouldn't be surprising if people steered clear of controversy.
-I replied: but when forming the original let's-be-apolitical vision in 2008, we did not anticipate that _whether or not I should cut my dick off_ would _become_ a political issue. That was _new evidence_ about whether the original vision was wise! I wasn't trying to do politics with my idiosyncratic special interest; I was trying to _think seriously_ about the most important thing in my life and only do the minimum amount of politics necessary to protect my ability to think. If 2019-era "rationalists" were going to commit a trivial epistemology mistake that interfered with my ability to think seriously about the most important thing in my life, but couldn't correct the mistake, then the 2019-era "rationalists" were _worse than useless_ to me personally. This probably didn't matter causally (I wasn't an AI researcher, therefore I didn't matter), but it might matter timelessly (if I was part of a reference class that includes AI researchers).
+I replied: but when forming the original let's-be-apolitical vision in 2008, we did not anticipate that _whether or not I should cut my dick off_ would _become_ a political issue. That was _new evidence_ about whether the original vision was wise! I wasn't trying to do politics with my idiosyncratic special interest; I was trying to _think seriously_ about the most important thing in my life and only do the minimum amount of politics necessary to protect my ability to think. If 2019-era "rationalists" were going to commit a trivial epistemology mistake that interfered with my ability to think seriously about the most important thing in my life, but couldn't correct the mistake even after it was pointed out, then the 2019-era "rationalists" were _worse than useless_ to me personally. This probably didn't matter causally (I wasn't an AI researcher, therefore I didn't matter), but it might matter timelessly (if I was part of a reference class that includes AI researchers).
Fundamentally, I was skeptical that you _could_ do consisently high-grade reasoning as a group without committing heresy, because of the mechanism that Yudkowsky described in ["Entangled Truths, Contagious Lies"](https://www.lesswrong.com/posts/wyyfFfaRar2jEdeQK/entangled-truths-contagious-lies) and ["Dark Side Epistemology"](https://www.lesswrong.com/posts/XTWkjCJScy2GFAgDt/dark-side-epistemology): the need to lie about lying and cover up cover-ups propagates recursively. Anna in particular was unusually skillful at thinking things without saying them; I thought most people facing similar speech restrictions just get worse at thinking (plausibly[^plausibly] including Yudkowsky), and the problem gets worse as the group effort scales. (It's easier to recommend ["What You Can't Say"](http://www.paulgraham.com/say.html) to your housemates than to put it on a canonical reading list, for obvious reasons.) You _can't_ optimize your group's culture for not-talking-about-atheism without also optimizing against understanding [Occam's razor](https://www.lesswrong.com/posts/f4txACqDWithRi7hs/occam-s-razor); you _can't_ optimize for not questioning gender self-identity without also optimizing against understanding the [37 ways that words can be wrong](https://www.lesswrong.com/posts/FaJaCgqBKphrDzDSj/37-ways-that-words-can-be-wrong).
Despite Math and Wellness Month and my "intent" to take a break from the religious civil war, I kept reading _Less Wrong_ during May 2019, and ended up scoring a couple of victories in the civil war (at some cost to Wellness).
-MIRI researcher Scott Garrabrant wrote a post about how ["Yes Requires the Possibility of No"](https://www.lesswrong.com/posts/G5TwJ9BGxcgh5DsmQ/yes-requires-the-possibility-of-no). Information-theoretically, a signal sent with probability one transmits no information: you can only learn something from hearing a "Yes" if there was some chance that the answer could have been "No". I saw an analogy to my philosophy-of-language thesis, and commented about it: if you want to believe that _x_ belongs to category _C_, you might try redefining _C_ in order to make the question "Is _x_ a _C_?" come out "Yes", but you can only do so at the expense of making _C_ less useful. Meaningful category-membership (Yes) requires the possibility of non-membership (No).
+MIRI researcher Scott Garrabrant wrote a post about how ["Yes Requires the Possibility of No"](https://www.lesswrong.com/posts/G5TwJ9BGxcgh5DsmQ/yes-requires-the-possibility-of-no). Information-theoretically, a signal sent with probability one transmits no information: you can only learn something from hearing a "Yes" if believed that the answer could have been "No". I saw an analogy to my philosophy-of-language thesis, and mentioned it in a comment: if you want to believe that _x_ belongs to category _C_, you might try redefining _C_ in order to make the question "Is _x_ a _C_?" come out "Yes", but you can only do so at the expense of making _C_ less useful. Meaningful category-membership (Yes) requires the possibility of non-membership (No).
-[TODO: explain scuffle on "Yes Requires the Possibility"—
+MIRI research associate Vanessa Kosoy [commented](https://www.lesswrong.com/posts/WwTPSkNwC89g3Afnd/comment-section-from-05-19-2019?commentId=FxSZwECjhgYE7p2du):
- * Vanessa comment on hobbyhorses and feeling attacked
- * my reply about philosophy got politicized, and MDL/atheism analogy
- * Ben vs. Said on political speech and meta-attacks; Goldenberg on feelings
- * 139-comment trainwreck got so bad, the mods manually moved the comments into their own thread https://www.lesswrong.com/posts/WwTPSkNwC89g3Afnd/comment-section-from-05-19-2019
- * based on the karma scores and what was said, this went pretty well for me and I count it as a victory
-
-]
+> I find it unpleasant that you always bring your hobbyhorse in, but in an "abstract" way that doesn't allow discussing the actual object level question. It makes me feel attacked in a way that allows for no legal recourse to defend myself.
+
+I [replied](https://www.lesswrong.com/posts/WwTPSkNwC89g3Afnd/comment-section-from-05-19-2019?commentId=32GPaijsSwX2NSFJi) that that was understandable, but that I hoped it was also understandable that I found it unpleasant that our standard Bayesian philosophy of language somehow got politicized (!?), such that my attempts to do _correct epistemology_ were perceived as attacking people. Imagine living in a world where [posts about the minimum description length principle](https://www.lesswrong.com/posts/mB95aqTSJLNR9YyjH/message-length) were perceived as an attack on Christians—or if that analogy seemed loaded (because our subculture pattern matches atheism as "the good guys"), imagine some racist getting _really interested_ in the statistics of the normal distribution, and posting about the ratio of areas in the right tails of normal distributions with different means. I could see how that would be annoying—maybe even threatening—which would make it all the more satisfying if you could find a _mistake_ in the bastard's math. But if you _couldn't_ find a mistake—if, in fact, the post is on-topic for the forum and correct in the literal things that it literally says, then complaining about the author's motive for being interested in the normal distribution wouldn't seem like an obviously positive contribution to the discourse? I saw the problem, of course, and didn't mean to play dumb about it. But what, realistically, did Kosoy expect the atheist—or the racist, or me—to do?
+
+In a subthread in which I contested Kosoy's characterization of me as a "voice with an agenda which, if implemented, would put [her] in physical danger" ("I don't think of myself as having a lot of strong political beliefs," I said, "but I'm going to take a definite stand here: I am _against_ people being in physical danger"), Ben [pointed out that](https://www.lesswrong.com/posts/WwTPSkNwC89g3Afnd/comment-section-from-05-19-2019?commentId=TXbgr7goFtSAZEvZb)
+
+> Some speech acts lower the message length of proposals to attack some groups, or raise the message length of attempts to prevent such attacks. This is a kind of meta-attack or threat, like concentrating troops on a country's border.
+
+Norms discouraging "political" speech could aggravate the problem, if the response looked "political" but the original threat didn't. If Kosoy wanted to put in the work to explain why my philosophy of language blogging was causing problems for her, she would face legitimate doubt whether her defensive measures would be "admissible".
+
+The trainwreck got so bad that the mods manually [moved the comments to their own post](https://www.lesswrong.com/posts/WwTPSkNwC89g3Afnd/comment-section-from-05-19-2019). Based on the karma scores and what was said (Said Achmiz gave [a particularly helpful defense of disregarding community members' feelings](https://www.lesswrong.com/posts/WwTPSkNwC89g3Afnd/comment-section-from-05-19-2019?commentId=EsSdLMrFcCpSvr3pG)), I count it as a "victory" for me.
On 31 May 2019, a [draft of a new _Less Wrong_ FAQ](https://www.lesswrong.com/posts/MqrzczdGhQCRePgqN/feedback-requested-draft-of-a-new-about-welcome-page-for) included a link to "... Not Man for the Categories" as one of Scott Alexander's best essays. I argued that it would be better to cite _almost literally_ any other _Slate Star Codex_ post (most of which, I agreed, were exemplary). I claimed that the following disjunction was true: _either_ Alexander's claim that "There's no rule of rationality saying that [one] shouldn't" "accept an unexpected [X] or two deep inside the conceptual boundaries of what would normally be considered [Y] if it'll save someone's life" was a blatant lie, _or_ one had no grounds to criticize me for calling it a blatant lie, because there's no rule of rationality that says I shouldn't draw the category boundaries of "blatant lie" that way. The mod [was persuaded on reflection](https://www.lesswrong.com/posts/MqrzczdGhQCRePgqN/feedback-requested-draft-of-a-new-about-welcome-page-for?commentId=oBDjhXgY5XtugvtLT), and "... Not Man for the Categories" was not included in the final FAQ. Another "victory."
-But winning "victories" wasn't particularly comforting when I resented this becoming a political slapfight at all. I thought a lot of the objections I faced in the derailed "Possibility of No" thread were insane.
+But winning "victories" wasn't particularly comforting when I resented this becoming a political slapfight at all. I thought a lot of the objections I (and my "allies") faced in the derailed "Possibility of No" thread were insane.
I wrote to Anna and Steven Kaas (who I was trying to "recruit" onto our side of the civil war). In ["What You Can't Say"](http://www.paulgraham.com/say.html), Paul Graham had written, "The problem is, there are so many things you can't say. If you said them all you'd have no time left for your real work." But surely that depends on what _is_ one's real work. For someone like Paul Graham, whose goal was to make a lot of money writing software, "Don't say it" (except for this one meta-level essay) was probably the right choice. But someone whose goal is to improve our collective ability to reason, should probably be doing _more_ fighting than Paul Graham (although still preferably on the meta- rather than object-level), because political restrictions on speech and thought directly hurt the mission of "improving our collective ability to reason", in a way that they don't hurt the mission of "make a lot of money writing software."
I agreed that tractability needs to be addressed, but I felt like—we were in a coal mine, and my favorite one of our canaries just died, and I was freaking out about this, and represenatives of the Caliphate (Yudkowsky, Alexander, Anna, Steven) were like, Sorry, I know you were really attached to that canary, but it's just a bird; you'll get over it; it's not really that important to the coal-mining mission.
-And I was like, I agree that I was unreasonably emotionally attached to that particular bird, which was the direct cause of why I-in-particular was freaking out, but that's not why I expected _them_ to care. The problem was not the dead bird; the problem was what the bird was _evidence_ of: if you're doing systematically correct reasoning, you should be able to get the right answer even when the question _doesn't matter_. (The causal graph is the fork "canary-death ← mine-gas → human-danger" rather than the direct link "canary-death → human-danger".) Ben and Michael and Jessica claim to have spotted their own dead canaries. I felt like the old-timer Rationality Elders should be able to get on the same page about the canary-count issue?
+And I was like, I agree that I was unreasonably emotionally attached to that particular bird, which was the direct cause of why I-in-particular was freaking out, but that's not why I expected _them_ to care. The problem was not the dead bird; the problem was what the bird was _evidence_ of: if you're doing systematically correct reasoning, you should be able to get the right answer even when the question _doesn't matter_. (The causal graph is the fork "canary-death ← mine-gas → human-danger" rather than the direct link "canary-death → human-danger".) Ben and Michael and Jessica claimed to have spotted their own dead canaries. I felt like the old-timer Rationality Elders should have been able to get on the same page about the canary-count issue?
-Math and Wellness Month ended up being mostly a failure: the only math I ended up learning was [a fragment of group theory](http://zackmdavis.net/blog/2019/05/group-theory-for-wellness-i/), and [some probability/information theory](http://zackmdavis.net/blog/2019/05/the-typical-set/) that [actually turned out to super-relevant to understanding sex differences](/2021/May/sexual-dimorphism-in-the-sequences-in-relation-to-my-gender-problems/#typical-point). So much for taking a break.
+Math and Wellness Month ended up being mostly a failure: the only math I ended up learning was [a fragment of group theory](http://zackmdavis.net/blog/2019/05/group-theory-for-wellness-i/), and [some probability/information theory](http://zackmdavis.net/blog/2019/05/the-typical-set/) that [later turned out to super-relevant to understanding sex differences](/2021/May/sexual-dimorphism-in-the-sequences-in-relation-to-my-gender-problems/#typical-point). So much for taking a break.
-[TODO:
- * I had posted a linkpost to "No, it's not The Incentives—it's You", which generated a lot of discussion, and Jessica (17 June) identified Ray's comments as the last straw.
+In June 2019, I made [a linkpost on _Less Wrong_](https://www.lesswrong.com/posts/5nH5Qtax9ae8CQjZ9/tal-yarkoni-no-it-s-not-the-incentives-it-s-you) to Tal Yarkoni's ["No, It's Not The Incentives—It's you"](https://www.talyarkoni.org/blog/2018/10/02/no-its-not-the-incentives-its-you/), about how professional scientists should stop using career incentives as an excuse for doing poor science. It generated a lot of discussion.
+
+Looking over the thread in retrospect, [these words from David Xu seem significant](https://www.lesswrong.com/posts/5nH5Qtax9ae8CQjZ9/tal-yarkoni-no-it-s-not-the-incentives-it-s-you?commentId=qDPdneAQ4s7HMt3ys):
+
+> _We all know that falsifying data is bad._ But if that's the way the incentives point (and that's a very important if!), then it's _also_ bad to call people out for doing it. If you do that, then you're using moral indignation as a weapon—a way to not only coerce other people into using up their willpower, but to come out of it looking good yourself.
+
+In an email (Subject: "LessWrong.com is dead to me"), Jessica identified the thread as her last straw:
> LessWrong.com is a place where, if the value of truth conflicts with the value of protecting elites' feelings and covering their asses, the second value will win.
>
-> Trying to get LessWrong.com to adopt high-integrity norms is going to fail, hard, without a _lot_ of conflict. (Enforcing high-integrity norms is like violence; if it doesn't work, you're not doing enough of it).
+> Trying to get LessWrong.com to adopt high-integrity norms is going to fail, hard, without a _lot_ of conflict. (Enforcing high-integrity norms is like violence; if it doesn't work, you're not doing enough of it). People who think being exposed as fraudulent (or having their friends exposed as fraudulent) is a terrible outcome, are going to actively resist high-integrity discussion norms.
+
+Posting on _Less Wrong_ made sense as harm-reduction, but the only way to get people to stick up for truth would be to convert them to _a whole new worldview_, which would require a lot of in-person discussions. She bought up the idea of starting a new forum to replace _Less Wrong_.
+
+Ben said that trying to discuss with the _Less Wrong_ mod team would be a good intermediate step, after we clarified to ourselves what was going on; it might be "good practice in the same way that the Eliezer initiative was good practice." He was less optimistic about harm-reduction; participating on the site was implicitly endorsing it by submitting the rule of the karma and curation systems.
+
+Secret posse member expressed sadness about how the discussion on "The Incentives" demonstrated that the community he loved—including dear friends—was in a very bad way. Michael (in a separate private discussion) had said he was glad to hear about the belief-update. Secret posse member said that Michael saying that also made them sad, because it seemed discordant to be happy about sad news. Michael wrote (in the thread):
+
+> I['m] sorry it made you sad. From my perspective, the question is no[t] "can we still be friends with such people", but "how can we still be friends with such people" and I am pretty certain that understanding their perspective if an important part of the answer. If clarity seems like death to them and like life to us, and we don't know this, IMHO that's an unpromising basis for friendship.
- * posting on Less Wrong was harm-reduction; the only way to get people to stick up for truth would be to convert them to _a whole new worldview_; Jessica proposed the idea of a new discussion forum
- * Ben thought that trying to discuss with the other mods would be a good intermediate step, after we clarified to ourselves what was going on; talking to other mods might be "good practice in the same way that the Eliezer initiative was good practice"; Ben is less optimistic about harm reduction; "Drowning Children Are Rare" was barely net-upvoted, and participating was endorsing the karma and curation systems
- * David Xu's comment on "The Incentives" seems important?
- * secret posse member: Ray's attitude on "Is being good costly?"
+[TODO—
* Jessica: scortched-earth campaign should mostly be in meatspace social reality
* my comment on emotive conjugation (https://www.lesswrong.com/posts/qaYeQnSYotCHQcPh8/drowning-children-are-rare#GaoyhEbzPJvv6sfZX)
* secret posse member: level of social-justice talk makes me not want to interact with this post in any way
]
-On 4 July, Scott Alexander published ["Some Clarifications on Rationalist Blogging"](https://slatestarcodex.com/2019/07/04/some-clarifications-on-rationalist-blogging/), disclaiming any authority as a "rationalist" leader. ("I don't want to claim this blog is doing any kind of special 'rationality' work beyond showing people interesting problems [...] Insofar as [_Slate Star Codex_] makes any pretensions to being 'rationalist', it's a rationalist picnic and not a rationalist monastery.") I assumed this was inspired by Ben's request back in March that Scott "alter the beacon" so as to not confuse people about what the current-year community was. I appreciated it.
+On 4 July 2019, Scott Alexander published ["Some Clarifications on Rationalist Blogging"](https://slatestarcodex.com/2019/07/04/some-clarifications-on-rationalist-blogging/), disclaiming any authority as a "rationalist" leader. ("I don't want to claim this blog is doing any kind of special 'rationality' work beyond showing people interesting problems [...] Insofar as [_Slate Star Codex_] makes any pretensions to being 'rationalist', it's a rationalist picnic and not a rationalist monastery.") I assumed this was inspired by Ben's request back in March that Scott "alter the beacon" so as to not confuse people about what the current-year community was. I appreciated it.
-[TODO: "AI Timelines Scam"
- * I still sympathize with the "mainstream" pushback against the scam/fraud/&c. language being used to include Elephant-in-the-Brain-like distortions
- * Ben: "What exactly is a scam, if it's not misinforming people systematically about what you have to offer, in a direction that moves resources towards you? Investigations of financial fraud don't inquire as to the conscious motives of the perp."
- * 11 Jul: I think the law does count _mens rea_ as a thing: we do discriminate between vehicular manslaughter and first-degree murder, because traffic accidents are less disincentivizable than offing one's enemies
- * call with Michael about GiveWell vs. the Pope
-]
+Also in early July 2019, Jessica published ["The AI Timelines Scam"](https://www.lesswrong.com/posts/KnQs55tjxWopCzKsk/the-ai-timelines-scam), arguing that the recent popularity of "short" (_e.g._, 2030) AI timelines was better explained by political factors, rather than any technical arguments: just as in previous decades, people had incentives to bluff and exaggerate about the imminence of AGI in order to attract resources to their own project.
+
+(Remember, this was 2019. After seeing what GPT-3/PaLM/DALL-E/_&c._ could do during the "long May 2020", it's now looking to me like the short-timelines people had better intuitions than Jessica gave them credit for. I won't say, "How could I have known?", but at the time, I didn't, actually, know.)
+
+I still sympathized with the "mainstream" pushback against using "scam"/"fraud"/"lie"/_&c._ language to include motivated [elephant-in-the-brain](https://en.wikipedia.org/wiki/The_Elephant_in_the_Brain)-like distortions. I conceded that this was a _boring_ semantic argument, but I feared that until we invented better linguistic technology, the _boring_ semantic argument was going to _continue_ sucking up discussion bandwidth with others when it didn't need to.
+
+"Am I being too tone-policey here?" I asked the coordination group. "Is it better if I explicitly disclaim, 'This is marketing advice; I'm not claiming to be making a substantive argument'?" (Subject: "Re: reception of 'The AI Timelines Scam' is better than expected!")
+
+Ben replied, "What exactly is a scam, if it's not misinforming people systematically about what you have to offer, in a direction that moves resources towards you? Investigations of financial fraud don't inquire as to the conscious motives of the perp."
+
+I replied that the concept of [_mens rea_](https://www.law.cornell.edu/wex/mens_rea) did seem necessary for maintaining good incentives, at least in some contexts. The law needs to distinguish between accidentally hitting a pedestrian in one's car ("manslaughter") from premeditated killing ("first-degree murder"), because traffic accidents are significantly less disincentivizable than offing one's enemies. (Anyone who drives at all is probably going to have unlucky analogues in nearby possible worlds who are guilty of vehicular manslaughter.) The manslaughter case was simpler than misinformation-that-moves-resources,[^manslaughter-disanalogy] and it might not be _easy_ for the court to determine "intent", but I didn't see what would reverse the extremely weak principle that intent matters, sometimes.
+
+[^manslaughter-disanalogy]: For one extremely important disanalogy, perps don't _gain_ from committing manslaughter.
+
+Ben replied that what mattered in the determination of manslaughter _vs._ murder was whether there was long-horizon optimization power towards the outcome of someone's death; it wasn't about what sentiments the killer rehearsed in their working memory.
+
+On a phone call, Michael made an analogy between EA and Catholicism. The Pope was fraudulent, because the legitimacy of the Pope's position (and his claims to power and resources) rested on the pretense that he had a direct relationship with God, which wasn't true, and the Pope _knew_ (he _had_ to know, at some level) that it asn't true. (I agreed that this usage of _fraud_ made sense to me.) In Michael's view, Ben's charges against GiveWell were similar: GiveWell's legitimacy rested on the pretense that they were making decisions based on numbers, and they had to know at some level that they weren't doing that.
[TODO: secret thread with Ruby; "uh, guys??" to Steven and Anna; people say "Yes, of course criticism and truthseeking is important; I just think that tact is important, too," only to go on and dismiss any _particular_ criticism as insufficiently tactful.]
In August 2019's ["Schelling Categories, and Simple Membership Tests"](https://www.lesswrong.com/posts/edEXi4SpkXfvaX42j/schelling-categories-and-simple-membership-tests), I explained a nuance that had only merited a passion mention in "... Boundaries?": sometimes you might want categories for different agents to _coordinate_ on, even at the cost of some statistical "fit." (This was of course generalized from a "pro-trans" argument that had occured to me, [that self-identity is an easy Schelling point when different people disagree about what "gender" they perceive someone as](/2019/Oct/self-identity-is-a-schelling-point/).)
-In September 2019's ["Heads I Win, Tails?—Never Heard of Her; Or, Selective Reporting and the Tragedy of the Green Rationalists"](https://www.lesswrong.com/posts/DoPo4PDjgSySquHX8/heads-i-win-tails-never-heard-of-her-or-selective-reporting), I presented a toy mathematical model of how censorship distorts group beliefs. I was surprised by how well-received it was (high karma, Curated within a few days, later included in the Best-of-2019 collection), especially given that it was explicitly about politics (albeit at a meta level, of course). Ben and Jessica had discouraged me from bothering when I sent them a draft.
+In September 2019's ["Heads I Win, Tails?—Never Heard of Her; Or, Selective Reporting and the Tragedy of the Green Rationalists"](https://www.lesswrong.com/posts/DoPo4PDjgSySquHX8/heads-i-win-tails-never-heard-of-her-or-selective-reporting), I presented a toy mathematical model of how censorship distorts group beliefs. I was surprised by how well-received it was (high karma, Curated within a few days, later included in the Best-of-2019 collection), especially given that it was explicitly about politics (albeit at a meta level, of course). Ben and Jessica had discouraged me from bothering when I sent them a draft. (Jessica said that it was obvious even to 10-year-olds that partisan politics distorts impressions by filtering evidence. "[D]o you think we could get a ten-year-old to explain it to Eliezer Yudkowsky?" I asked.)
In October 2019's ["Algorithms of Deception!"](https://www.lesswrong.com/posts/fmA2GJwZzYtkrAKYJ/algorithms-of-deception), I exhibited some toy Python code modeling different kinds of deception. A function that faithfully passes observations it sees as input to another function, lets the second function constructing a well-calibrated probability distribution. But if the first function outright fabricates evidence, or selectively omits some evidence, or gerrymanders the categories by which it interprets its observations as evidence, the second function comes up with a worse (less accurate) probability distribution.
-Also in October 2019, in ["Maybe Lying Doesn't Exist"](https://www.lesswrong.com/posts/bSmgPNS6MTJsunTzS/maybe-lying-doesn-t-exist), I replied to Scott Alexander's ["Against Lie Inflation"](https://slatestarcodex.com/2019/07/16/against-lie-inflation/), which was itself a generalized rebuke of Jessica's "The AI Timelines Scam". Scott thought Jessica was wrong to use language like "lie", "scam", _&c._ to describe someone thing (purportedly) motivatedly wrong, but not necessarily _consciously_ lying.
+Also in October 2019, in ["Maybe Lying Doesn't Exist"](https://www.lesswrong.com/posts/bSmgPNS6MTJsunTzS/maybe-lying-doesn-t-exist), I replied to Scott Alexander's ["Against Lie Inflation"](https://slatestarcodex.com/2019/07/16/against-lie-inflation/), which was itself a generalized rebuke of Jessica's "The AI Timelines Scam". Scott thought Jessica was wrong to use language like "lie", "scam", _&c._ to describe someone being (purportedly) motivatedly wrong, but not necessarily _consciously_ lying.
+
+I was _furious_ when "Against Lie Inflation" came out. (Furious at what I perceived as hypocrisy, not because I particularly cared about defending Jessica's usage.) Oh, so _now_ Scott agreed that making language less useful is a problem?! But on further consideration, I realized Alexander actually was being consistent in admitting appeals-to-consequences as legitimate. In objecting to the expanded definition of "lying", Alexander was counting "everyone is angrier" (because of more frequent lying-accusations) as a cost. Whereas on my philosophy, that wasn't a legitimate cost. (If everyone _is_ lying, maybe people _should_ be angry!)
+
+-----
+
+While visiting Valinor on 7 August 2019, Merlin Blume (age 2¾ years) asked me, "Why are you a boy?"
+
+After a long pause, I said, "Yes," as if I had misheard the question as "Are you a boy?" I think it was a motivated mishearing: it was only after I answered that I consciously realized that's not what the kid asked.
-I was _furious_ when "Against Lie Inflation" came out. (Furious at what I perceived as hypocrisy, not because I particularly cared about defending Jessica's usage.) Oh, so _now_ Scott agreed that making language less useful is a problem?! But on further consideration, I real I realized Alexander actually was being consistent in admitting appeals-to-consequences as legitimate. In objecting to the expanded definition of "lying", Alexander was counting "everyone is angrier" (because of more frequent lying-accusations) as a cost. Whereas on my philosophy, that wasn't a legitimate cost. (If everyone _is_ lying, maybe people _should_ be angry!)
+I think I would have preferred to say, "Because I have a penis, like you." But it didn't seem appropriate.
------
Scott (and Yudkowsky and Anna and the rest of the Caliphate) seemed to accept this as an inevitable background fact of existence, like the weather. But I saw a Schelling point off in the distance where us witches stick together for Free Speech, and it was _awfully_ tempting to try to jump there. (Of course, it would be _better_ if there was a way to organize just the good witches, and exclude all the Actually Bad witches, but the [Sorites problem](https://plato.stanford.edu/entries/sorites-paradox/) on witch Badness made that hard to organize without falling back to the falling back to the one-heresy-per-thinker equilibrium.)
-Jessica thought my use of "heresy" was conflating factual beliefs with political movements. (There are no intrinsically "right wing" _facts_.) I agreed that conflating political positions with facts would be bad (and that it would be bad if I were doing that without "intending" to). I wasn't interested in defending the "alt-right" (whatever that means) broadly. But I had _learned stuff_ from reading far-right authors (most notably Moldbug), and from talking with my very smart neoreactionary (and former _Less Wrong_-er) friend. I was starting to appreciate [what Michael had said about "Less precise is more violent" back in April](#less-precise-is-more-violent) (when I was talking about criticizing "rationalists").
+Jessica thought my use of "heresy" was conflating factual beliefs with political movements. (There are no intrinsically "right wing" _facts_.) I agreed that conflating political positions with facts would be bad (and that it would be bad if I were doing that without "intending" to). I wasn't interested in defending the "alt-right" (whatever that means) broadly. But I had _learned stuff_ from reading far-right authors (most notably Moldbug), and from talking with "Wilhelm". I was starting to appreciate [what Michael had said about "Less precise is more violent" back in April](#less-precise-is-more-violent) (when I was talking about criticizing "rationalists").
Jessica asked if my opinion would change depending on whether Yudkowsky thought neoreaction was intellectually worth engaging with. (Yudkowsky [had said years ago](https://www.lesswrong.com/posts/6qPextf9KyWLFJ53j/why-is-mencius-moldbug-so-popular-on-less-wrong-answer-he-s?commentId=TcLhiMk8BTp4vN3Zs) that Moldbug was low quality.)
My "negotiating with terrorists" criticism did _not_ apply to the 2013 statement. "More Right" _was_ brand encroachment on Anissimov's part that Yudkowsky had a legitimate interest in policing, _and_ the "I try not to dismiss ideas out of hand" disclaimer importantly avoided legitimizing [the McCarthyist persecution](https://www.unqualified-reservations.org/2013/09/technology-communism-and-brown-scare/).
-The question was, what had specifically happened in the last six years to shift Eliezer's opinion on neoreaction from (paraphrased) "Scott says it's wrong, so I stopped reading" to (verbatim) "actively hostile"? Note especially the inversion from (both paraphrased) "I don't support neoreaction" (fine, of course) to "I don't even want _them_ supporting _me_" [(_?!?!_)](https://twitter.com/zackmdavis/status/1164329446314135552).[^them-supporting-me]
+The question was, what had specifically happened in the last six years to shift Eliezer's opinion on neoreaction from (paraphrased) "Scott says it's wrong, so I stopped reading" to (verbatim) "actively hostile"? Note especially the inversion from (both paraphrased) "I don't support neoreaction" (fine, of course) to "I don't even want _them_ supporting _me_" ([**?!?!**](https://twitter.com/zackmdavis/status/1164329446314135552)—humans with very different views on politics nevertheless have a common interest in not being transformed into paperclips).
-[^them-supporting-me]: Humans with very different views on politics nevertheless have a common interest in not being transformed into paperclips!
+Did Yudkowsky get _new information_ about neoreaction's hidden Badness parameter sometime between 2013 and 2019, or did moral coercion on him from the left intensify (because Trump and [because Berkeley](https://thezvi.wordpress.com/2017/08/12/what-is-rationalist-berkleys-community-culture/))? My bet was on the latter.
-Did Yudkowsky get _new information_ about neoreaction's hidden Badness parameter sometime between 2019, or did moral coercion on him from the left intensify (because Trump and [because Berkeley](https://thezvi.wordpress.com/2017/08/12/what-is-rationalist-berkleys-community-culture/))? My bet was on the latter.
+------
-However it happened, it didn't seem like the brain damage was limited to "political" topics, either. In November, we saw another example of Yudkowsky destroying language for the sake of politeness, this time the non-Culture-War context of him [_trying to wirehead his fiction subreddit by suppressing criticism-in-general_](https://www.reddit.com/r/rational/comments/dvkv41/meta_reducing_negativity_on_rrational/).
+However it happened, it didn't seem like the brain damage was limited to "political" topics, either. In November 2019, we saw another example of Yudkowsky destroying language for the sake of politeness, this time the non-Culture-War context of him [_trying to wirehead his fiction subreddit by suppressing criticism-in-general_](https://www.reddit.com/r/rational/comments/dvkv41/meta_reducing_negativity_on_rrational/).
That's _my_ characterization, of course: the post itself talks about "reducing negativity". [In a followup comment, Yudkowsky wrote](https://www.reddit.com/r/rational/comments/dvkv41/meta_reducing_negativity_on_rrational/f7fs88l/) (bolding mine):
[^communism-analogy]: That is, there's an analogy between economically valuable labor, and intellectually productive criticism: if you accept the necessity of paying workers money in order to get good labor out of them, you should understand the necessity of awarding commenters status in order to get good criticism out of them.
-There's a striking contrast between the Yudkowsky of 2019 who wrote the "Reducing Negativity" post, and an earlier Yudkowsky (from even before the Sequences) who maintained [a page on Crocker's rules](http://sl4.org/crocker.html): if you declare that you operate under Crocker's rules, you're consenting to other people optimizing their speech for conveying information rather than being nice to you. If someone calls you an idiot, that's not an "insult"; they're just informing you about the fact that you're an idiot, and you should plausibly thank them for the tip. (If you _were_ an idiot, wouldn't you be better off knowing rather than not-knowing?)
+There's a striking contrast between the Yudkowsky of 2019 who wrote the "Reducing Negativity" post, and an earlier Yudkowsky (from even before the Sequences) who maintained [a page on Crocker's rules](http://sl4.org/crocker.html): if you declare that you operate under Crocker's rules, you're consenting to other people optimizing their speech for conveying information rather than being nice to you. If someone calls you an idiot, that's not an "insult"; they're just informing you about the fact that you're an idiot, and you should probably thank them for the tip. (If you _were_ an idiot, wouldn't you be better off knowing rather than not-knowing?)
It's of course important to stress that Crocker's rules are _opt in_ on the part of the _receiver_; it's not a license to unilaterally be rude to other people. Adopting Crocker's rules as a community-level norm on an open web forum does not seem like it would end well.
Still, there's something precious about a culture where people appreciate the _obvious normative ideal_ underlying Crocker's rules, even if social animals can't reliably live up to the normative ideal. Speech is for conveying information. People can say things—even things about me or my work—not as a command, or as a reward or punishment, but just to establish a correspondence between words and the world: a map that reflects a territory.
-Appreciation of this obvious normative ideal seems almost entirely absent from Yudkowsky's modern work—as if he's given up on the idea that using Speech in public in order to reason is useful or possible.
+Appreciation of this obvious normative ideal seems strikingly absent from Yudkowsky's modern work—as if he's given up on the idea that using Speech in public in order to reason is useful or possible.
The "Reducing Negativity" post also warns against the failure mode of attempted "author telepathy": _attributing_ bad motives to authors and treating those attributions as fact without accounting for uncertainty or distinguishing observations from inferences. I should be explicit, then: when I say negative things about Yudkowsky's state of mind, like it's "as if he's given up on the idea that reasoning in public is useful or possible", that's definitely an inference, not an observation. I definitely don't think Yudkowsky _thinks of himself_ as having given up on Speech _in those words_.
-----
-While visiting Valinor one evening in September 2019, Merlin Blume (almost age 3) asked me, "Why are you a boy?"
+On 3 November 2019, I received an interesting reply on my philosophy-of-categorization thesis from MIRI researcher Abram Demski. Abram asked: ideally, shouldn't all conceptual boundaries be drawn with appeal-to-consequences? Wasn't the problem just with bad (motivated, shortsighted) appeals to consequences? Agents categorize in order to make decisions. The best classifer for an application depends on the costs and benefits. As a classic example, it's very important for evolved prey animals to avoid predators, so it makes sense for their predator-detection classifiers to be configured such that they jump away from every rustling in the bushes, even if it's usually not a predator.
-After a long pause, I said, "Yes," as if I had misheard the question as "Are you a boy?"
+I had thought of the "false-positives are better than false-negatives when detecting predators" example as being about the limitations of evolution as an AI designer: messy evolved animal brains don't bother to track probability and utility separately the way a cleanly-designed AI could. As I had explained in "... Boundaries?", it made sense for _what_ variables you paid attention to, to be motivated by consequences. But _given_ the subspace that's relevant to your interests, you want to run an "epistemically legitimate" clustering algorithm on the data you see there, which depends on the data, not your values. The only reason value-dependent gerrymandered category boundaries seem like a good idea if you're not careful about philosophy is because it's _wireheading_. Ideal probabilistic beliefs shouldn't depend on consequences.
-I think I would have preferred to say, "Because I have a penis, like you." But it didn't seem appropriate.
+Abram didn't think the issue was so clear-cut. Where do "probabilities" come from, in the first place? The reason we expect something like Bayesianism to be an attractor among self-improving agents is because probabilistic reasoning is broadly useful: epistemology can be _derived_ from instrumental concerns. He agreed that severe wireheading issues _potentially_ arise if you allow consequentialist concerns to affect your epistemics.
------
-
-On 3 November 2019, I received an interesting reply on my philosophy-of-categorization thesis from MIRI researcher Abram Demski. Abram asked: ideally, shouldn't all conceptual boundaries be drawn with appeal-to-consequences? Wasn't the problem just with bad (motivated, shortsighted) appeals to consequences? Agents categorize in order to make decisions. The best classifer for an application depends on the costs and benefits. As a classic example, it's very important for evolved prey animals to avoid predators, so it makes sense for their predator-detection classifiers to be configured such that they jump away from every rustling in the bushes, even if it's usually not a predator.
+But the alternative view had its own problems. If your AI consists of a consequentialist module that optimizes for utility in the world, and an epistemic module that optimizes for the accuracy of its beliefs, that's _two_ agents, not one: how could that be reflectively coherent? You could, perhaps, bite the bullet here, for fear that consequentialism doesn't tile and that wireheading was inevitable. On this view, Abram explained, "Agency is an illusion which can only be maintained by crippling agents and giving them a split-brain architecture where an instrumental task-monkey does all the important stuff while an epistemic overseer supervises." Whether this view was ultimately tenable or not, this did show that trying to forbid appeals-to-consequences entirely led to strange places.
-I had thought of the "false-positives are better than false-negatives when detecting predators" example as being about the limitations of evolution as an AI designer: messy evolved animal brains don't bother to track probability and utility separately the way a cleanly-designed AI could. As I had explained in "... Boundaries?", it made sense for _what_ variables you paid attention to, to be motivated by consequences. But _given_ the subspace that's relevant to your interests, you want to run an epistemically legitimate clustering algorithm on the data you see there, which depends on the data, not your values. The only reason value-dependent gerrymandered category boundaries seem like a good idea if you're not careful about philosophy is because it's _wireheading_. Ideal probabilistic beliefs shouldn't depend on consequences.
+I didn't immediately have an answer for Abram, but I was grateful for the engagement. (Abram was clearly addressing the real philosophical issues, and not just trying to mess with me in the sense that almost everyone else in Berkeley was trying to mess with me.)
-Abram didn't think the issue was so clear-cut. Where do "probabilities" come from, in the first place? The reason we expect something like Bayesianism to be an attractor among self-improving agents is _because_ probabilistic reasoning is broadly useful: epistemology can be _derived_ from instrumental concerns. He agreed that severe wireheading issues _potentially_ arise if you allow consequentialist concerns to affect your epistemics.
-
-But the alternative view had its own problems. If your AI consists of a consequentialist module that optimizes for utility in the world, and an epistemic module that optimizes for the accuracy of its beliefs, that's _two_ agents, not one: how could that be reflectively coherent? You could, perhaps, bite the bullet here, for fear that consequentialism doesn't tile and that wireheading was inevitable. On this view, Abram explained, "Agency is an illusion which can only be maintained by crippling agents and giving them a split-brain architecture where an instrumental task-monkey does all the important stuff while an epistemic overseer supervises." Whether this view was ultimately tenable or not, this did show that trying to forbid appeals-to-consequences entirely led to strange places. I didn't immediately have an answer for Abram, but I was grateful for the engagement. (Abram was clearly addressing the real philosophical issues, and not just trying to mess with me the way almost everyone else in Berkeley was trying to mess with me.)
+------
-Also in November 2019, I wrote to Ben about how I was still stuck on writing the grief-memoir. My _plan_ had been that it should have been possibly to tell the story of the Category War while glomarizing about the content of private conversations, then offer Scott and Eliezer pre-publication right of reply (because it's only fair to give your former-hero-current-[frenemies](https://en.wikipedia.org/wiki/Frenemy) warning when you're about to publicly call them intellectually dishonest), then share it to _Less Wrong_ and the /r/TheMotte culture war thread, and then I would have the emotional closure to move on with my life (learn math, go to gym, chop wood, carry water) and not be a mentally-dominated cultist.
+Also in November 2019, I wrote to Ben about how I was still stuck on writing the grief-memoir. My _plan_ had been that it should have been possible to tell the story of the Category War while glomarizing about the content of private conversations, then offer Scott and Eliezer pre-publication right of reply (because it's only fair to give your former-hero-current-[frenemies](https://en.wikipedia.org/wiki/Frenemy) warning when you're about to publicly call them intellectually dishonest), then share it to _Less Wrong_ and the /r/TheMotte culture war thread, and then I would have the emotional closure to move on with my life (learn math, go to gym, chop wood, carry water) and not be a mentally-dominated cultist.
-The reason it _should_ have been safe to write was because Explaining Things is Good. It should be possible to say, "This is not a social attack; I'm not saying 'rationalists Bad, Yudkowsky Bad'; I'm just trying to carefully _tell the true story_ about why, as a matter of cause-and-effect, I've been upset this year, including addressing counterarguments for why some would argue that I shouldn't be upset, why other people could be said to be behaving 'reasonably' given their incentives, why I nevertheless wish they'd be braver and adhere to principle rather than 'reasonably' following incentives, _&c_."
+The reason it _should_ have been safe to write was because Explaining Things Is Good. It should be possible to say, "This is not a social attack; I'm not saying 'rationalists Bad, Yudkowsky Bad'; I'm just trying to carefully _tell the true story_ about why, as a matter of cause-and-effect, I've been upset this year, including addressing counterarguments for why some would argue that I shouldn't be upset, why other people could be said to be behaving 'reasonably' given their incentives, why I nevertheless wish they'd be braver and adhere to principle rather than 'reasonably' following incentives, _&c_."
So why couldn't I write? Was it that I didn't know how to make "This is not a social attack" credible? Maybe because ... it wasn't true?? I was afraid that telling a story about our leader being intellectually dishonest was "the nuclear option" in a way that I couldn't credibly cancel with "But I'm just telling a true story about a thing that was important to me that actually happened" disclaimers. If you're slowly-but-surely gaining territory in a conventional war, _suddenly_ escalating to nukes seems pointlessly destructive. This metaphor is horribly non-normative ([arguing is not a punishment!](https://srconstantin.github.io/2018/12/15/argue-politics-with-your-best-friends.html) carefully telling a true story _about_ an argument is not a nuke!), but I didn't know how to make it stably go away.
I said I would bite that bullet: yes! Yes, I was trying to figure out whether I was being fair to my abusers, and it was an important question to get right! "Other people's lack of standards harmed me, therefore I don't need to hold myself to standards in my response because I have [extenuating circumstances](https://www.lesswrong.com/posts/XYrcTJFJoYKX2DxNL/extenuating-circumstances)" would be a _lame excuse_.
-(This seemed correlated with the recurring stalemated disagreement within our coordination group, where Michael/Ben/Jessica would say, "Fraud, if that word _ever_ meant anything", and while I agreed that they were pointing to an important way in which things were messed up, I was still sympathetic to the Caliphate-defender's reply that the Vassarite usage of "fraud" was motte-and-baileying between vastly different senses of _fraud_; I wanted to do _more work_ to formulate a _more precise theory_ of the psychology of deception to describe exactly how things are messed up a way that wouldn't be susceptible to the motte-and-bailey charge.)
+(This seemed correlated with the recurring stalemated disagreement within our coordination group, where Michael/Ben/Jessica would say, "Fraud, if that word _ever_ meant anything", and while I agreed that they were pointing to an important way in which things were messed up, I was still sympathetic to the Caliphate-defender's reply that this usage of "fraud" was motte-and-baileying between vastly different senses of _fraud_; I wanted to do _more work_ to formulate a _more precise theory_ of the psychology of deception to describe exactly how things are messed up a way that wouldn't be susceptible to the motte-and-bailey charge.)
[TODO: Ziz's protest;
[TODO: complicity and friendship]
-[TODO: I had a productive winter blogging vacation in December 2019
-pull the trigger on "On the Argumentative Form"; I was worried about leaking info from private conversations, but I'm in the clear "That's your hobbyhorse" is an observation anyone could make from content alone]
+-----
+
+I had a pretty productive blogging spree in late 2019.
-[TODO: "Firming Up ..." Dec 2019: combatting Yudkowsky's not-technically-lying shenanigans]
+[TODO: "The Legend of the Extortion War"]
+
+I polished and pulled the trigger on ["On the Argumentative Form 'Super-Proton Things Tend to Come In Varieties'"](/2019/Dec/on-the-argumentative-form-super-proton-things-tend-to-come-in-varieties/), my reply to Yudkowsky's implicit political concession to me back in March. I had been reluctant to post it based on an intuition of, "My childhood hero was trying to _do me a favor_; it would be a betrayal to reject the gift." The post itself explained why that intuition was crazy, but _that_ just brought up more anxieties about whether the explanation constituted leaking information from private conversations—but I had chosen my words carefully such that it wasn't. ("Even if Yudkowsky doesn't know you exist [...] he's _effectively_ doing your cause a favor" was something I could have plausibly written in the possible world where the antecedent was true.) Jessica said the post seemed good.
+
+I also published ["Firming Up Not-Lying Around Its Edge-Cases Is Less Broadly Useful Than One Might Initially Think"](https://www.lesswrong.com/posts/MN4NRkMw7ggt9587K/firming-up-not-lying-around-its-edge-cases-is-less-broadly), a critique of Yudkowsky's [code of "meta-honesty"](https://www.lesswrong.com/posts/xdwbX9pFEr7Pomaxv/meta-honesty-firming-up-honesty-around-its-edge-cases), on the occasion of the latter's nomination for a Best-of-2018 Review. (The ungainly title of my post was "softened" from an earlier draft, which said "... Surprisingly Useless".)
+
+-----
[TODO: plan to reach out to Rick 14 December
Anna's reply 21 December
At this point it was almost 2 _p.m._ (the paragraphs above summarize a larger volume of typing), and Scott mentioned that he wanted to go to the Event Horizon Christmas party, and asked if I wanted to come and continue the discussion there. I assented, and thanked him for his time; it would be really exciting if we could avoid a rationalist civil war. (I thought my "you need accurate models before you can do utilitarianism" philosophy was also near the root of Ben's objections to the EA movement.)
-When I arrived at the party, people were doing a reading of [the "Hero Licensing" dialogue epilogue](https://www.lesswrong.com/posts/dhj9dhiwhq3DX6W8z/hero-licensing) to _Inadequate Equilibria_. Yudkowsky himself was, playing the part of the Mysterious Stranger in the dialogue. At some point, Scott and I retreated upstairs to continue our discussion. By the end of it, I was at least feeling more assured of Scott's sincerity (rather than him being coerced into not saying anything incriminating over email). Scott said he would edit in a disclaimer note at the end of "... Not Man for the Categories".
+When I arrived at the party, people were doing a reading of [the "Hero Licensing" dialogue epilogue](https://www.lesswrong.com/posts/dhj9dhiwhq3DX6W8z/hero-licensing) to _Inadequate Equilibria_, with Yudkowsky himself was playing the part of the Mysterious Stranger. At some point, Scott and I retreated upstairs to continue our discussion. By the end of it, I was at least feeling more assured of Scott's sincerity (rather than him being coerced into not saying anything incriminating over email). Scott said he would edit in a disclaimer note at the end of "... Not Man for the Categories".
-It would have been interesting if I also got the chance to talk to Yudkowsky for a few minutes, but if I did, I wouldn't be allowed to recount any details of that here due to the privacy norm I'm following in this document.
+It would have been interesting if I also got the chance to talk to Yudkowsky for a few minutes, but if I did, I wouldn't be allowed to recount any details of that here due to the privacy norm I'm following in this post.
The rest of the party was nice. People were reading funny GPT-2 quotes from their phones. At one point, conversation happened to zag in a way that let me show off the probability fact I had learned during Math and Wellness Month. A MIRI researcher sympathetically told me that it would be sad if I had to leave the Bay Area, which I thought was nice. There was nothing about the immediate conversational context to suggest that I might have to leave the Bay, but I guess by this point, my existence had become a context.
Another effect of my feeling better after the party was that my motivation to keep working on my memoir of the Category War vanished—as if I was still putting weight on a [zero-sum frame](https://unstableontology.com/2019/09/10/truth-telling-is-aggression-in-zero-sum-frames/) in which the memoir was a nuke that I only wanted to use as an absolute last resort.
-Ben wrote:
+Ben wrote (Subject: "Re: state of Church leadership"):
> It seems to that according to Zack's own account, even writing the memoir _privately_ feels like an act of war that he'd rather avoid, not just using his own territory as he sees fit to create _internal_ clarity around a thing.
>
(It does not, actually, have a happy ending where everyone comes to their senses.)
+------
+
+While visiting Valinor on 4 February 2020, I remember my nose dripping while I was holding Koios, the baby. Alicorn offered me a tissue. I asked if I shouldn't be holding the baby while my nose was dripping and I therefore plausibly had a cold. She said it was fine. On the topic of possible sickness, I said that I hoped the novel coronavirus people were talking about didn't go pandemic.
+
+It did. The Berkeley rats took social distancing guidelines very seriously, so it would be a while before I could visit again.
+
+------
+
+On 10 February 2020, Scott Alexander published ["Autogenderphilia Is Common and Not Especially Related to Transgender"](https://slatestarcodex.com/2020/02/10/autogenderphilia-is-common-and-not-especially-related-to-transgender/), an analysis of the results of the autogynephilia/autoandrophilia questions on the recent _Slate Star Codex_ survey.
+
+I appreciated the gesture of getting real data, but I was deeply unimpressed with Alexander's analysis for reasons that I found difficult to write up in a timely manner. Three years later, I eventually got around to [polishing my draft and throwing it up as a standalone post](/2023/Mar/reply-to-scott-alexander-on-autogenderphilia/), rather than cluttering the present narrative with my explanation.
+
+Briefly, based on eyballing the survey data, Alexander proposes "if you identify as a gender, and you're attracted to that gender, it's a natural leap to be attracted to yourself being that gender" as a "very boring" theory, but on my worldview, a hypothesis that puts "gay people (cis and trans)" in the antecedent is _not_ boring and actually takes on a big complexity penalty: I just don't think the group of gay men _and_ lesbians _and_ straight males with female gender identities _and_ straight females with male gender identities have much in common with each other, except sociologically (being "queer"), and by being human.
+
+(I do like the [hypernym](https://en.wikipedia.org/wiki/Hyponymy_and_hypernymy) _autogenderphilia_.)
+
-------
There's another extremely important part of the story that _would_ fit around here chronologically, but I again find myself constrained by privacy norms: everyone's common sense of decency (this time, even including my own) screams that it's not my story to tell.
Thus, it's unclear how much mere adherence to norms helps, when people's wills are actually misaligned. If I'm furious at Yudkowsky for prevaricating about my Something to Protect, and am in fact _more_ furious rather than less that he managed to do it without violating the norm against "lying", I should not be so foolish as to think myself innocent and beyond reproach for not having "really said it."
-Having considered all this, here's what I think I can say: I spent many hours in the first half of 2020 working on a private Document about a disturbing hypothesis that had occured to me.
+Having considered all this, here's what I think I can say: I spent a number of hours from early May 2020 to early July 2020 working on a private Document about a disturbing hypothesis that had occured to me earlier that year.
-Previously, I had _already_ thought it was nuts that trans ideology was exerting influence the rearing of gender-non-conforming children, that is, children who are far outside the typical norm of _behavior_ (_e.g._, social play styles) for their sex: very tomboyish girls and very feminine boys. Under recent historical conditions in the West, these kids were mostly "pre-gay" rather than trans. (The stereotype about lesbians being masculine and gay men being feminine is, like most stereotypes, basically true: sex-atypical childhood behavior between gay and straight adults [has been meta-analyzed at _d_ ≈ 1.31 for men and _d_ ≈ 0.96 for women](/papers/bailey-zucker-childhood_sex-typed_behavior_and_sexual_orientation.pdf).) A solid supermajority of children diagnosed with gender dysphoria [ended up growing out of it by puberty](/papers/steensma_et_al-factors_associated_with_desistence_and_persistence.pdf). In the culture of the current year, it seemed likely that a lot of those kids would get affirmed into a cross-sex identity (and being a lifelong medical patient) much earlier, even though most of them would have otherwise (under [a "watchful waiting" protocol](/papers/de_vries-cohen-kettenis-clinical_management_of_gender_dysphoria_in_children.pdf)) grown up to be ordinary gay men and lesbians.
+Previously, I had _already_ thought it was nuts that trans ideology was exerting influence on the rearing of gender-non-conforming children, that is, children who are far outside the typical norm of _behavior_ (_e.g._, social play styles) for their sex: very tomboyish girls and very effeminate boys. Under recent historical conditions in the West, these kids were mostly "pre-gay" rather than trans. (The stereotype about lesbians being masculine and gay men being feminine is, like most stereotypes, basically true: sex-atypical childhood behavior between gay and straight adults [has been meta-analyzed at _d_ ≈ 1.31 for men and _d_ ≈ 0.96 for women](/papers/bailey-zucker-childhood_sex-typed_behavior_and_sexual_orientation.pdf).) A solid supermajority of children diagnosed with gender dysphoria [ended up growing out of it by puberty](/papers/steensma_et_al-factors_associated_with_desistence_and_persistence.pdf). In the culture of the current year, it seemed likely that a lot of those kids would get affirmed into a cross-sex identity (and being a lifelong medical patient) much earlier, even though most of them would have otherwise (under [a "watchful waiting" protocol](/papers/de_vries-cohen-kettenis-clinical_management_of_gender_dysphoria_in_children.pdf)) grown up to be ordinary gay men and lesbians.
What made this crazy, in my view, was not just that child transition is a dubious treatment decision, but that it's a dubious treatment decision made on the basis of the obvious falsehood that "trans" was one thing: the cultural phenomenon of "trans kids" was being used to legitimize trans _adults_, even though the vast supermajority of trans adults were in the AGP taxon and therefore _had never resembled_ these HSTS-taxon kids. That is: pre-gay kids are being sterilized in order to affirm the narcissistic delusions of _guys like me_.
Scott Alexander has written about how [concept-shaped holes can be impossible to notice](https://slatestarcodex.com/2017/11/07/concept-shaped-holes-can-be-impossible-to-notice/). A culture whose [civic religion](https://slatestarcodex.com/2019/07/08/gay-rites-are-civil-rites/) celebrates being trans, and denies that gender has truth conditions other than the individual's say-so, has concept-shaped holes that make it hard to notice the hypothesis "I'm having a systematically worse childhood than I otherwise would have because all the grown-ups in my life have agreed I was a girl since I was three years old, even though all of my actual traits are sampled from the joint distribution of juvenile male humans, not juvenile female humans", even if it's true.
-... anyway, that's just a hypothesis that occured to me in early 2020, about something that _could_ happen in the culture of the current year, hypothetically, as far as I know. I'm not a parent and I haven't studied child development. (And even if the "Clever Hans" etiological pathway I conjectured is real, the extent to which it might apply to any particular case is complex; you could imagine a kid who was "actually trans", whose social transition merely happened earlier than it otherwise would have due to these dynamics.)
+... anyway, that's just a hypothesis that occured to me in early 2020, about something that _could_ happen in the culture of the current year, hypothetically, as far as I know. I'm not a parent and I haven't studied child development. And even if the "Clever Hans" etiological pathway I conjectured is real, the extent to which it might apply to any particular case is complex; you could imagine a kid who _was_ "actually trans", whose social transition merely happened earlier than it otherwise would have due to these dynamics.
-For some reason, it seemed really important that I draft a Document about it with lots of citations to send to a few friends. If I get around to it, I might clean it up and publish it as a blog post (working title: "Trans Kids on the Margin; and, Harms from Misleading Training Data").
+For some reason, it seemed really important that I draft a Document about it with lots of citations to send to a few friends. If I get around to it, I might clean it up and publish it as its own blog post (working title: "Trans Kids on the Margin; and, Harms from Misleading Training Data").
-Given that I spent so many hours on this little research/writing project in early 2020, I think it makes sense for me to mention at this point in the memoir, where it fits in chronologically. I have an inalienable right to talk about my own research interests, and talking about my research interests obviously doesn't violate any norm against leaking private information about someone else's family, or criticizing someone's parenting decisions.
+Given that I spent so many hours on this little research/writing project in May–July 2020, I think it makes sense for me to mention it at this point in the memoir, where it fits in chronologically. I have an inalienable right to talk about my own research interests, and talking about my research interests obviously doesn't violate any norm against leaking private information about someone else's family, or criticizing someone else's parenting decisions.
(Only—you two have such beautiful children!)
-----
-[TODO: pandemic starts]
-
-[TODO: "Autogenderphilia Is Common" https://slatestarcodex.com/2020/02/10/autogenderphilia-is-common-and-not-especially-related-to-transgender/]
-
------
-
On 1 June 2020, I received a Twitter DM from _New York Times_ reporter Cade Metz, who said he was "exploring a story about the intersection of the rationality community and Silicon Valley". I sent him an email saying that I would be happy to talk, but that I'd actually been pretty disappointed with the community lately: I was worried that the social pressures of trying to _be_ a "community" and protect the group's status (_e.g._, from _New York Times_ reporters who might portray us in an unflattering light??) incentivize people to compromise on the ideals of _systematically correct reasoning_ that made the community valuable in the first place.
He never got back to me. Three weeks later, all existing _Slate Star Codex_ posts were taken down.
But if you _do_ have the math, a moment of introspection will convince you that the analogy between category "boundaries" and national borders is not a particularly deep or informative one.
-A two-dimensional political map tells you which areas of the Earth's surface are under the jurisdiction of what government.
-
-In contrast, category "boundaries" tell you which regions of very high-dimensional configuration space correspond to a word/concept, which is useful _because_ that structure is useful for making probabilistic inferences: you can use your observastions of some aspects of an entity (some of the coordinates of a point in configuration space) to infer category-membership, and then use category membership to make predictions about aspects that you haven't yet observed.
+A two-dimensional political map tells you which areas of the Earth's surface are under the jurisdiction of what government. In contrast, category "boundaries" tell you which regions of very high-dimensional configuration space correspond to a word/concept, which is useful _because_ that structure is useful for making probabilistic inferences: you can use your observastions of some aspects of an entity (some of the coordinates of a point in configuration space) to infer category-membership, and then use category membership to make predictions about aspects that you haven't yet observed.
But the trick only works to the extent that the category is a regular, non-squiggly region of configuration space: if you know that egg-shaped objects tend to be blue, and you see a black-and-white photo of an egg-shaped object, you can get _close_ to picking out its color on a color wheel. But if egg-shaped objects tend to blue _or_ green _or_ red _or_ gray, you wouldn't know where to point to on the color wheel.
How could I make this rigorous? Did I want to be talking about the _variance_ of my features conditional on category-membership? Was "connectedness" intrinsically the what I wanted, or was connectedness only important because it cut down the number of possibilities? (There are 8!/(6!2!) = 28 ways to choose two elements from `{1..8}`, but only 7 ways to choose two contiguous elements.) I thought connectedness _was_ intrinsically important, because we didn't just want _few_ things, we wanted things that are _similar enough to make similar decisions about_.
-I put the question to a few friends (Subject: "rubber duck philosophy"), and Jessica said that my identification of the variance as the key quantity sounded right: it amounted to the expected squared error of someone trying to guess the values of the features given the category. It was okay that this wasn't a purely information-theoretic criterion, because for problems involving guessing a numeric quantity, bits that get you closer to the right answer were more valuable than bits that didn't.
+I put the question to a few friends in July 2020 (Subject: "rubber duck philosophy"), and Jessica said that my identification of the variance as the key quantity sounded right: it amounted to the expected squared error of someone trying to guess the values of the features given the category. It was okay that this wasn't a purely information-theoretic criterion, because for problems involving guessing a numeric quantity, bits that get you closer to the right answer were more valuable than bits that didn't.
------
* Yudkowsky made a stray remark about social media causing people to say crazy thing
* I got enraged, posted a couple Tweets, including a preview of "Unnatural Categories"
* something in my boiled over, and I eventually ended up staying up late writing an angry email
+ * this ruins my chances for being a "neutral" bridge between the Vassarites and the Caliphate, but that's OK
+ * mostly reproduce below (with a few redactions for either brevity or compliance with privacy norms, but I'm not going to clarify which)
]
[TODO: "out of patience" email]
_ex cathedra_ statement that gender categories are not an exception to the rule, only 1 year and 8 months after asking for it
]
-And really, that _should_ have been the end of the story. At the trifling cost of two years of my life, we finally got a clarification from Yudkowsky that you can't define the word _woman_ any way you like. I didn't think I was entitled to anything more than that. I was satsified. I still published "Unnatural Categories Are Optimized for Deception" in January 2021, but if I hadn't been further provoked, I wouldn't have occasion to continue waging the robot-cult religious civil war.
+And really, that _should_ have been the end of the story. At the trifling cost of two years of my life, we finally got a clarification from Yudkowsky that you can't define the word _woman_ any way you like. I didn't think I was entitled to anything more than that. I was satisfied. If I hadn't been further provoked, I wouldn't have occasion to continue waging the robot-cult religious civil war.
-[TODO: psychiatric disaster, breakup with Vassar group, this was really bad for me
-[As it is written](https://www.alessonislearned.com/), "A lesson is learned but the damage is irreversible."
-]
+I still published ["Unnatural Categories Are Optimized for Deception"](https://www.lesswrong.com/posts/onwgTH6n8wxRSo2BJ/unnatural-categories-are-optimized-for-deception) in January 2021.
-[TODO: "Unnatural Categories Are Optimized for Deception"
+I wrote back to Abram Demski regarding his comments from fourteen months before: on further thought, he was right. Even granting my point that evolution didn't figure out how to track probability and utility separately, as Abram had pointed out, the _fact_ that it didn't meant that not tracking it could be an effective AI design. Just because evolution takes shortcuts that human engineers wouldn't didn't mean shortcuts are "wrong". (Rather, there are laws governing which kinds of shortcuts _work_.)
-Abram was right
+Abram was also right that it would be weird if reflective coherence was somehow impossible: the AI shouldn't have to fundamentally reason differently about "rewriting code in some 'external' program" and "rewriting 'its own' code." In that light, it made sense to regard "have accurate beliefs" as _merely_ a convergent instrumental subgoal, rather than what rationality is about—as sacrilegious as that felt to type.
-the fact that it didn't means that not tracking it can be an effective AI design! Just because evolution takes shortcuts that human engineers wouldn't doesn't mean shortcuts are "wrong" (instead, there are laws governing which kinds of shortcuts work).
+And yet, somehow, "have accurate beliefs" seemed _more fundamental_ than other convergent instrumental subgoals like "seek power and resources". Could this be made precise? As a stab in the dark, was it possible that the [theorems on the ubiquity of power-seeking](https://www.lesswrong.com/posts/6DuJxY8X45Sco4bS2/seeking-power-is-often-robustly-instrumental-in-mdps) might generalize to a similar conclusion about "accuracy-seeking"? If it _didn't_, the reason why it didn't might explain why accuracy seems more fundamental.
-Embedded agency means that the AI shouldn't have to fundamentally reason differently about "rewriting code in some 'external' program" and "rewriting 'my own' code." In that light, it makes sense to regard "have accurate beliefs" as merely a convergent instrumental subgoal, rather than what rationality is about
-somehow accuracy seems more fundamental than power or resources ... could that be formalized?
+[TODO: psychiatric disaster, breakup with Vassar group, this was really bad for me
+[As it is written](https://www.alessonislearned.com/), "A lesson is learned but the damage is irreversible."
]
projects / Ultimately_Untrue_Thought.git / blobdiff