I perceived a pattern where people who are in trouble with the orthodoxy feel an incentive to buy their own safety by denouncing other heretics: not just disagreeing with the other heretics because those other heresies are in fact mistaken, which would be right and proper Discourse, but denouncing them ("actively hostile to") as a way of paying Danegeld.
-Suppose there are five true heresies, but anyone who's on the record believing more than one gets burned as a witch. Then it's impossible to have a unified rationalist community, because people who want to talk about one heresy can't let themselves be seen in the company of people who believe another. That's why Scott Alexander couldn't get the philosophy-of-categorization right in full generality (even though his writings revealed an implicit understanding of the correct way)[^implicit-understanding], and he and I had a common enemy in the social-justice egregore). He couldn't afford to. He'd already spent his Overton budget [on anti-feminism](https://slatestarcodex.com/2015/01/01/untitled/).
+Suppose there are five true heresies, but anyone who's on the record believing more than one gets burned as a witch. Then it's [impossible to have a unified rationalist community](https://www.lesswrong.com/posts/DoPo4PDjgSySquHX8/heads-i-win-tails-never-heard-of-her-or-selective-reporting), because people who want to talk about one heresy can't let themselves be seen in the company of people who believe another. That's why Scott Alexander couldn't get the philosophy of categorization right in full generality, even though his writings revealed an implicit understanding of the correct way,[^implicit-understanding] and he and I had a common enemy in the social-justice egregore. He couldn't afford to. He'd already spent his Overton budget [on anti-feminism](https://slatestarcodex.com/2015/01/01/untitled/).
-[^implicit-understanding]: As I had [explained to him earlier](/2023/Jul/a-hill-of-validity-in-defense-of-meaning/#noncentral-fallacy), Alexander's famous [post on the noncentral fallacy](https://www.lesswrong.com/posts/yCWPkLi8wJvewPbEp/the-noncentral-fallacy-the-worst-argument-in-the-world)
+[^implicit-understanding]: As I had [explained to him earlier](/2023/Jul/a-hill-of-validity-in-defense-of-meaning/#noncentral-fallacy), Alexander's famous [post on the noncentral fallacy](https://www.lesswrong.com/posts/yCWPkLi8wJvewPbEp/the-noncentral-fallacy-the-worst-argument-in-the-world) condemned the same shenanigans he praised in the context of gender identity: Alexander's examples of the noncentral fallacy had largely been arguable edge-cases of a negative-valence category being inappropriately framed as typical (abortion is murder, taxation is theft), but "trans women are women" was the same thing, but with a positive-valence category.
-[TODO—stitch together the language here:
-
-Most of Alexander's examples focus on
-
-In ["Does the Glasgow Coma Scale exist? Do comas?"](https://slatestarcodex.com/2014/08/11/does-the-glasgow-coma-scale-exist-do-comas/) (published just three months before "... Not Man for the Categories"), Alexander
-
-]
+ In ["Does the Glasgow Coma Scale exist? Do comas?"](https://slatestarcodex.com/2014/08/11/does-the-glasgow-coma-scale-exist-do-comas/) (published just three months before "... Not Man for the Categories"), Alexander defends the usefulness of "comas" and "intelligence" in terms of their predictive usefulness. (The post uses the terms "predict", "prediction", "predictive power", _&c._ 16 times.) He doesn't say that the Glasgow Coma Scale is justified because it makes people happy for comas to be defined that way, because that would be absurd.
Alexander (and Yudkowsky and Anna and the rest of the Caliphate) seemed to accept this as an inevitable background fact of existence, like the weather. But I saw a Schelling point off in the distance where us witches stick together for Free Speech, and it was tempting to try to jump there. (It would probably be better if there were a way to organize just the good witches, and exclude all the Actually Bad witches, but the [Sorites problem](https://plato.stanford.edu/entries/sorites-paradox/) on witch Badness made that hard to organize without falling back to the falling back to the one-heresy-per-thinker equilibrium.)
-Jessica thought my use of "heresy" was conflating factual beliefs with political movements. (There are no intrinsically "right wing" _facts_.) I agreed that conflating political positions with facts would be bad (and that it would be bad if I were doing that without "intending" to). I wasn't interested in defending the "alt-right" (whatever that means) broadly. But I had learned stuff from reading far-right authors (most notably Moldbug), and from talking with "Thomas". I was starting to appreciate [what Michael had said about "Less precise is more violent" back in April](#less-precise-is-more-violent) (when I was talking about criticizing "rationalists").
+Jessica thought my use of "heresy" was conflating factual beliefs with political movements. (There are no intrinsically "right wing" _facts_.) I agreed that conflating political positions with facts would be bad. I wasn't interested in defending the "alt-right" (whatever that means) broadly. But I had learned stuff from reading far-right authors [(most notably Mencius Moldbug)](/2023/Jul/blanchards-dangerous-idea-and-the-plight-of-the-lucid-crossdreamer/#unqualified-reservations) and from talking with "Thomas". I was starting to appreciate [what Michael had said about "Less precise is more violent" back in April](/2023/Jul/a-hill-of-validity-in-defense-of-meaning/#less-precise-is-more-violent) (when I was talking about criticizing "rationalists").
Jessica asked if my opinion would change depending on whether Yudkowsky thought neoreaction was intellectually worth engaging with. (Yudkowsky [had said years ago](https://www.lesswrong.com/posts/6qPextf9KyWLFJ53j/why-is-mencius-moldbug-so-popular-on-less-wrong-answer-he-s?commentId=TcLhiMk8BTp4vN3Zs) that Moldbug was low quality.)
-I did believe that Yudkowsky believed that neoreaction was not worth engaging with. I would never fault anyone for saying "I vehemently disagree with what little I've read and/or heard of this-and-such author." I wasn't accusing Yudkowsky of being insincere.
+I did believe that Yudkowsky believed that neoreaction was not worth engaging with. I would never fault anyone for saying "I vehemently disagree with what little I've read and/or heard of this-and-such author." I wasn't accusing him of being insincere.
-What I _did_ think was that the need to keep up appearances of not-being-a-right-wing-Bad-Guy was a pretty serious distortion on people's beliefs, because there are at least a few questions-of-fact where believing the correct answer can, in today's political environment, be used to paint one as a right-wing Bad Guy. I would have hoped for Yudkowsky to _notice that this is a rationality problem_, and to _not actively make the problem worse_, and I was counting "I do not welcome support from those quarters" as making the problem worse insofar as it would seem to imply that the extent to which I think I've learned valuable things from Moldbug, made me less welcome in Yudkowsky's fiefdom.
+What I did think was that the need to keep up appearances of not-being-a-right-wing-Bad-Guy was a serious distortion on people's beliefs, because there are at least a few questions of fact where believing the correct answer can, in today's political environment, be used to paint one as a right-wing Bad Guy. I would have hoped for Yudkowsky to _notice that this is a rationality problem_, and to _not actively make the problem worse_, and I was counting "I do not welcome support from those quarters" as making the problem worse insofar as it would seem to imply that the extent to which I think I've learned valuable things from Moldbug, made me less welcome in Yudkowsky's fiefdom.
-Yudkowsky certainly wouldn't endorse "Even learning things from these people makes you unwelcome" _as stated_, but "I do not welcome support from those quarters" still seemed like a _pointlessly_ partisan silencing/shunning attempt, when one could just as easily say, "I'm not a neoreactionary, and if some people who read me are, that's _obviously not my fault_."
+Yudkowsky certainly wouldn't endorse "Even learning things from these people makes you unwelcome" _as stated_, but "I do not welcome support from those quarters" still seemed like a pointlessly partisan silencing/shunning attempt, when one could just as easily say, "I'm not a neoreactionary, and if some people who read me are, that's _obviously not my fault_."
-Jessica asked if Yudkowsky denouncing neoreaction and the alt-right would still seem harmful, if he were to _also_ to acknowledge, _e.g._, racial IQ differences?
+Jessica asked if Yudkowsky denouncing neoreaction and the alt-right would still seem harmful, if he were to also to acknowledge, _e.g._, racial IQ differences?
I agreed that it would be helpful, but realistically, I didn't see why Yudkowsky should want to poke the race-differences hornet's nest. This was the tragedy of recursive silencing: if you can't afford to engage with heterodox ideas, you either become an [evidence-filtering clever arguer](https://www.lesswrong.com/posts/kJiPnaQPiy4p9Eqki/what-evidence-filtered-evidence), or you're not allowed to talk about anything except math. (Not even the relationship between math and human natural language, as we had found out recently.)
-It was as if there was a "Say Everything" attractor, and a "Say Nothing" attractor, and _my_ incentives were pushing me towards the "Say Everything" attractor—but that was only because I had [Something to Protect](/2019/Jul/the-source-of-our-power/) in the forbidden zone and I was a good programmer (who could therefore expect to be employable somewhere, just as [James Damore eventually found another job](https://twitter.com/JamesADamore/status/1034623633174478849)). Anyone in less extreme circumstances would find themselves being pushed to the "Say Nothing" attractor.
+It was as if there was a "Say Everything" attractor, and a "Say Nothing" attractor, and my incentives were pushing me towards the "Say Everything" attractor—but that was only because I had [Something to Protect](/2019/Jul/the-source-of-our-power/) in the forbidden zone and I was a decent programmer (who could therefore expect to be employable somewhere, just as [James Damore eventually found another job](https://twitter.com/JamesADamore/status/1034623633174478849)). Anyone in less extreme circumstances would find themselves being pushed to the "Say Nothing" attractor.
-It was instructive to compare this new disavowal of neoreaction with one from 2013 (quoted by [Moldbug](https://www.unqualified-reservations.org/2013/11/mr-jones-is-rather-concerned/) and [others](https://medium.com/@2045singularity/white-supremacist-futurism-81be3fa7020d)[^linkrot]), in response to a _TechCrunch_ article citing former MIRI employee Michael Anissimov's neoreactionary blog _More Right_:
+It was instructive to compare Yudkowsky's new disavowal of neoreaction with one from 2013, in response to a _TechCrunch_ article citing former MIRI employee Michael Anissimov's neoreactionary blog _More Right_:[^linkrot]
-[^linkrot]: The original _TechCrunch_ comment would seem to have succumbed to [linkrot](https://www.gwern.net/Archiving-URLs#link-rot).
+[^linkrot]: The original _TechCrunch_ comment would seem to have succumbed to [linkrot](https://www.gwern.net/Archiving-URLs#link-rot), but Yudkowsky's comment was quoted by [Moldbug](https://www.unqualified-reservations.org/2013/11/mr-jones-is-rather-concerned/) and [others](https://medium.com/@2045singularity/white-supremacist-futurism-81be3fa7020d)
> "More Right" is not any kind of acknowledged offspring of Less Wrong nor is it so much as linked to by the Less Wrong site. We are not part of a neoreactionary conspiracy. We are and have been explicitly pro-Enlightenment, as such, under that name. Should it be the case that any neoreactionary is citing me as a supporter of their ideas, I was never asked and never gave my consent. [...]
>
> Also to be clear: I try not to dismiss ideas out of hand due to fear of public unpopularity. However I found Scott Alexander's takedown of neoreaction convincing and thus I shrugged and didn't bother to investigate further.
-My "negotiating with terrorists" criticism did _not_ apply to the 2013 statement. "More Right" _was_ brand encroachment on Anissimov's part that Yudkowsky had a legitimate interest in policing, _and_ the "I try not to dismiss ideas out of hand" disclaimer importantly avoided legitimizing [the McCarthyist persecution](https://www.unqualified-reservations.org/2013/09/technology-communism-and-brown-scare/).
+My "negotiating with terrorists" criticism did not apply to the 2013 statement. "More Right" _was_ brand encroachment on Anissimov's part that Yudkowsky had a legitimate interest in policing, and the "I try not to dismiss ideas out of hand" disclaimer importantly avoided legitimizing [the McCarthyist persecution](https://www.unqualified-reservations.org/2013/09/technology-communism-and-brown-scare/).
-The question was, what had specifically happened in the last six years to shift Eliezer's opinion on neoreaction from (paraphrased) "Scott says it's wrong, so I stopped reading" to (verbatim) "actively hostile"? Note especially the inversion from (both paraphrased) "I don't support neoreaction" (fine, of course) to "I don't even want _them_ supporting _me_" ([**?!?!**](https://twitter.com/zackmdavis/status/1164329446314135552)—humans with very different views on politics nevertheless have a common interest in not being transformed into paperclips).
+The question was, what had specifically happened in the last six years to shift Yudkowsky's opinion on neoreaction from (paraphrased) "Scott says it's wrong, so I stopped reading" to (verbatim) "actively hostile"? Note especially the inversion from (both paraphrased) "I don't support neoreaction" (fine, of course) to "I don't even want _them_ supporting _me_" ([which was bizarre](https://twitter.com/zackmdavis/status/1164329446314135552); humans with very different views on politics nevertheless have a common interest in not being transformed into paperclips).
-Did Yudkowsky get _new information_ about neoreaction's hidden Badness parameter sometime between 2013 and 2019, or did moral coercion on him from the left intensify (because Trump and [because Berkeley](https://thezvi.wordpress.com/2017/08/12/what-is-rationalist-berkleys-community-culture/))? My bet was on the latter.
+Did Yudkowsky get new information about neoreaction's hidden Badness parameter sometime between 2013 and 2019, or did moral coercion on him from the left intensify (because Trump and [because Berkeley](https://thezvi.wordpress.com/2017/08/12/what-is-rationalist-berkleys-community-culture/))? My bet was on the latter.
------
-However it happened, it didn't seem like the brain damage was limited to "political" topics, either. In November 2019, we saw another example of Yudkowsky destroying language for the sake of politeness, this time the non-Culture-War context of him [_trying to wirehead his fiction subreddit by suppressing criticism-in-general_](https://www.reddit.com/r/rational/comments/dvkv41/meta_reducing_negativity_on_rrational/).
+However it happened, it didn't seem like the brain damage was limited to "political" topics, either. In November 2019, we saw another example of Yudkowsky destroying language for the sake of politeness, this time the non-Culture-War context of him [trying to wirehead his fiction subreddit by suppressing criticism-in-general](https://www.reddit.com/r/rational/comments/dvkv41/meta_reducing_negativity_on_rrational/).
That's _my_ characterization, of course: the post itself talks about "reducing negativity". [In a followup comment, Yudkowsky wrote](https://www.reddit.com/r/rational/comments/dvkv41/meta_reducing_negativity_on_rrational/f7fs88l/) (bolding mine):
> On discussion threads for a work's particular chapter, people may debate the well-executedness of some particular feature of that work's particular chapter. Comments saying that nobody should enjoy this whole work are still verboten. **Replies here should still follow the etiquette of saying "Mileage varied: I thought character X seemed stupid to me" rather than saying "No, character X was actually quite stupid."**
-But ... "I thought X seemed Y to me"[^pleonasm] and "X is Y" _do not mean the same thing_. [The map is not the territory](https://www.lesswrong.com/posts/KJ9MFBPwXGwNpadf2/skill-the-map-is-not-the-territory). [The quotation is not the referent](https://www.lesswrong.com/posts/np3tP49caG4uFLRbS/the-quotation-is-not-the-referent). [The planning algorithm that maximizes the probability of doing a thing is different from the algorithm that maximizes the probability of having "tried" to do the thing](https://www.lesswrong.com/posts/WLJwTJ7uGPA5Qphbp/trying-to-try). [If my character is actually quite stupid, I want to believe that my character is actually quite stupid.](https://www.lesswrong.com/tag/litany-of-tarski)
+But ... "I thought X seemed Y to me"[^pleonasm] and "X is Y" do not mean the same thing! [The map is not the territory](https://www.lesswrong.com/posts/KJ9MFBPwXGwNpadf2/skill-the-map-is-not-the-territory). [The quotation is not the referent](https://www.lesswrong.com/posts/np3tP49caG4uFLRbS/the-quotation-is-not-the-referent). [The planning algorithm that maximizes the probability of doing a thing is different from the algorithm that maximizes the probability of having "tried" to do the thing](https://www.lesswrong.com/posts/WLJwTJ7uGPA5Qphbp/trying-to-try). [If my character is actually quite stupid, I want to believe that my character is actually quite stupid.](https://www.lesswrong.com/tag/litany-of-tarski)
[^pleonasm]: The pleonasm here ("to me" being redundant with "I thought") is especially galling coming from someone who's usually a good writer!
-It might seem like a little thing of no significance—requiring "I" statements is commonplace in therapy groups and corporate sensitivity training—but this little thing _coming from Eliezer Yudkowsky setting guidelines for an explicitly "rationalist" space_ made a pattern click. If everyone is forced to only make narcissistic claims about their map ("_I_ think", "_I_ feel"), and not make claims about the territory (which could be construed to call other people's maps into question and thereby threaten them, because [disagreement is disrespect](http://www.overcomingbias.com/2008/09/disagreement-is.html)), that's great for reducing social conflict, but it's not great for the kind of collective information processing that actually accomplishes cognitive work, like good literary criticism. A rationalist space _needs to be able to talk about the territory_.
+It might seem like a little thing of no significance—requiring ["I" statements](https://en.wikipedia.org/wiki/I-message) is commonplace in therapy groups and corporate sensitivity training—but this little thing coming from Eliezer Yudkowsky setting guidelines for an explicitly "rationalist" space made a pattern click. If everyone is forced to only make narcissistic claims about their map ("_I_ think", "_I_ feel"), and not make claims about the territory (which could be construed to call other people's maps into question and thereby threaten them, because [disagreement is disrespect](http://www.overcomingbias.com/2008/09/disagreement-is.html)), that's great for reducing social conflict, but it's not great for the kind of collective information processing that accomplishes cognitive work,[^i-statements] like good literary criticism. A rationalist space needs to be able to talk about the territory.
+
+[^i-statements]: At best, "I" statements make sense in a context where everyone's speech is considered part of the "official record". Wrapping claims in "I think" removes the need for dissenters to immediately object for fear that the claim will be accepted onto the shared map.
-I understand that Yudkowsky wouldn't agree with that characterization, and to be fair, the same comment I quoted also lists "Being able to consider and optimize literary qualities" is one of the major considerations to be balanced. But I think (_I_ think) it's also fair to note that (as we had seen on _Less Wrong_ earlier that year), lip service is cheap. It's easy to _say_, "Of course I don't think politeness is more important than truth," while systematically behaving as if you did.
+To be fair, the same comment I quoted also lists "Being able to consider and optimize literary qualities" is one of the major considerations to be balanced. But I think (_I_ think) it's also fair to note that (as we had seen on _Less Wrong_ earlier that year), lip service is cheap. It's easy to say, "Of course I don't think politeness is more important than truth," while systematically behaving as if you did.
"Broadcast criticism is adversely selected for critic errors," Yudkowsky wrote in the post on reducing negativity, correctly pointing out that if a work's true level of mistakenness is _M_, the _i_-th commenter's estimate of mistakenness has an error term of _E<sub>i</sub>_, and commenters leave a negative comment when their estimate _M_ + _E<sub>i</sub>_ is greater than their threshold for commenting _T<sub>i</sub>_, then the comments that get posted will have been selected for erroneous criticism (high _E<sub>i</sub>_) and commmenter chattiness (low _T<sub>i</sub>_).
-I can imagine some young person who really liked _Harry Potter and the Methods_ being intimidated by the math notation, and uncritically accepting this wisdom from the great Eliezer Yudkowsky as a reason to be less critical, specifically. But a somewhat less young person who isn't intimidated by math should notice that the the math here is just [regression to the mean](https://en.wikipedia.org/wiki/Regression_toward_the_mean). The same argument applies to praise!
+I can imagine some young person who liked _Harry Potter and the Methods_ being intimidated by the math notation, and uncritically accepting this wisdom from the great Eliezer Yudkowsky as a reason to be less critical, specifically. But a somewhat less young person who isn't intimidated by math should notice that this is just [regression to the mean](https://en.wikipedia.org/wiki/Regression_toward_the_mean). The same argument applies to praise!
-What I would hope for from a rationality teacher and a rationality community, would be efforts to instill the _general_ skill of modeling things like regression to the mean and selection effects, as part of the general project of having a discourse that does collective information-processing.
+What I would hope for from a rationality teacher and a rationality community, would be efforts to instill the general skill of modeling things like regression to the mean and selection effects, as part of the general project of having a discourse that does collective information-processing.
-And from the way Yudkowsky writes these days, it looks like he's ... not interested in collective information-processing? Or that he doesn't actually believe that's a real thing? "Credibly helpful unsolicited criticism should be delivered in private," he writes! I agree that the positive purpose of public criticism isn't solely to help the author. (If it were, there would be no reason for anyone but the author to read it.) But readers _do_ benefit from insightful critical commentary. (If they didn't, why would they read the comments section?) When I read a story, and am interested in reading the comments _about_ a story, it's because _I want to know what other readers were actually thinking about the work_. I don't _want_ other people to self-censor comments on any plot holes or [Fridge Logic](https://tvtropes.org/pmwiki/pmwiki.php/Main/FridgeLogic) they noticed for fear of dampening someone else's enjoyment or hurting the author's feelings.
+And from the way Yudkowsky writes these days, it looks like he's ... not interested in collective information-processing? Or that he doesn't actually believe that's a real thing? "Credibly helpful unsolicited criticism should be delivered in private," he writes! I agree that the positive purpose of public criticism isn't solely to help the author. (If it were, there would be no reason for anyone but the author to read it.) But readers _do_ benefit from insightful critical commentary. (If they didn't, why would they read the comments section?) When I read a story, and am interested in reading the comments _about_ a story, it's because I want to know what other readers think about the work. I don't want other people to self-censor comments on any plot holes or [Fridge Logic](https://tvtropes.org/pmwiki/pmwiki.php/Main/FridgeLogic) they noticed for fear of dampening someone else's enjoyment or hurting the author's feelings.
-Yudkowsky claims that criticism should be given in private because then the target "may find it much more credible that you meant only to help them, and weren't trying to gain status by pushing them down in public." I'll buy this as a reason why credibly _altruistic_ unsolicited criticism should be delivered in private. Indeed, meaning _only_ to help the target just doesn't seem like a plausible critic motivation in most cases. But the fact that critics typically have non-altruistic motives, doesn't mean criticism isn't helpful. In order to incentivize good criticism, you _want_ people to be rewarded with status for making good criticisms. You'd have to be some sort of communist to disagree with this![^communism-analogy]
+Yudkowsky claims that criticism should be given in private because then the target "may find it much more credible that you meant only to help them, and weren't trying to gain status by pushing them down in public." I'll buy this as a reason why credibly _altruistic_ unsolicited criticism should be delivered in private.[^altruistic-criticism] Indeed, meaning _only_ to help the target just doesn't seem like a plausible critic motivation in most cases. But the fact that critics typically have non-altruistic motives, doesn't mean criticism isn't helpful. In order to incentivize good criticism, you _want_ people to be rewarded with status for making good criticisms. You'd have to be some sort of communist to disagree with this![^communism-analogy]
+
+[^altruistic-criticism]: Specifically, altruism towards the author. Altruistic benefits to other readers are a reason for criticism to be public.
[^communism-analogy]: That is, there's an analogy between economically valuable labor, and intellectually productive criticism: if you accept the necessity of paying workers money in order to get good labor out of them, you should understand the necessity of awarding commenters status in order to get good criticism out of them.
There's a striking contrast between the Yudkowsky of 2019 who wrote the "Reducing Negativity" post, and an earlier Yudkowsky (from even before the Sequences) who maintained [a page on Crocker's rules](http://sl4.org/crocker.html): if you declare that you operate under Crocker's rules, you're consenting to other people optimizing their speech for conveying information rather than being nice to you. If someone calls you an idiot, that's not an "insult"; they're just informing you about the fact that you're an idiot, and you should probably thank them for the tip. (If you _were_ an idiot, wouldn't you be better off knowing rather than not-knowing?)
-It's of course important to stress that Crocker's rules are _opt in_ on the part of the _receiver_; it's not a license to unilaterally be rude to other people. Adopting Crocker's rules as a community-level norm on an open web forum does not seem like it would end well.
-
-Still, there's something precious about a culture where people appreciate the _obvious normative ideal_ underlying Crocker's rules, even if social animals can't reliably live up to the normative ideal. Speech is for conveying information. People can say things—even things about me or my work—not as a command, or as a reward or punishment, but just to establish a correspondence between words and the world: a map that reflects a territory.
+It's of course important to stress that Crocker's rules are opt in on the part of the receiver; it's not a license to unilaterally be rude to other people. Adopting Crocker's rules as a community-level norm on an open web forum does not seem like it would end well.
-Appreciation of this obvious normative ideal seems strikingly absent from Yudkowsky's modern work—as if he's given up on the idea that using Speech in public in order to reason is useful or possible.
+Still, there's something precious about a culture where people appreciate the obvious normative ideal underlying Crocker's rules, even if social animals can't reliably live up to the normative ideal. Speech is for conveying information. People can say things—even things about me or my work—not as a command, or as a reward or punishment, but just to establish a correspondence between words and the world: a map that reflects a territory.
-The "Reducing Negativity" post also warns against the failure mode of attempted "author telepathy": _attributing_ bad motives to authors and treating those attributions as fact without accounting for uncertainty or distinguishing observations from inferences. I should be explicit, then: when I say negative things about Yudkowsky's state of mind, like it's "as if he's given up on the idea that reasoning in public is useful or possible", that's definitely an inference, not an observation. I definitely don't think Yudkowsky _thinks of himself_ as having given up on Speech _in those words_.
+Appreciation of this obvious normative ideal seems strikingly absent from Yudkowsky's modern work—as if he's given up on the idea that reasoning in public is useful or possible.
-Why attribute motives to people that they don't attribute to themselves, then? Because I need to, in order to make sense of the world. Words aren't imbued with intrinsic "meaning"; just to _interpret_ text entails building some model of the mind on the other side.
+The "Reducing Negativity" post also warns against the failure mode of attempted "author telepathy": attributing bad motives to authors and treating those attributions as fact without accounting for uncertainty or distinguishing observations from inferences. I should be explicit, then: when I say negative things about Yudkowsky's state of mind, like it's "as if he's given up on the idea that reasoning in public is useful or possible", that's a probabilistic inference, not a certain observation.
-The text that Yudkowsky emitted in 2007–2009 made me who I am. The text that Yudkowsky has emitted since at least March 2016 _looks like_ it's being generated by a different and _much less trustworthy_ process. According to the methods I was taught in 2007–2009, I have a _duty_ to notice the difference, and try to make sense of the change—even if I'm not a superhuman neuroscience AI and have no hope of getting it right in detail. And I have a right to try to describe the change I'm seeing to you.
-
-_Good_ criticism is hard. _Accurately_ inferring authorial ["intent"](https://www.lesswrong.com/posts/sXHQ9R5tahiaXEZhR/algorithmic-intent-a-hansonian-generalized-anti-zombie) is much harder. There is certainly no shortage of bullies in the world eager to make _bad_ criticism or _inaccurately_ infer authorial intent in order to achieve their social goals. But I don't think that's a good reason to give up on _trying_ to do good criticism and accurate intent-attribution. If there's any hope for humans to think together and work together, it has to go though distiguishing good criticism from bad criticism, and treating them differently. Suppressing criticism-in-general is intellectual suicide.
+But I think making probabilistic inferences is ... fine? The sentence "Credibly helpful unsolicited criticism should be delivered in private" sure does look to me like text that's likely to have been generated by a state of mind that doesn't believe that reasoning in public is useful or possible. Someone who did believe in public reason would have noticed that criticism has information content separate from its potential to harm an author's reputation or feelings. If you think I'm getting this inference wrong, feel free to let me _and other readers_ know why in the comments.
-----
-On 3 November 2019, I received an interesting reply on my philosophy-of-categorization thesis from MIRI researcher Abram Demski. Abram asked: ideally, shouldn't all conceptual boundaries be drawn with appeal-to-consequences? Wasn't the problem just with bad (motivated, shortsighted) appeals to consequences? Agents categorize in order to make decisions. The best classifer for an application depends on the costs and benefits. As a classic example, it's very important for evolved prey animals to avoid predators, so it makes sense for their predator-detection classifiers to be configured such that they jump away from every rustling in the bushes, even if it's usually not a predator.
+On 3 November 2019, I received an interesting reply on my philosophy-of-categorization thesis from MIRI researcher Abram Demski. Abram asked: ideally, shouldn't all conceptual boundaries be drawn with appeal-to-consequences? Wasn't the problem just with bad (motivated, shortsighted) appeals to consequences? Agents categorize in order to make decisions. The best classifer for an application depends on the costs and benefits. As a classic example, it's important for evolved prey animals to avoid predators, so it makes sense for their predator-detection classifiers to be configured such that they jump away from every rustling in the bushes, even if it's usually not a predator.
-I had thought of the "false-positives are better than false-negatives when detecting predators" example as being about the limitations of evolution as an AI designer: messy evolved animal brains don't bother to track probability and utility separately the way a cleanly-designed AI could. As I had explained in "... Boundaries?", it made sense for _what_ variables you paid attention to, to be motivated by consequences. But _given_ the subspace that's relevant to your interests, you want to run an "epistemically legitimate" clustering algorithm on the data you see there, which depends on the data, not your values. The only reason value-dependent gerrymandered category boundaries seem like a good idea if you're not careful about philosophy is because it's _wireheading_. Ideal probabilistic beliefs shouldn't depend on consequences.
+I had thought of the "false-positives are better than false-negatives when detecting predators" example as being about the limitations of evolution as an AI designer: messy evolved animal brains don't bother to track probability and utility separately the way a cleanly-designed AI could. As I had explained in "... Boundaries?", it made sense for what variables you paid attention to, to be motivated by consequences. But given the subspace that's relevant to your interests, you want to run an "epistemically legitimate" clustering algorithm on the data you see there, which depends on the data, not your values. The only reason value-dependent gerrymandered category boundaries seem like a good idea if you're not careful about philosophy is because it's _wireheading_. Ideal probabilistic beliefs shouldn't depend on consequences.
-Abram didn't think the issue was so clear-cut. Where do "probabilities" come from, in the first place? The reason we expect something like Bayesianism to be an attractor among self-improving agents is because probabilistic reasoning is broadly useful: epistemology can be _derived_ from instrumental concerns. He agreed that severe wireheading issues _potentially_ arise if you allow consequentialist concerns to affect your epistemics.
+Abram didn't think the issue was so clear-cut. Where do "probabilities" come from, in the first place? The reason we expect something like Bayesianism to be an attractor among self-improving agents is because probabilistic reasoning is broadly useful: epistemology can be derived from instrumental concerns. He agreed that severe wireheading issues potentially arise if you allow consequentialist concerns to affect your epistemics.
But the alternative view had its own problems. If your AI consists of a consequentialist module that optimizes for utility in the world, and an epistemic module that optimizes for the accuracy of its beliefs, that's _two_ agents, not one: how could that be reflectively coherent? You could, perhaps, bite the bullet here, for fear that consequentialism doesn't tile and that wireheading was inevitable. On this view, Abram explained, "Agency is an illusion which can only be maintained by crippling agents and giving them a split-brain architecture where an instrumental task-monkey does all the important stuff while an epistemic overseer supervises." Whether this view was ultimately tenable or not, this did show that trying to forbid appeals-to-consequences entirely led to strange places.
------
-Also in November 2019, I wrote to Ben about how I was still stuck on writing the grief-memoir. My _plan_ had been that it should have been possible to tell the story of the Category War while glomarizing about the content of private conversations, then offer Scott and Eliezer pre-publication right of reply (because it's only fair to give your former-hero-current-[frenemies](https://en.wikipedia.org/wiki/Frenemy) warning when you're about to publicly call them intellectually dishonest), then share it to _Less Wrong_ and the /r/TheMotte culture war thread, and then I would have the emotional closure to move on with my life (learn math, go to gym, chop wood, carry water) and not be a mentally-dominated cultist.
+Also in November 2019, I wrote to Ben about how I was still stuck on writing the grief-memoir. My plan had been that it should have been possible to tell the story of the Category War while glomarizing about the content of private conversations, then offer Scott and Eliezer pre-publication right of reply (because it's only fair to give your former-hero-current-[frenemies](https://en.wikipedia.org/wiki/Frenemy) warning when you're about to publicly call them intellectually dishonest), then share it to _Less Wrong_ and the /r/TheMotte culture war thread, and then I would have the emotional closure to move on with my life (learn math, go to gym, chop wood, carry water) and not be a mentally-dominated cultist.
-The reason it _should_ have been safe to write was because Explaining Things Is Good. It should be possible to say, "This is not a social attack; I'm not saying 'rationalists Bad, Yudkowsky Bad'; I'm just trying to carefully _tell the true story_ about why, as a matter of cause-and-effect, I've been upset this year, including addressing counterarguments for why some would argue that I shouldn't be upset, why other people could be said to be behaving 'reasonably' given their incentives, why I nevertheless wish they'd be braver and adhere to principle rather than 'reasonably' following incentives, _&c_."
+The reason it _should_ have been safe to write was because it's good to explain things. It should be possible to say, "This is not a social attack; I'm not saying 'rationalists Bad, Yudkowsky Bad'; I'm just trying to carefully tell the true story about why, as a matter of cause-and-effect, I've been upset this year, including addressing counterarguments for why some would argue that I shouldn't be upset, why other people could be said to be behaving 'reasonably' given their incentives, why I nevertheless wish they'd be braver and adhere to principle rather than 'reasonably' following incentives, _&c_."
-So why couldn't I write? Was it that I didn't know how to make "This is not a social attack" credible? Maybe because ... it wasn't true?? I was afraid that telling a story about our leader being intellectually dishonest was "the nuclear option" in a way that I couldn't credibly cancel with "But I'm just telling a true story about a thing that was important to me that actually happened" disclaimers. If you're slowly-but-surely gaining territory in a conventional war, _suddenly_ escalating to nukes seems pointlessly destructive. This metaphor is horribly non-normative ([arguing is not a punishment!](https://srconstantin.github.io/2018/12/15/argue-politics-with-your-best-friends.html) carefully telling a true story _about_ an argument is not a nuke!), but I didn't know how to make it stably go away.
+So why couldn't I write? Was it that I didn't know how to make "This is not a social attack" credible? Maybe because ... it wasn't true?? I was afraid that telling a story about our leader being intellectually dishonest was "the nuclear option" in a way that I couldn't credibly cancel with "But I'm just telling a true story about a thing that was important to me that actually happened" disclaimers. If you're slowly-but-surely gaining territory in a conventional war, suddenly escalating to nukes seems pointlessly destructive. This metaphor was horribly non-normative ([arguing is not a punishment](https://srconstantin.github.io/2018/12/15/argue-politics-with-your-best-friends.html); carefully telling a true story _about_ an argument is not a nuke), but I didn't know how to make it stably go away.
-A more motivationally-stable compromise would be to try to split off whatever _generalizable insights_ that would have been part of the story into their own posts that don't make it personal. ["Heads I Win, Tails?—Never Heard of Her"](https://www.lesswrong.com/posts/DoPo4PDjgSySquHX8/heads-i-win-tails-never-heard-of-her-or-selective-reporting) had been a huge success as far as I was concerned, and I could do more of that kind of thing, analyzing the social stuff I was worried about, without making it personal, even if, secretly, it actually was personal.
+A more motivationally-stable compromise would be to split off whatever generalizable insights that would have been part of the story into their own posts that didn't make it personal. ["Heads I Win, Tails?—Never Heard of Her"](https://www.lesswrong.com/posts/DoPo4PDjgSySquHX8/heads-i-win-tails-never-heard-of-her-or-selective-reporting) had been a huge success as far as I was concerned, and I could do more of that kind of thing, analyzing the social stuff I was worried about without making it personal (even if, secretly, it was personal).
Ben replied that it didn't seem like it was clear to me that I was a victim of systemic abuse, and that I was trying to figure out whether I was being fair to my abuser. He thought if I could internalize that, I would be able to forgive myself a lot of messiness, which would reduce the perceived complexity of the problem.
-I said I would bite that bullet: yes! Yes, I was trying to figure out whether I was being fair to my abusers, and it was an important question to get right! "Other people's lack of standards harmed me, therefore I don't need to hold myself to standards in my response because I have [extenuating circumstances](https://www.lesswrong.com/posts/XYrcTJFJoYKX2DxNL/extenuating-circumstances)" would be a _lame excuse_.
+I said I would bite that bullet: yes! Yes, I was trying to figure out whether I was being fair to my abusers, and it was an important question to get right! "Other people's lack of standards harmed me, therefore I don't need to hold myself to standards in my response because I have [extenuating circumstances](https://www.lesswrong.com/posts/XYrcTJFJoYKX2DxNL/extenuating-circumstances)" would be a lame excuse.
-(This seemed correlated with the recurring stalemated disagreement within our coordination group, where Michael/Ben/Jessica would say, "Fraud, if that word _ever_ meant anything", and while I agreed that they were pointing to an important way in which things were messed up, I was still sympathetic to the Caliphate-defender's reply that this usage of "fraud" was motte-and-baileying between vastly different senses of _fraud_; I wanted to do _more work_ to formulate a _more precise theory_ of the psychology of deception to describe exactly how things are messed up a way that wouldn't be susceptible to the motte-and-bailey charge.)
+This seemed correlated with the recurring stalemated disagreement within our coordination group, where Michael/Ben/Jessica would say, "Fraud, if the word ever meant anything", and while I agreed that they were pointing to an important pattern of false-representations-optimized-to-move-resources, I was still sympathetic to the Caliphate-defender's reply that this usage of "fraud" was motte-and-baileying between different senses of _fraud_. (Most people would say that the things we were alleging MIRI and CfAR had done wrong were distinct from the things Enron and Bernie Madoff had done wrong.) I wanted to do _more work_ to formulate a more precise theory of the psychology of deception to describe exactly how things were messed up a way that wouldn't be susceptible to the motte-and-bailey charge.
-------
-On 12 and 13 November 2019, Ziz [published](https://archive.ph/GQOeg) [several](https://archive.ph/6HsvS) [blog](https://archive.ph/jChxP) [posts](https://archive.ph/TPei9) laying out her greviances against MIRI and CfAR. On the fifteenth, Ziz and three collaborators staged a protest at the CfAR reunion being held at a retreat center in the North Bay near Camp Meeker. A call to the police falsely alleged that the protestors had a gun, [resulting in a](http://web.archive.org/web/20230316210946/https://www.pressdemocrat.com/article/news/deputies-working-to-identify-suspects-in-camp-meeker-incident/) [dramatic police reaction](http://web.archive.org/web/20201112041007/https://www.pressdemocrat.com/article/news/authorities-id-four-arrested-in-westminster-woods-protest/) (SWAT team called, highway closure, children's group a mile away being evacuated—the works).
+On 12 and 13 November 2019, Ziz [published](https://archive.ph/GQOeg) [several](https://archive.ph/6HsvS) [blog](https://archive.ph/jChxP) [posts](https://archive.ph/TPei9) laying out [her](/2019/Oct/self-identity-is-a-schelling-point/) greviances against MIRI and CfAR. On the fifteenth, Ziz and three collaborators staged a protest at the CfAR reunion being held at a retreat center in the North Bay near Camp Meeker. A call to the police falsely alleged that the protestors had a gun, [resulting in a](http://web.archive.org/web/20230316210946/https://www.pressdemocrat.com/article/news/deputies-working-to-identify-suspects-in-camp-meeker-incident/) [dramatic police reaction](http://web.archive.org/web/20201112041007/https://www.pressdemocrat.com/article/news/authorities-id-four-arrested-in-westminster-woods-protest/) (SWAT team called, highway closure, children's group a mile away being evacuated—the works).
-I was tempted to email the links to the blog posts to the Santa Rosa _Press-Democrat_ reporter covering the incident (as part of my information-sharing is Good virtue ethics), but decided to refrain because I predicted that Anna would prefer I didn't.
+I was tempted to email links to the blog posts to the Santa Rosa _Press-Democrat_ reporter covering the incident (as part of my information-sharing-is-good virtue ethics), but decided to refrain because I predicted that Anna would prefer I didn't.
The main relevance of this incident to my Whole Dumb Story is that Ziz's memoir–manifesto posts included [a 5500 word section about me](https://archive.ph/jChxP#selection-1325.0-1325.4). Ziz portrays me as a slave to social reality, throwing trans women under the bus to appease the forces of cissexism. (I mostly don't think that's what's going on with me, but I can see why the theory was appealing.) I was flattered that someone had so much to say about me, even if I was being portrayed negatively.
--------
-I had an interesting interaction with Somni, one of the "Meeker Four"—presumably out on bail at this time?—on 12 December 2019.
+I had an interesting interaction with [Somni](https://somnilogical.tumblr.com/), one of the "Meeker Four"—presumably out on bail at this time?—on 12 December 2019.
-I told her, from a certain perspective, it's surprising that you spend so much time complaining about CfAR, Anna Salamon, Kelsey Piper, _&c._, but _I_ seemed to get along fine with her—because "naïvely", one would think that my views were so much worse. Was I getting a pity pass because she thought false consciousness was causing me to act against my own transfem class interests? Or what?
+I told her, from a certain perspective, it's surprising that she spent so much time complaining about CfAR, Anna Salamon, Kelsey Piper, _&c._, but _I_ seemed to get along fine with her—because naïvely, one would think that my views were so much worse. Was I getting a pity pass because she thought false consciousness was causing me to act against my own transfem class interests? Or what?
-In order to be absolutely clear about my terrible views, I said that I was privately modeling a lot of transmisogyny complaints as something like—a certain neurotype-cluster of non-dominant male is latching onto locally-ascendant social-justice ideology in which claims to victimhood can be leveraged into claims to power. Traditionally, men are moral agents, but not patients; women are moral patients, but not agents. If weird non-dominant men aren't respected if identified as such (because low-ranking males aren't valuable allies, and don't have intrinsic moral patiency of women), but _can_ get victimhood/moral-patiency points for identifying as oppressed transfems, that creates an incentive gradient for them to do so, and no one was allowed to notice this except me, because everyone prefers to stay on the good side of social-justice ideology unless they have Something to Protect that requires defying it.
+In order to be absolutely clear about my terrible views, I said that I was privately modeling a lot of transmisogyny complaints as something like—a certain neurotype-cluster of non-dominant male is latching onto locally-ascendant social-justice ideology in which claims to victimhood can be leveraged into claims to power. Traditionally, men are moral agents, but not patients; women are moral patients, but not agents. If weird non-dominant men aren't respected if identified as such (because low-ranking males aren't valuable allies, and don't have intrinsic moral patiency of women), but _can_ get victimhood/moral-patiency points for identifying as oppressed transfems, that creates an incentive gradient for them to do so, and no one was allowed to notice this except me, because everybody [who's anybody](https://thezvi.wordpress.com/2019/07/02/everybody-knows/) prefers to stay on the good side of social-justice ideology unless they have Something to Protect that requires defying it.
-Somni said that it was because I was being victimized by the same forces of gaslighting, and that I wasn't lying about my agenda. Maybe she _should_ be complaining about me?—but I seemed to be following a somewhat earnest epistemic process, whereas Kelsey, Scott, and Anna were not. If I were to start going, "Here's my rationality org; rule #1: no transfems (except me); rule #2, no telling people about rule #1", then she would talk about it.
+Somni said that it was because I was being victimized by the same forces of gaslighting and that I wasn't lying about my agenda. Maybe she _should_ be complaining about me?—but I seemed to be following a somewhat earnest epistemic process, whereas Kelsey, Scott, and Anna were not. If I were to start going, "Here's my rationality org; rule #1: no transfems (except me); rule #2, no telling people about rule #1", then she would talk about it.
I would later remark to Anna that Somni and Ziz saw themselves as being oppressed by people's hypocritical and manipulative social perceptions and behavior. Merely using the appropriate language ("Somni ... she", _&c._) protected her against threats from the Political Correctness police, but it actually didn't protect against threats from _them_. It was as if the mere fact that I wasn't optimizing for PR (lying about my agenda, as Somni said) was what made me not a direct enemy (although still a collaborator) in their eyes.
--------
-I had a phone call with Michael in which he took issue with Anna having described Ziz as having threatened to kill Gwen, when that wasn't a fair paraphrase of what Ziz's account actually said.[^ziz-gwen-account] In Michael's view, this was tantamount to indirect attempted murder using the State as a weapon to off her organization's critics: Anna casting Ziz as a Scary Bad Guy in [the improv scene of social reality](https://www.lesswrong.com/posts/AqbWna2S85pFTsHH4/the-intelligent-social-web) is the kind of maneuver that contributes to the legal system ruining weird people's lives with spurious charges because weird gets [cast as villains in the act](https://unstableontology.com/2018/11/17/act-of-charity/).
+I had a phone call with Michael in which he took issue with Anna having described Ziz as having threatened to kill Gwen, when that wasn't a fair paraphrase of what Ziz's account actually said.[^ziz-gwen-account] In Michael's view, this was tantamount to indirect attempted murder using the State as a weapon to off her organization's critics: Anna casting Ziz as a Scary Bad Guy in [the improv scene of social reality](https://www.lesswrong.com/posts/AqbWna2S85pFTsHH4/the-intelligent-social-web) is the kind of maneuver that contributes to the legal system ruining weird people's lives with spurious charges, because weird gets [cast as villains in the act](https://unstableontology.com/2018/11/17/act-of-charity/).
[^ziz-gwen-account]: The relevant passage from [one of Ziz's memoir posts](https://archive.ph/an5rp#selection-419.0-419.442) is:
I told Michael that this made me think I might need to soul-search about having been complicit with injustice, but I couldn't clearly articulate why.
-I figured it out later (Subject: "complicity and friendship"). I think part of my emotional reaction to finding out about Ziz's legal trouble was the hope that it would lead to less pressure on Anna. I had been nagging Anna a lot on the theme of "rationality actually requires free speech", and she would sometimes defend her policy of guardedness on the grounds of (my paraphrase—), "Hey, give me some credit, oftentimes I do take a calculated risk of telling people things. Or I did, but then ... Ziz."
+I figured it out later (Subject: "complicity and friendship"). I think part of my emotional reaction to finding out about Ziz's legal trouble was the hope that it would lead to less pressure on Anna. I had been nagging Anna a lot on the theme of "rationality actually requires free speech", and she would sometimes defend her policy of guardedness on the grounds of (my paraphrase:), "Hey, give me some credit, oftentimes I do take a calculated risk of telling people things. Or I did, but then ... Ziz."
I think at some level, I was imagining being able to tell Anna, "See, you were so afraid that telling people things would make enemies, and you used Ziz as evidence that you weren't cautious enough. But look, Ziz _isn't going to be a problem for you anymore_. Your fear of making enemies actually happened, and you're fine! This is evidence in favor of my view that you were far too cautious, rather than not being cautious enough!"
-But that was complicit with injustice, because the _reason_ I felt that Ziz wasn't going to be a problem for Anna anymore was because Ziz's protest got SWATted, which didn't have anything to do with the merits of Ziz's claims against Anna. I still wanted Anna to feel safer to speak, but I now realized that more specifically, I wanted Anna to feel safe _because_ Speech can actually win. Feeling safe because one's enemies can be crushed by the state wasn't the same thing.
+But that was complicit with injustice, because the _reason_ I felt that Ziz wasn't going to be a problem for Anna anymore was because Ziz's protest ran afoul of the cops, which didn't have anything to do with the merits of Ziz's claims against Anna. I still wanted Anna to feel safer to speak, but I now realized that more specifically, I wanted Anna to feel safe _because_ Speech can actually win. Feeling safe because one's enemies can be crushed by the state wasn't the same thing.
--------
[^defensive]: Criticism is "defensive" in the sense of trying to _prevent_ new beliefs from being added to our shared map; a critic of an idea "wins" when the idea is not accepted (such that the set of accepted beliefs remains at the _status quo ante_).
-More significantly, in reaction to Yudkowsky's ["Meta-Honesty: Firming Up Honesty Around Its Edge Cases"](https://www.lesswrong.com/posts/xdwbX9pFEr7Pomaxv/meta-honesty-firming-up-honesty-around-its-edge-cases), I published ["Firming Up Not-Lying Around Its Edge-Cases Is Less Broadly Useful Than One Might Initially Think"](https://www.lesswrong.com/posts/MN4NRkMw7ggt9587K/firming-up-not-lying-around-its-edge-cases-is-less-broadly), explaining why merely refraining from making false statments is an unproductively narrow sense of "honesty", because the ambiguity of natural language makes it easy to deceive people in practice without technically lying. (The ungainly title of my post was "softened" from an earlier draft following feedback from the posse; I had originally written "... Surprisingly Useless".)
-
-I thought this one cut to the heart of the shocking behavior that we had seen from Yudkowsky lately. (Less shocking as the months rolled on, and I told myself to let the story end.) The "hill of meaning in defense of validity" affair had been been driven by Yudkowsky's pathological obsession with not-technically-lying, on two levels: he had proclaimed that asking for new pronouns "Is. Not. Lying." (as if _that_ were the matter that anyone cared about—as if conservatives and gender-critical feminists should just pack up and go home after it had been demonstrated that trans people aren't _lying_), and he had seen no interest in clarifying his position on the philosophy of language, because he wasn't lying when he said that preferred pronouns weren't lies (as if _that_ were the matter that my posse cared about—as if I should keep honoring him as my Caliph after it had been demonstrated that he hadn't _lied_). But his Sequences had articulated a _higher standard_ than merely not-lying. If he didn't remember, I could at least hope to remind everyone else.
-
-I also wrote a little post on 20 December 2019, ["Free Speech and Triskadekaphobic Calculators: A Reply to Hubinger on the Relevance of Public Online Discussion to Existential Risk"](https://www.lesswrong.com/posts/yaCwW8nPQeJknbCgf/free-speech-and-triskaidekaphobic-calculators-a-reply-to).
-
-Wei Dai had written ["Against Premature Abstraction of Political Issues"](https://www.lesswrong.com/posts/bFv8soRx6HB94p5Pg/against-premature-abstraction-of-political-issues)—itself plausibly an abstraction inspired by my philosophy-of-language blogging?—and had cited a clump of _Less Wrong_ posts about gender and pick-up artistry back in 'aught-nine as a successful debate that would have been harder to have if everyone had to obsfuscate the concrete topics of interest.
-
-A MIRI researcher, Evan Hubinger, asked:
-
-> Do you think having that debate online was something that needed to happen for AI safety/x-risk? Do you think it benefited AI safety at all? I'm genuinely curious. My bet would be the opposite—that it caused AI safety to be more associated with political drama that helped further taint it.
-
-In my reply post, I claimed that our belief that AI safety was the most important problem in the world was causally downstream from from people like Yudkowsky and Nick Bostrom trying to do good reasoning, and following lines of reasoning to where they led. The [cognitive algorithm](https://www.lesswrong.com/posts/HcCpvYLoSFP4iAqSz/rationality-appreciating-cognitive-algorithms) of assuming that your current agenda was the most important thing, and then distorting the process of inquiry to preserve its political untaintedness wouldn't have led us to _noticing_ the alignment problem, and I didn't think it would be sufficient to solve it.
-
-In some sense, it should be easier to have a rationality/alignment community that _just_ does systematically correct reasoning, rather than a politically-savvy community that does systematically correct reasoning _except_ when that would taint AI safety with political drama, analogously to how it's easier to build a calculator that just does correct arithmetic, than a calculator that does correct arithmetic _except_ that it never displays the result 13.
-
-In order to build a "triskadekaphobic calculator", you would need to "solve arithmetic" anyway, and the resulting product would be limited not only in its ability to correctly compute `6 + 7`, but also the infinite family of calculations that included 13 as an intermediate result: if you can't count on `(6 + 7) + 1` being the same as `6 + (7 + 1)`, you lose the associativity of addition. And so on. (I had the "calculator that won't display 13" analogy cached from previous email correspondence.)
-
-It could have been a comment instead of a top-level post, but I wanted to bid for the extra attention. I think, at some level, putting Hubinger's name in the post title was deliberate. It wasn't inappropriate—"Reply to Author's Name on Topic Name" is a very standard academic title format, [which](/2016/Oct/reply-to-ozy-on-agp/) [I](/2016/Nov/reply-to-ozy-on-two-type-mtf-taxonomy/) [often](/2019/Dec/reply-to-ozymandias-on-fully-consensual-gender/) [use](/2018/Apr/reply-to-the-unit-of-caring-on-adult-human-females/) [myself](https://www.lesswrong.com/posts/aJnaMv8pFQAfi9jBm/reply-to-nate-soares-on-dolphins)—but it also wasn't necessary, and might have been a little weird given that I was mostly using Hubinger's comment as a jumping-off point for my Free Speech for Shared Maps campaign, rather than responding point-by-point to a longer piece Hubinger might have written. It's as if the part of my brain that chose that subtitle wanted to set an example, that arguing for cowardice, being in favor of concealing information for fear of being singled out by a mob, would just get you singled out _more_.
+More significantly, in reaction to Yudkowsky's ["Meta-Honesty: Firming Up Honesty Around Its Edge Cases"](https://www.lesswrong.com/posts/xdwbX9pFEr7Pomaxv/meta-honesty-firming-up-honesty-around-its-edge-cases), I published ["Firming Up Not-Lying Around Its Edge-Cases Is Less Broadly Useful Than One Might Initially Think"](https://www.lesswrong.com/posts/MN4NRkMw7ggt9587K/firming-up-not-lying-around-its-edge-cases-is-less-broadly),[^not-lying-title] explaining why merely refraining from making false statments is an unproductively narrow sense of "honesty", because the ambiguity of natural language makes it easy to deceive people in practice without technically lying.
-I had [an exchange with Scott Alexander in the comment section](https://www.greaterwrong.com/posts/yaCwW8nPQeJknbCgf/free-speech-and-triskaidekaphobic-calculators-a-reply-to/comment/JdsknCuCuZMAo8EbP).
+[^not-lying-title]: The ungainly title was "softened" from an earlier draft following feedback from the posse; I had originally written "... Surprisingly Useless".
-"I know a bunch of people in academia who do various verbal gymnastics to appease the triskaidekaphobics, and when you talk to them in private they get everything 100% right," [he said](https://www.lesswrong.com/posts/yaCwW8nPQeJknbCgf/free-speech-and-triskaidekaphobic-calculators-a-reply-to?commentId=mHrHTvzg8MGNH2CwB) (in a follow-up comment on 5 January 2020).
+I thought this one cut to the heart of the shocking behavior that we had seen from Yudkowsky lately. (Less shocking as the months rolled on, and I told myself to let the story end.) The "hill of meaning in defense of validity" affair had been been driven by Yudkowsky's pathological obsession with not-technically-lying, on two levels: he had proclaimed that asking for new pronouns "Is. Not. Lying." (as if _that_ were the matter that anyone cared about—as if conservatives and gender-critical feminists should just pack up and go home after it had been demonstrated that trans people aren't _lying_), and he had seen no interest in clarifying his position on the philosophy of language, because he wasn't lying when he said that preferred pronouns weren't lies (as if _that_ were the matter that my posse cared about—as if I should keep honoring him as my Caliph after it had been demonstrated that he hadn't _lied_). But his Sequences had [articulated a higher standard](https://www.lesswrong.com/posts/9f5EXt8KNNxTAihtZ/a-rational-argument) than merely not-lying. If he didn't remember, I could at least hope to remind everyone else.
-I'm happy for them, I replied, but I thought the _point_ of having taxpayer-funded academic departments was so that people who _aren't_ insider experts can have accurate information with which to inform decisions?
+I also wrote a little post on 20 December 2019, ["Free Speech and Triskadekaphobic Calculators"](https://www.lesswrong.com/posts/yaCwW8nPQeJknbCgf/free-speech-and-triskaidekaphobic-calculators-a-reply-to), arguing that it should be easier to have a rationality/alignment community that _just_ does systematically correct reasoning, rather than a politically-savvy community that does systematically correct reasoning _except_ when that would taint AI safety with political drama, analogously to how it's easier to build a calculator that just does correct arithmetic, than a calculator that does correct arithmetic _except_ that it never displays the result 13. In order to build a "triskadekaphobic calculator", you would need to "solve arithmetic" anyway, and the resulting product would be limited not only in its ability to correctly compute `6 + 7`, but also the infinite family of calculations that included 13 as an intermediate result: if you can't count on `(6 + 7) + 1` being the same as `6 + (7 + 1)`, you lose the associativity of addition. And so on.
-----
projects / Ultimately_Untrue_Thought.git / commitdiff