+Some readers who aren't part of my robot cult—and maybe some who are but didn't drink as many cups of the Kool-Aid as I did—might be puzzled at why I've been _so freaked out_ for _an entire year_ (!?!) by people being wrong about philosophy. And for almost anyone else in the world, I would just shrug, [set the bozo bit](https://en.wikipedia.org/wiki/Bozo_bit#Dismissing_a_person_as_not_worth_listening_to), and move on with my day. But when the _universally-acknowledged leading thinkers of my robot cult_ do it ...
+
+Even people who aren't religious still have the same [species-typical psychological mechanisms](https://www.lesswrong.com/posts/Cyj6wQLW6SeF6aGLy/the-psychological-unity-of-humankind) that make religions work. The systematically-correct-reasoning community had come to fill [a similar niche in my psychology](https://www.lesswrong.com/posts/PMr6f7ZocEWFtCYXj/is-humanism-a-religion-substitute) [as a religion](https://www.lesswrong.com/posts/p5DmraxDmhvMoZx8J/church-vs-taskforce). I knew this, but the _hope_ was that this wouldn't come with the pathologies of a religion, because _our_ pseudo-religion was _about_ the rules of systematically correct reasoning. The system is _supposed_ to be self-correcting: if people are obviously, _demonstratably_ wrong, all you have to do is show them the argument that they're wrong, and then they'll understand the obvious argument and change their minds.
+
+So to get a sense of the emotional impact here, imagine being a devout Catholic hearing their local priest deliver a sermon that _blatantly_ contradicts something said in the Bible—or at least, will predictably be interpreted by the typical parishioner as contradicting the obvious meaning of the Bible, even if the sermon also admits some contrived interpretation that's _technically_ compatible with the Bible. Suppose it's an ever-so-slightly-alternate-history 2014, and the sermon suggests that Christians who oppose same-sex marriage have no theological ground to stand on.
+
+You _know_ this is wrong. Okay, maybe there's _some_ way that same-sex marriage could be compatible with the Church's teachings. But you would have to _argue_ for that; you _can't_ just say there's no arguments _against_ it and call that the end of the discussion! [1 Corinthians 6:9–10](https://www.biblegateway.com/passage/?search=1+Corinthians+6%3A9-10&version=NKJV): "Do you not know that the unrighteous will not inherit the kingdom of God? Do not be deceived. Neither fornicators [...] nor homosexuals, nor sodomites [...] will inherit the kingdom of God." It's _right there_. There's [a bunch of passages like that](https://www.livingout.org/the-bible-and-ssa). You _can't possibly_ not see it.
+
+As a man of faith and loyal parishioner, you would _expect_ to be able to resolve the matter by bringing your concern to the priest, who would then see how the sermon had been accidentally misleading, and issue a clarification at next week's sermon, so that the people would not be led astray from the path of God.
+
+The priest doesn't agree; he insists on the contrived technically-not-heresy interpretation. This would be a shock, but it wouldn't, yet, shatter your trust in the Church as an institution. Even the priest is still a flawed mortal man.
+
+Then the Pope misinterets the Bible in the same way in his next encyclical. With the help of some connections, you appeal your case all the way to the Vatican—and the Pope himself comes back with the same _bullshit_ technically-not-heresy.
+
+You realize that you _cannot take the Pope's words literally_.
+
+That would be _pretty upsetting_, right? To lose faith in, not your religion itself—_obviously_ the son of God still died for our sins—but the _institution_ that claims to faithfully implement your religion, but is actually doing something else. You can understand why recovering from that might take a year or so.
+
+(Alternate-alternate title for this post: "[37](https://www.lesswrong.com/posts/FaJaCgqBKphrDzDSj/37-ways-that-words-can-be-wrong) Theses".)
+
+Or maybe imagine an idealistic young lawyer working for the prosecution in the [Selective Draft Law Cases](https://en.wikipedia.org/wiki/Selective_Draft_Law_Cases) challenging the World War I draft. Since 1865, the Constitution _says_, "Neither slavery nor involuntary servitude, except as a punishment for crime whereof the party shall have been duly convicted, shall exist within the United States, or any place subject to their jurisdiction." If the words "involuntary servitude not as a punishment for a crime" _mean anything_, they surely include the draft. So the draft is unconstitutional. Right?
+
+Someone asked me: "Wouldn't it be embarrassing if the community solved Friendly AI and went down in history as the people who created Utopia forever, and you had rejected it because of gender stuff?"
+
+But the _reason_ it seemed _at all_ remotely plausible that our little robot cult could be pivotal in creating Utopia forever was _not_ "[Because we're us](http://benjaminrosshoffman.com/effective-altruism-is-self-recommending/), the world-saving good guys", but rather _because_ we were going to discover and refine the methods of _systematically correct reasoning_.
+
+**If you're doing systematically correct reasoning, you should be able to get the right answer even when the question _doesn't matter_.** Obviously, the safety of the world does not _directly_ depend on being able to think clearly about trans issues. In the same way, the safety of a coal mine for humans does not _directly_ depend on [whether it's safe for canaries](https://en.wiktionary.org/wiki/canary_in_a_coal_mine): the dead canaries are just _evidence about_ properties of the mine relevant to human health.
+
+The "discourse algorithm" (the collective generalization of "cognitive algorithm") that can't just _get this shit right_ in 2019 (because being out of step with the reigning Bay Area ideological fashion is deemed too expensive by a consequentialism that counts unpopularity as a cost), also can't get heliocentrism right in 1632 _for the same reason_—and I really doubt it can get AI alignment theory right in 2039.
+
+If the people _marketing themselves_ as the good guys who are going to save the world using systematically correct reasoning are _not actually interested in doing systematically correct reasoning_ (because systematically correct reasoning leads to two or three conclusions that are politically "impossible" to state clearly in public, and no one has the guts to [_not_ shut up and thereby do the politically impossible](https://www.lesswrong.com/posts/nCvvhFBaayaXyuBiD/shut-up-and-do-the-impossible)), that's arguably _worse_ than the situation where "the community" _qua_ community doesn't exist at all.
+
+[...]
+