"I Tell Myself": the canary in the discourse algorithm

diff --git a/notes/i-tell-myself-notes.txt b/notes/i-tell-myself-notes.txt
index 067d5d9..5a22041 100644 (file)
--- a/notes/i-tell-myself-notes.txt
+++ b/notes/i-tell-myself-notes.txt
@@ -87,8 +87,6 @@ _not_ gaslight me about the most important thing in my life?
 
 I don't think I'm setting my price for joining particularly high here?
 
-if you're doing systematically correct reasoning, you should be able to get the right answer even on things that don't matter
-
 There could be similarly egregious errors that I'm not as sensitive too
 
 I don't think you can build an aligned superintelligence from a culture this crazy
@@ -535,4 +533,3 @@ https://twitter.com/ESYudkowsky/status/1096769579362115584
 
 Ooh, how I wish they'd want me to stay.
 
-

diff --git a/notes/i-tell-myself-sections.md b/notes/i-tell-myself-sections.md
index 7fbd40b..cd02ad1 100644 (file)
--- a/notes/i-tell-myself-sections.md
+++ b/notes/i-tell-myself-sections.md
@@ -99,6 +99,10 @@ Someone asked me: "Wouldn't it be embarrassing if the community solved Friendly
 
 But the _reason_ it seemed _at all_ remotely plausible that our little robot cult could be pivotal in creating Utopia forever was _not_ "[Because we're us](http://benjaminrosshoffman.com/effective-altruism-is-self-recommending/), the world-saving good guys", but rather _because_ we were going to discover and refine the methods of _systematically correct reasoning_.
 
+If you're doing systematically correct reasoning, you should be able to get the right answer even when the question _doesn't matter_. Obviously, the safety of the world does not directly depend on being able to think clearly about trans issues. In the same way, the safety of a coal mine for humans does not _directly_ depend on [whether it's safe for canaries](https://en.wiktionary.org/wiki/canary_in_a_coal_mine).
+
+The "discourse algorithm" (the collective generalization of "cognitive algorithm") that can't just _get this shit right_ in 2019 (because being out of step with the reigning Bay Area ideological fashion is deemed too expensive by a consequentialist algorithm that counts unpopularity as a cost), also can't get heliocentrism right in 1632 _for the same reason_—and I really doubt it can get AI alignment theory right in 2039.
+
 If the people _marketing themselves_ as the good guys who are going to save the world using systematically correct reasoning are _not actually interested in doing systematically correct reasoning_ (because systematically correct reasoning leads to two or three conclusions that are politically "impossible" to state clearly in public, and no one has the guts to [_not_ shut up and thereby do the politically impossible](https://www.lesswrong.com/posts/nCvvhFBaayaXyuBiD/shut-up-and-do-the-impossible)), that's arguably _worse_ than the situation where the community doesn't exist at all.
 
 -----
@@ -184,3 +188,6 @@ I don't doubt Serano's report of her own _experiences_. But "it became obvious t
 -----
 
 [You "can't" define a word any way you want, or you "can"—what actually matters is the math]
+
+----
+