From 09b84cacb9de4c8cae5ab24b2227e4bc301c4662 Mon Sep 17 00:00:00 2001 From: "M. Taylor Saotome-Westlake" Date: Sun, 8 Dec 2019 12:10:09 -0800 Subject: [PATCH] "I Tell Myself": the canary in the discourse algorithm --- notes/i-tell-myself-notes.txt | 3 --- notes/i-tell-myself-sections.md | 7 +++++++ 2 files changed, 7 insertions(+), 3 deletions(-) diff --git a/notes/i-tell-myself-notes.txt b/notes/i-tell-myself-notes.txt index 067d5d9..5a22041 100644 --- a/notes/i-tell-myself-notes.txt +++ b/notes/i-tell-myself-notes.txt @@ -87,8 +87,6 @@ _not_ gaslight me about the most important thing in my life? I don't think I'm setting my price for joining particularly high here? -if you're doing systematically correct reasoning, you should be able to get the right answer even on things that don't matter - There could be similarly egregious errors that I'm not as sensitive too I don't think you can build an aligned superintelligence from a culture this crazy @@ -535,4 +533,3 @@ https://twitter.com/ESYudkowsky/status/1096769579362115584 Ooh, how I wish they'd want me to stay. - diff --git a/notes/i-tell-myself-sections.md b/notes/i-tell-myself-sections.md index 7fbd40b..cd02ad1 100644 --- a/notes/i-tell-myself-sections.md +++ b/notes/i-tell-myself-sections.md @@ -99,6 +99,10 @@ Someone asked me: "Wouldn't it be embarrassing if the community solved Friendly But the _reason_ it seemed _at all_ remotely plausible that our little robot cult could be pivotal in creating Utopia forever was _not_ "[Because we're us](http://benjaminrosshoffman.com/effective-altruism-is-self-recommending/), the world-saving good guys", but rather _because_ we were going to discover and refine the methods of _systematically correct reasoning_. +If you're doing systematically correct reasoning, you should be able to get the right answer even when the question _doesn't matter_. Obviously, the safety of the world does not directly depend on being able to think clearly about trans issues. In the same way, the safety of a coal mine for humans does not _directly_ depend on [whether it's safe for canaries](https://en.wiktionary.org/wiki/canary_in_a_coal_mine). + +The "discourse algorithm" (the collective generalization of "cognitive algorithm") that can't just _get this shit right_ in 2019 (because being out of step with the reigning Bay Area ideological fashion is deemed too expensive by a consequentialist algorithm that counts unpopularity as a cost), also can't get heliocentrism right in 1632 _for the same reason_—and I really doubt it can get AI alignment theory right in 2039. + If the people _marketing themselves_ as the good guys who are going to save the world using systematically correct reasoning are _not actually interested in doing systematically correct reasoning_ (because systematically correct reasoning leads to two or three conclusions that are politically "impossible" to state clearly in public, and no one has the guts to [_not_ shut up and thereby do the politically impossible](https://www.lesswrong.com/posts/nCvvhFBaayaXyuBiD/shut-up-and-do-the-impossible)), that's arguably _worse_ than the situation where the community doesn't exist at all. ----- @@ -184,3 +188,6 @@ I don't doubt Serano's report of her own _experiences_. But "it became obvious t ----- [You "can't" define a word any way you want, or you "can"—what actually matters is the math] + +---- + -- 2.17.1