So far, I've been writing from the perspective of caring (and expecting Yudkowsky to care) about human rationality as a cause in its own right—about wanting to make sense, and wanting to live in a Society that made sense, for its own sake, not as a convergently instrumental subgoal of saving the world.
That's pretty much always where I've been at. I _never_ wanted to save the world. I got sucked in to this robot cult because Yudkowsky's philsophy-of-science blogging was just that good. I did do a little bit of work for the Singularity Institute back in the day (a "we don't pay you, but you can sleep in the garage" internship in 2009, some data-entry-like work manually adding Previous/Next links to the Sequences, designing several PowerPoint presentations for Anna Salamon, writing some Python scripts to organize their donor database), but that was because it was my social tribe and I had connections. To the extent that I took at all seriously the whole save/destroy/take-over the world part (about how we needed to encode all of human morality into a recursively self-improving artificial intelligence to determine our entire future light cone until the end of time), I was scared rather than enthusiastic.
So far, I've been writing from the perspective of caring (and expecting Yudkowsky to care) about human rationality as a cause in its own right—about wanting to make sense, and wanting to live in a Society that made sense, for its own sake, not as a convergently instrumental subgoal of saving the world.
That's pretty much always where I've been at. I _never_ wanted to save the world. I got sucked in to this robot cult because Yudkowsky's philsophy-of-science blogging was just that good. I did do a little bit of work for the Singularity Institute back in the day (a "we don't pay you, but you can sleep in the garage" internship in 2009, some data-entry-like work manually adding Previous/Next links to the Sequences, designing several PowerPoint presentations for Anna Salamon, writing some Python scripts to organize their donor database), but that was because it was my social tribe and I had connections. To the extent that I took at all seriously the whole save/destroy/take-over the world part (about how we needed to encode all of human morality into a recursively self-improving artificial intelligence to determine our entire future light cone until the end of time), I was scared rather than enthusiastic.