This seemed correlated with the recurring stalemated disagreement within our coordination group, where Michael/Ben/Jessica would say, "Fraud, if the word ever meant anything", and while I agreed that they were pointing to an important pattern of false representations optimized to move resources, I was still sympathetic to the Caliphate-defender's reply that this usage of "fraud" was motte-and-baileying between different senses of _fraud_. (Most people would say that the things we were alleging MIRI and CfAR had done wrong were qualitatively different from the things Enron and Bernie Madoff had done wrong.) I wanted to do _more work_ to formulate a more precise theory of the psychology of deception to describe exactly how things were messed up a way that wouldn't be susceptible to the motte-and-bailey charge.
+Looking back four years later, I still feel that way—but my desire for nuance itself demands nuance.
+
+[TODO— FTX and nuance epilogue—
+ * On the one hand, I think I'm right to worry about the "jump to evaluation" failure mode, where you substitute a compressed hostile description
+ * If Gloria does a crime and lies about it and you call her a fraud, people are going to correctly notice that your description failed to match reality; you're obscuring what's actually bad about it
+ * On the other hand, I want to give the posse's worldview credit
+ * In April 2019, Ben tried to describe the Blight to me, saying, "People are systematically conflating corruption, accumulation of dominance, and theft, with getting things done"
+ * ordinary grown-up EAs would describe this as uncharitable, rude, &c.
+ * But look at the FTX blowup. This was, actually, one of the greatest financial frauds of our time, and it was made possible by EA: Lewis's book explains that early Alameda recruited from movement EAs, https://thezvi.wordpress.com/2023/10/24/book-review-going-infinite/
+ * Ordinary grown-ups say, "How could we have known?", but this is much less surprising on Ben's view of what EA is. (And doing good as a charity is even harder than running a crypto exchange, where the job you're doing for stakeholders is more legible; if a health intervention doesn't owrk)
+]
+
-------
On 12 and 13 November 2019, Ziz [published](https://archive.ph/GQOeg) [several](https://archive.ph/6HsvS) [blog](https://archive.ph/jChxP) [posts](https://archive.ph/TPei9) laying out [her](/2019/Oct/self-identity-is-a-schelling-point/) grievances against MIRI and CfAR. On the fifteenth, Ziz and three collaborators staged a protest at the CfAR reunion being held at a retreat center in the North Bay near Camp Meeker. A call to the police falsely alleged that the protesters had a gun, [resulting in a](http://web.archive.org/web/20230316210946/https://www.pressdemocrat.com/article/news/deputies-working-to-identify-suspects-in-camp-meeker-incident/) [dramatic police reaction](http://web.archive.org/web/20201112041007/https://www.pressdemocrat.com/article/news/authorities-id-four-arrested-in-westminster-woods-protest/) (SWAT team called, highway closure, children's group a mile away being evacuated—the works).
Which brings me to the second reason the naïve anti-extortion argument might fail: [what counts as "extortion" depends on the relevant "property rights", what the "default" action is](https://www.lesswrong.com/posts/Qjaaux3XnLBwomuNK/countess-and-baron-attempt-to-define-blackmail-fail). If having free speech is the default, being excluded from the dominant coalition for defying the orthodoxy could be construed as extortion. But if being excluded from the coalition is the default, maybe toeing the line of orthodoxy is the price you need to pay in order to be included.
-Yudkowsky has [a proposal for how bargaining should work between agents with different notions of "fairness"](https://www.lesswrong.com/posts/z2YwmzuT7nWx62Kfh/cooperating-with-agents-with-different-ideas-of-fairness). Suppose Greg and Heather are splitting a pie, and if they can't initially agree on how to split it, they have to fight over it until they do agree, destroying some of the pie in the process. Greg thinks the fair outcome is that they each get half the pie. Heather claims that she contributed more ingredients to the baking process and that it's therefore fair that she gets 75% of the pie, pledging to fight if offered anything less.
+Yudkowsky has [a proposal for how bargaining should work between agents with different notions of "fairness"](https://www.lesswrong.com/posts/z2YwmzuT7nWx62Kfh/cooperating-with-agents-with-different-ideas-of-fairness). Suppose Hayley and Isaac are splitting a pie, and if they can't initially agree on how to split it, they have to fight over it until they do agree, destroying some of the pie in the process. Hayley thinks the fair outcome is that they each get half the pie. Isaac claims that he contributed more ingredients to the baking process and that it's therefore fair that he gets 75% of the pie, pledging to fight if offered anything less.
-If Greg were a causal decision theorist, he might agree to the 75/25 split, reasoning that 25% of the pie is better than fighting until the pie is destroyed. Yudkowsky argues that this is irrational: if Greg is willing to agree to a 75/25 split, then Heather has no incentive not to adopt such a self-favoring definition of "fairness". (And _vice versa_ if Heather's concept of fairness is the "correct" one.)
+If Hayley were a causal decision theorist, she might agree to the 75/25 split, reasoning that 25% of the pie is better than fighting until the pie is destroyed. Yudkowsky argues that this is irrational: if Hayley is willing to agree to a 75/25 split, then Isaac has no incentive not to adopt such a self-favoring definition of "fairness". (And _vice versa_ if Isaac's concept of fairness is the "correct" one.)
-Instead, Yudkowsky argues, Greg should behave so as to only do worse than the fair outcome if Heather also does worse: for example, by accepting a 48/32 split in Heather's favor (after 100−(32+48) = 20% of the pie has been destroyed by the costs of fighting) or an 42/18 split (where 40% of the pie has been destroyed). This isn't Pareto-optimal (it would be possible for both Greg and Heather to get more pie by reaching an agreement with less fighting), but it's worth it to Greg to burn some of Heather's utility fighting in order to resist being exploited by her, and at least it's better than the equilibrium where the entire pie gets destroyed (which is Nash because neither party can unilaterally stop fighting).
+Instead, Yudkowsky argues, Hayley should behave so as to only do worse than the fair outcome if Isaac also does worse: for example, by accepting a 48/32 split in Isaac's favor (after 100−(32+48) = 20% of the pie has been destroyed by the costs of fighting) or an 42/18 split (where 40% of the pie has been destroyed). This isn't Pareto-optimal (it would be possible for both Hayley and Isaac to get more pie by reaching an agreement with less fighting), but it's worth it to Hayley to burn some of Isaac's utility fighting in order to resist being exploited by him, and at least it's better than the equilibrium where the entire pie gets destroyed (which is Nash because neither party can unilaterally stop fighting).
It seemed to me that in the contest over the pie of Society's shared map, the rationalist Caliphate was letting itself get exploited by the progressive Egregore, doing worse than the fair outcome without dealing any damage to the Egregore in return. Why?
[The logic of dump stats](/2023/Dec/agreeing-with-stalin-in-ways-that-exhibit-generally-rationalist-principles/#dump-stats), presumably. Bargaining to get AI risk on the shared map—not even to get it taken seriously as we would count "taking it seriously", but just acknowledged at all—was hard enough. Trying to challenge the Egregore about an item that it actually cared about would trigger more fighting than we could afford.
-In my illustrative story, if Greg and Heather destroy the pie fighting, then neither of them get any pie. But in more complicated scenarios (including the real world), there was no guarantee that non-Pareto Nash equilibria were equally bad for everyone.
+In my illustrative story, if Hayley and Isaac destroy the pie fighting, then neither of them get any pie. But in more complicated scenarios (including the real world), there was no guarantee that non-Pareto Nash equilibria were equally bad for everyone.
I had a Twitter exchange with Yudkowsky in January 2020 that revealed some of his current-year thinking about Nash equilibria. I [had Tweeted](https://twitter.com/zackmdavis/status/1206718983115698176):
It's totally understandable to not want to get involved in a political scuffle because xrisk reduction is astronomically more important! But I don't see any plausible case that metaphorically sucking Scott's dick in public reduces xrisk. It would be so easy to just not engage in this kind of cartel behavior!
-An analogy: racist jokes are also just jokes. Irene says, "What's the difference between a black dad and a boomerang? A boomerang comes back." Jonas says, "That's super racist! Tons of African-American fathers are devoted parents!!" Irene says, "Chill out, it was just a joke." In a way, Irene is right. It was just a joke; no sane person could think that Irene was literally claiming that all black men are deadbeat dads. But the joke only makes sense in the first place in context of a culture where the black-father-abandonment stereotype is operative. If you thought the stereotype was false, or if you were worried about it being a self-fulfilling prophecy, you would find it tempting to be a humorless scold and get angry at the joke-teller.[^offensive-jokes-reflect-conceptual-links]
+An analogy: racist jokes are also just jokes. Jocelyn says, "What's the difference between a black dad and a boomerang? A boomerang comes back." Keith says, "That's super racist! Tons of African-American fathers are devoted parents!!" Jocelyn says, "Chill out, it was just a joke." In a way, Jocelyn is right. It was just a joke; no sane person could think that Jocelyn was literally claiming that all black men are deadbeat dads. But the joke only makes sense in the first place in context of a culture where the black-father-abandonment stereotype is operative. If you thought the stereotype was false, or if you were worried about it being a self-fulfilling prophecy, you would find it tempting to be a humorless scold and get angry at the joke-teller.[^offensive-jokes-reflect-conceptual-links]
[^offensive-jokes-reflect-conceptual-links]: I once wrote [a post whimsically suggesting that trans women should owe cis women royalties](/2019/Dec/comp/) for copying the female form (as "intellectual property"). In response to a reader who got offended, I [ended up adding](/source?p=Ultimately_Untrue_Thought.git;a=commitdiff;h=03468d274f5) an "epistemic status" line to clarify that it was not a serious proposal.
✓ quote Yudkowsky's LW moderation policy
✓ hint at "Yes Requires" objector being trans
✓ quote Jack on timelines anxiety
-_ confusing people and ourselves about what the exact crime is
-_ FTX validated Ben's view of EA!! ("systematically conflating corruption, accumulation of dominance, and theft, with getting things done")
+- nuance and FTX epilogue
_ clarify "A Lesson is Learned"
----
_ being friends with dogs (it's good, but do I have the wordcount budget?)
_ Anna on Paul Graham
_ Yudkowsky thinking reasoning wasn't useful
_ Jessica brought up race & IQ (she skimmed the previous draft, so I should highlight this)
+_ FTX and nuance epilogue (new)
_ Michael's SLAPP against REACH (new)
_ Michael on creepy and crazy men (new)
_ elided Sasha disaster (new)
projects / Ultimately_Untrue_Thought.git / commitdiff