highlight surprise in Turing scoring
[Ultimately_Untrue_Thought.git] / content / 2017 / thing-of-things-transgender-intellectual-turing-test-predictions-and-commentary.md
1 Title: Thing of Things Transgender Intellectual Turing Test Predictions and Commentary
2 Date: 2017-03-29 19:26
3 Category: commentary
4 Tags: ozy, two-type taxonomy
5
6 Friend of the blog—I mean, I _hope_ we're [still friends](http://unremediatedgender.space/2017/Jan/the-counter/) even though I'm kind of [trying to overthrow them](http://unremediatedgender.space/tag/ozy/) as _de facto_ Gender Czar of the [_Less Wrong_](http://lesswrong.com/) diaspora—Ozymandias of [_Thing of Things_](https://thingofthings.wordpress.com/) has been [running an intellectual Turing test](https://thingofthings.wordpress.com/2017/02/15/transgender-intellectual-turing-test/) challenging adherents of the gender-identity and two-type theories of transgenderedness to try to impersonate each other for the good of our collective epistemology!
7
8 (An aside on credit-assignment and the history of ideas: Ozy says _Blanchard–Bailey_ where I've usually been trying to say _two-type_ in order to avoid the [tricky problem of optimal eponymy](http://unremediatedgender.space/2017/Mar/nothing-new-under-the-sun/), but if you are going to be eponymous about it, I can understand just saying "Blanchard" but feel like it's unfair to include Bailey but _not_ Anne Lawrence. My understanding of the history—and I think Michael Bailey reads this blog and I trust him to send me an angry email if I got this wrong—is that [Bailey's research](http://faculty.wcas.northwestern.edu/JMichael-Bailey/research.html) had mostly been about sexual orientation and from-childhood gender nonconformity, not the two-type taxonomy as such. Bailey's popular-level book _The Man Who Would Be Queen_ drew controversy for _explaining_ the two-type taxonomy for a nonspecialist audience (in the last part of a book that was mostly about the androphilic/feminine-from-early-childhood people, not my people), but the critics who disparage _Queen_ as "unscientific" are missing the point: popular-level books that _present_ a scientific theory _aren't supposed_ to capitulate the evidence for the theory—for that, you need to follow the citations and read the primary literature for yourself. In analogy, it should not be construed as a disparagement of Richard Dawkins to note that it would be weird if people talked about the "Darwin–Dawkins theory of evolution"!)
9
10 In the intellectual Turing test, contestants answer a set of questions both as themselves, and while trying to pass as someone who believes the other thing, while the audience tries to discriminate the honest entries from the fakes. Below are my probability assignments for this contest (I think it's important to assign probabilities rather than binary guesses, so that you can assess your rationality with a Bayesian [strictly proper scoring rule](http://yudkowsky.net/rational/technical/) rather than a crude "number correct"), along with an optional brief comment—
11
12 ***Update, 5 June***: Two months after the [results were posted](https://thingofthings.wordpress.com/2017/04/04/intellectual-turing-test-results/), I finally got around to scoring these. ("Bayes-score" is the base-two [logarithmic score](https://en.wikipedia.org/wiki/Scoring_rule#Logarithmic_scoring_rule). Someone who, claiming complete ignorance, gave a 0.5/0.5 distribution for each entry would lose a [bit](https://en.wikipedia.org/wiki/Self-information) on each question for a final score of −18.)
13
14 **Gender identity entries**
15
16 [#1](https://thingofthings.wordpress.com/2017/03/03/itt-1-gender-identity/): GI: 0.65, BBL: 0.35 (strong philosophy of language; if telling the truth about being a cis woman, ignorance of non-dysphoric AGP is plausible), Actual: GI ✔, Bayes-score: −0.621  
17 [#2](https://thingofthings.wordpress.com/2017/03/06/itt-2-gender-identity/): GI: 0.4, BBL: 0.6 (awareness of 4chan shows non-naïveté about what's actually going on), Actual: GI ✘, Bayes-score: −1.322  
18 [#3](https://thingofthings.wordpress.com/2017/03/07/itt-3-gender-identity/): GI: 0.6, BBL: 0.4 (maybe a little _too_ doctrinaire??), Actual: BBL ✘, Bayes-score: −1.322  
19 [#4](https://thingofthings.wordpress.com/2017/03/08/itt-4-gender-identity/): GI: 0.6, BBL: 0.4, Actual: BBL ✘, Bayes-score: −1.322  
20 [#5](https://thingofthings.wordpress.com/2017/03/09/itt-5-gender-identity/): GI: 0.6, BBL: 0.4, Actual: GI ✔, Bayes-score: −0.737  
21 [#6](https://thingofthings.wordpress.com/2017/03/10/itt-6-gender-identity/): GI: 0.7, BBL: 0.3 (seemingly sincere trans man), Actual: GI ✔, Bayes-score: −0.515  
22 [#7](https://thingofthings.wordpress.com/2017/03/14/itt-8-gender-identity/): GI: 0.7, BBL: 0.3 (standard trans woman rationalizations), Actual: GI ✔, Bayes-score: −0.515  
23 [#8](https://thingofthings.wordpress.com/2017/03/15/itt-9-gender-identity/): GI: 0.65, BBL: 0.35 (really knows her stuff; this is what a smart, intellectually-honest BBL skeptic looks like, and I'd like to believe that they exist!), Actual: GI ✔, Bayes-score: −0.621  
24 [#9](https://thingofthings.wordpress.com/2017/03/16/itt-7-gender-identity/): GI: 0.7, BBL: 0.3, Actual: BBL ✘, Bayes-score: −1.737  
25
26 **Blanchard–Bailey–Lawrence entries**
27
28 [#1](https://thingofthings.wordpress.com/2017/03/17/itt-2-blanchard-bailey/): GI: 0.6, BBL: 0.4, Actual: GI ✔, Bayes-score: −0.737  
29 [#2](https://thingofthings.wordpress.com/2017/03/20/itt-3-blanchard-bailey/): GI: 0.4, BBL: 0.6, Actual: GI ✘, Bayes-score: −1.322  
30 [#3](https://thingofthings.wordpress.com/2017/03/21/itt-4-blanchard-bailey/): GI: 0.4, BBL: 0.6, Actual: BBL ✔, Bayes-score: −0.737  
31 [#4](https://thingofthings.wordpress.com/2017/03/22/itt-5-blanchard-bailey/): GI: 0.9, BBL: 0.1 (shibboleth fail!—people who believe in biology do not say "assigned at birth" when describing their own beliefs! Also, failure to notice the obvious "for the same reasons men are" re programmers), Actual: BBL (!!) ✘, Bayes-score: −3.322  
32 [#5](https://thingofthings.wordpress.com/2017/03/23/itt-6-blanchard-bailey/): GI: 0.2, BBL: 0.8 (preach it!), Actual: GI ✘, Bayes-score: −2.322  
33 [#6](https://thingofthings.wordpress.com/2017/03/24/itt-7-blanchard-bailey/): GI: 0.8, BBL: 0.2 ("male socialization, which unlike androphilic trans women they actually tend to absorb as kids" sounds like someone who believes that innate gender identity determines what socialization you latch onto from your culture, rather than someone who actually believes in sexual dimorphism), Actual: GI ✔, Bayes-score: −0.322  
34 [#7](https://thingofthings.wordpress.com/2017/03/27/itt-8-blanchard-bailey/): GI: 0.9, BBL: 0.1 (shibboleth fail again!—[my comment at _Thing of Things_](https://thingofthings.wordpress.com/2017/03/27/itt-8-blanchard-bailey/#comment-25273)), Actual: GI ✔, Bayes-score: −0.152  
35 [#8](https://thingofthings.wordpress.com/2017/03/28/itt-9-blanchard-bailey/): GI: 0.1, BBL: 0.9 (raw reality), Actual: BBL ✔, Bayes-score: −0.152  
36 [#9](https://thingofthings.wordpress.com/2017/03/29/itt-1-blanchard-bailey/): GI: 0.85, BBL: 0.15 ([my comment](https://thingofthings.wordpress.com/2017/03/29/itt-1-blanchard-bailey/#comment-25321)), Actual: GI ✔, Bayes-score: −0.234
37
38 **Proportion correct** (construing assignment of probability greater than 0.5 to the actual answer as "correct"): 11/18  
39 **Total Bayes-score**: −18.012 (_just_ worse than chance)