Lately I’ve been practicing some statistical skepticism in regards to a recent op-ed that contained a lot of very obvious statistical and journalistic errors.
One of the issues I describe is Mac Donald using of a quote that doesn’t appear in the paper she is citing:
“no significant evidence of antiblack disparity in the likelihood of being fatally shot by police”.
This quote doesn’t appear in Johnson et al or its supporting documents. I’d also searched with various engines to no avail. When I mentioned the absence on Twitter it was brought to my attention that it is linked to in her city-journal piece (I’d read and replied to the WSJ piece only, and that was the subject of her Tweet). I followed this lead, wondering if it was just sloppy citation, but it turns out the quote comes from a pre-print. The is a draft of a reply Johnson et al made to a criticism of their work. As part of my due diligence in writing on the statistical wrong-doings, I had in fact read their reply. The reason I didn’t find this quote is because I read the final, peer-reviewed version, and much of the pre-print was removed prior to publication, including the quote Mac Donald presents.
The published version of the reply does not make the claim Mac Donald does.
Additionally, she presents that quote as if it was part of the peer-reviewed work, when it was not:
‘The latest in a series of studies undercutting the claim of systemic police bias was published in August 2019 in the Proceedings of the National Academy of Sciences. The researchers found that the more frequently officers encounter violent suspects from any given racial group, the greater the chance that a member of that group will be fatally shot by a police officer. There is “no significant evidence of antiblack disparity in the likelihood of being fatally shot by police,” they concluded.’
In the WSJ piece, the link to the Johnson paper is offered as a source despite the fact that it doesn’t contain the quote. It’s hard not to see this as deceitful. Mac Donald refers to peer-reviewed work, links to it, but then pulls a quote from a pre-print that ended up getting removed prior to publication. It’s an intellectual slight-of-hand - invoking rigor and credibility but slipping in something unsavory instead.
And to be clear on the timing of all this, the preprint was published August 16 of 2019. The peer-reviewed reply was published January 21, 2020. The WSJ op-ed was published June 2nd of 2020. Both were available to Mac Donald, indeed the peer-reviewed version is linked to at the top of the page of the Johnson paper she sends us to.