Potential and Pitfalls: A Review of Calling Bullshit

Title: Calling Bullshit: The Art of Skepticism in a Data-Driven World
Author: Carl T. Bergstrom, Jevin D. West
Publisher: Random House
Publishing Date: August 4, 2020
Pages: 336 (Hardcover)
ISBN: 978-0525509189

“The world is awash with bullshit, and we’re drowning in it.”

So begin Carl Bergstrom and Jevin West in their new book, Calling Bullshit: The Art of Skepticism in a Data-Driven World. This work began as a course at the University of Washington, where Bergstrom is a biologist and West a data scientist. The course and book both identify the precise nature of the bullshit that surrounds us in the age of “big data” and provide the reader strategies for defusing its influence¹ .

For their purposes, Bergstrom and West propose the following definition:

Bullshit involves language, statistical figures, data graphics, and other forms of presentation intended to persuade or impress an audience by distracting, overwhelming, or intimidating them with a blatant disregard for truth, logical coherence, or what information is actually being conveyed.

The book then focuses on the ways in which those fluent with mathematics and data science can create this special type of bullshit. The middle of the book identifies several typical sources of bullshit associated with those with a background in mathematics and statistics: spurious causation (“rising home prices are causing fewer babies to be born”), selection bias (“those who switch to Acme Insurance save an average of $500 every year”), and data graphics (e.g., different bin sizes on a histogram).

There is also a substantial discussion of two areas that are more challenging for the non-expert, but nonetheless extremely influential in the 21st century and worth our time. The first is a discussion of the bullshit enabled by artificial intelligence, machine learning, neural networks, and big data. These techniques differ from traditional computer algorithms in the sense that the quality of their output should improve over time as the algorithm “learns” how to analyze new data. However, even for those with substantial training in mathematics, statistics, and computer science, the algorithms themselves can be shrouded in mystery. Bergstrom and West correctly observe that the details of the machine learning algorithm do not need to be understood in order to call bullshit on the output. The standard principle of garbage in, garbage out applies: if the input data is flawed, the output of even the best algorithms will be flawed as well.² .

A second major topic of discussion is the ways in which the scientific enterprise is susceptible to bullshit production—competition for funding, publications of positive results, and p-hacking, among others, are helpfully described, along with several infamous examples. Nonetheless, the authors declare that “science works”; our use of the scientific method enables us to have working GPS, fly in airplanes, cure diseases, and so forth. Science, they declare, is largely self-correcting. This chapter is perhaps the hardest to follow at various points (the layperson may need to re-read their exploration of the prosecutor’s fallacy a few times to fully grasp its significance), but the reader’s persistence will be rewarded with a richer understanding of both the potential and the pitfalls of modern science .

The book closes with several tips on how to spot and refute this quantitative bullshit. They exhort their readers to humility and charity. Bergstrom and West assert that much of what we identify as bullshit in our everyday lives is more likely the result of incompetence than malice.

For a Christian audience in an age of fake news, Bergstrom and West’s book is especially compelling. In faithful obedience to fulfilling the cultural mandate, we expect to unfold a logical, coherent creation that can be rightly understood and developed for the flourishing of all humanity. But in order to carry this work out in good faith, we need to recognize the complexity of creation and treat it with intellectual honesty. Put simply, Christians should be neither producers nor consumers of bullshit. Numerical arguments are often granted a veneer of objectivity, partly because so few people feel equipped to challenge them. In this book, Bergsrom and West begin that equipping work by demonstrating many of the ways in which quantitative arguments are not neutral, and they provide a valuable resource for faithfully engaging the intricacies of creation in the 21st century.

1. A word on terminology. The word “bullshit,” while possibly not a term regularly employed in polite company, has a history as a term of art in academic circles. Bergstrom and West cite, among others, the philosopher Harry Frankfurt, whose famous essay describes bullshit as “what people create when they try to impress you or persuade you, without any concern for whether what they are saying is true or false, correct or incorrect” (39). Thus, bullshit may be true! What matters, then, is the bullshit artist’s intent and lack of regard for the truth.  ↩

2. As an example from Cathy O’Neil’s related 2016 book, Weapons of Math Destruction, police departments using predictive policing algorithms trained on old data will exhibit the same biases as the training data.  ↩