Weapons of Math Destruction

Although it was Hitler and his henchmen who unleashed death and destruction during the Second World War, someone had to design the railways, factories, warehouses, and machinery for their war effort. An article in the New Atlantis titled “The Architecture of Evil” opens with the provocative statement: “Someone designed the furnaces of the Nazi death camps.” The article goes on to describe the life and work of Albert Speer, Adolf Hitler’s “chief architect,” reminding us that Hitler did not work alone. The truth is that engineers and architects designed the technology that enabled the Nazi brutality. Speer later wrote “my obsessional fixation on production and output statistics blurred all considerations and feelings of humanity.”

While Speer’s situation seems like a dramatic example, the truth is that all engineering work involves some moral choices and responsibility. Even programmers writing logical, mathematical code need to recognize that their creations are not neutral and unbiased. Cathy O’Neil worked as a math professor until 2007 when a lucrative opportunity arose to use her PhD in mathematics at a hedge fund. Shortly thereafter, the financial crisis occurred and O’Neil found herself pondering her work and her role as a “quantitative analyst” (or what is often referred to as a “quant”) in the finance industry. Reflecting on this, she later wrote: “The housing crisis, the collapse of major financial institutions, the rise of unemployment – all that had been aided and abetted by mathematicians wielding magic formulas.” Her disillusionment led her to participate in the “occupy Wall Street” movement and eventually write a book titled Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy.

Technology is not neutral. Even equations and computer algorithms, which may initially appear cold and neutral, reflect the values and assumptions of the people and organizations that construct them. “Big data” sifts through vast oceans of data to find patterns that are then used to inform decisions in areas as diverse as finance, banking, hiring, marketing, policing, education, and politics. While mathematical models allow decision-making to be more efficient, they can sometimes hurt the poor, target predatory ads, and discriminate against minorities while serving to make the rich richer.

For example, should data like ZIP codes act as a proxy for creditworthiness, for hiring decisions, or for dating matches? It’s not hard to imagine how such decisions could perpetuate a cycle of poverty. Should the data taken from current employee profiles be used to guide future hiring decisions? Such a decision could perpetuate biases reducing diversity in the workplace. Some of the consequences resulting from data and mathematical models are unintentional, but in the words of O’Neil, these mathematical models are “opinions embedded in mathematics.”

The inner working of these systems is often opaque, and the verdicts are accepted without question. Often, the inner assumptions and values embedded in the math are hidden over concerns of intellectual property and trade secrets. This leaves those affected without any explanation or recourse for unfair decisions which may impact them.

The fact is that all of life is religious, and that even our technical and mathematical work has moral and ethical implications. Our big data algorithms and mathematical models can be directed in ways that are more obedient or less obedient to God’s intents for his world. In fact, as more decisions are informed by number-crunching computers, we will need to make sure that justice and transparency are emphasized.

“The Architecture of Evil” not only tells the story of Albert Speer, but goes on to suggest that “Today’s engineers need a more well-rounded education—one that stresses not only the analytical skills necessary to be a good engineer but also the liberal arts that are necessary… for young students to grow and mature as citizens with responsibilities beyond the immediate technical concerns of their work.” Our computer science and engineering schools need to attend to ethics and values if we hope to build a just society.

I would add that Christian engineers must see their technical work as a response to God, one in which even our mathematical models, computer programs, and architecture need to enhance justice and show mercy as we walk humbly with our God (Micah 6:8).

This article originally appeared in the July 10, 2017 issue of Christian Courier.