Combating Bias in Big Data, with Cathy O’Neil
March 27, 2019
This episode’s guest is Cathy O’Neil: mathematician, data scientist, and author of such works as Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. Cathy talks with Richard about her past roles as a math professor, a quantitative analyst for a hedge fund, and a data scientist for a dot com startup. She explains how her experiences inform the writing that she’s doing now about the societal impact of technology and big data.
You’ll hear about how Cathy first fell in love with math at the same time that she discovered its value as a social activity that brought together kids who might otherwise have felt excluded in school. She discusses how she came to see the value of mathematics as a collaborative, cooperative activity, and why she feels excessive emphasis on math competitions can detract from this. Cathy also talks to Richard about her experiences in college and grad school, where she worked to support greater diversity in the math department.
After completing her PhD from Harvard, Cathy worked as a math professor at Barnard College, a quantitative analyst at the hedge fund D.E. Shaw, and a data scientist in the New York start-up scene. She discusses why she ultimately left these jobs, and what she learned from them about issues of fairness in the math and technology world. Cathy explains how she came to the realization that the career she wanted was the one that involved working to make things better.
You’ll also hear Cathy talk about her book, Weapons of Math Destruction, which was longlisted for the National Book Award. She and Richard discuss the book’s observations regarding the potential dark side of big data, and the ways that predictive algorithms can end up reinforcing inequalities that already exist in society.
You can read some of Cathy’s writing on her blog, mathbabe (warning: adult language), and her Bloomberg column. You can also learn about her company, ORCAA, which audits algorithms for bias and fairness.