Hidden bias in machine-learning systems

A research paper on disparate impact and big-data systems for healthcare



  • Technology and Society
  • Simon Fraser University
  • July 2015


  • Literature review
  • Assembling paper
  • Revisions


  • Understand big data
  • Identify issues
  • Make recommendations

Following my literature review, it seems that big data debates are characterized by an ideological binary: that big data will either save us, or it will exacerbate problems. Realizing the lock this binary held over my own perception of big data, I sought to break it down by holding it up to scrutiny in my research.

In this paper, I sought to explore what it means for big data systems to enter the healthcare sector. I looked at the perspectives of IBM (creator of Watson Health) and health practitioners, as well as investigative journalists, lawyers, academics, and data scientists.

My position is that big data must not be considered neutral since the systems and algorithms are designed, and therefore, influenced by people. Ultimately, I side with those who urge caution over those who say to trust it entirely. This is not to say that big data systems are irredeemable. To the contrary, arguments urging caution contain valuable forethought about possible undemocratic outcomes—this can foster critical awareness and steer practitioners in a better direction.

Read Paper

Hidden Bias - Big Data (14 pages)


Critiquing technology comes with a great weight and responsibility—it’s easy to fall in deterministic or utopian-ist traps (the binaries are everywhere). Good critique looks at the landscape of perspectives before arriving at its own.

It’s also fascinating to me that we’re alive during a time when great systems—infrastructural—are changing because of massive volumes of data. Researchers are calling big data a new knowledge paradigm. As with any new communication technology (writing, the printing press, the telephone), big data will inherit the same pattern—it will change how humans arrive at knowledge (our epistemologies), and therefore how we perceive and experience reality (our ontologies). It is crucial that as big data creates opportunities for commercial and civic life, that we extend our imaginations and ask: “What will life be like? For myself? For others? Is this a better future?”

Next Project