Office Hours: The Built-in Bias of AI

Photo by Possessed Photography on Unsplash

The emergence of ChatGPT has created quite a stir in many parts of American society, none more hysterical than in the ivory towers of academia. Professors fear that students will use it to complete their assignments rather than doing the writing themselves. Multiple articles in The Chronicle of Higher Education and Inside HigherEd have ranged from predicting the downfall of humanity to praising AI's uses in the classroom. I was especially intrigued by an article by Maggie Melo about ChatGPT as an assistive technology for students and faculty with ADHD, a diagnosis that I have recently received myself.

What is less covered in all the hysteria is that artificial intelligence (AI) is already pervasive in our daily lives, and that its downsides include the possibility of increasing institutional bias in our society.

To the first point, every time you get a personalized ad on social media or Waze informs you of a police car spotted along your route, you are experiencing the products of artificial intelligence. AI makes online shopping easier, getting help from large corporations quicker and spam filtration automatic.

At the same time, we must remember that AI is not sentient; it does not think for itself. It is lines of computer code, and the biases of the programmers find their way into that code, often with problematic results. When I asked ChatGPT, "Is AI racist?" this was the result I received:

AI itself is not inherently racist, but it can exhibit biased behavior if it is trained on biased data or developed with biased algorithms. If the training data used to train an AI system is biased, such as data that reflects historical discrimination or stereotypes, the AI system may learn and reproduce these biases in its decision-making processes. This can result in the unfair treatment of certain groups of people.

For example, if an AI system is trained to recognize faces using a dataset that is mostly composed of light-skinned people, it may not be as accurate in recognizing faces of people with darker skin tones. This could result in biased outcomes, such as facial recognition systems misidentifying people of color at higher rates than white people.

To prevent AI from exhibiting biased behavior, it is important to ensure that the data used to train the AI system is diverse and representative of the populations it will encounter. Additionally, AI developers should test their systems for bias and continually monitor and update their algorithms to minimize any potential discriminatory outcomes.

If I wasn't a professor, I might let this response be the end of the blog post, my point made, but I am a professor and always want to have the last word.

In TV and movies, government agencies always use facial recognition from the increasingly large numbers of cameras recording citizens' every action to identify suspects. Sometimes they cannot make a positive identification because of the graininess or angle of the photo. But I've never seen an example of making an incorrect identification because of the suspect's race or gender. In real life, because of the over policing of Black men, their photos are overrepresented in mugshots used to train facial recognition software used by law enforcement in minoritized communities. Because of this, photos of Black men are more often matched with mugshots than other racial groups. In fact, a test by the American Civil Liberties Union (ACLU) showed that 28 members of Congress, disproportionately people of color, were misidentified as a person with a criminal arrest by Amazon's commercially available facial rec software.

And, similar to many indignities in American society, the intersectionality of being Black and female leads to the largest misidentification possibilities. In one 2018 study, 1% of white men were misgendered by facial recognition software compare to 35% of Black women. Another article describes the development of a content moderation tool that would be able to identify pornography and remove it from a social media site. Because the vast majority of programmers working on the project were white men, they didn't notice that the G-rated stock photos used to train the AI consisted primarily of light-skinned people, while the bulk of pornographic training photos featured people of color. Therefore, the AI they created associated dark skin with pornography. When compared with the recognition that white photographers fail to light Black bodies in a flattering manner, the possibility of mistaken identification increases.

The built-in biases of AI are not limited to misidentifying photos, however. Artificial intelligence is used by many digital transcription services to transform audio into text. To increase the accessibility of my podcast, The Politics Classroom, I am committed to creating a text account of each interview to accompany the audio. However, creating a transcript of an hour-long conversation from scratch would be prohibitively time consuming. The general rule is that it takes 4 times as long to transcribe audio than the original recording length. Therefore, an hour-long episode would take approximately 4 hours to transcribe, assuming the transcriber (me) knows what they're doing, which, in this instance, would be a terrible assumption to make. As a result, I rely on transcription software to achieve my accessibility goals.

However, transcription software is prone to the same pitfalls as all other types of generative AI, including programmer bias. This bias was clear when editing the transcripts of my recent podcast interviews with Chicago mayoral candidate, Ja'Mal Green, and the director of the UIC Disability Cultural Center, Dr. Margaret Fink.

Green, an activist, author, entrepreneur and political candidate, was raised on the South and West sides of Chicago and spoke African American Vernacular English (AAVE) during our conversation. AAVE is an English dialect with grammar, vocabulary and pronunciations that differ from Standard American English. That the traditional dialect of white Americans is considered "standard" tells you all you need to know about how AI-transcribed conversations were produced. I used Otter.ai to transcribe my interview, and it did not recognize many of the non-"standard" pronunciations that Green used. I will admit that I was unsure how to write words with the last consonant dropped (doin'?), but that was why I wanted a computer to handle the transcription. I'm not sure how many AAVE speakers were part of the AI's training, but the nightmare of untangling the nonsense that the service produced suggested that it wasn't a large number.

And lest we think that programming bias is only a function of race or gender, there is also evidence of ableism in AI. Dr. Margaret Fink, the director of the Disability Cultural Center, is deaf, and while I had no trouble understanding her speech during our podcast interview and subsequent in-person conversations, apparently Otter.ai was not so proficient. Dr. Fink's voice displays a different tonality from most non-deaf English speakers that seemed to confuse the AI, at least as judged by the transcription produced. Text-to-speech apps and programs, including Otter.ai, are hailed as an accessibility tool to facilitate the inclusion of deaf and hard of hearing people, but it seems as if the assumption is that the live transcription is for hearing people to communicate with deaf people. It does not appear that many programmers have thought that there might be a need to transcribe the voices of deaf or hard of hearing people for others. I recorded a podcast interview with a deaf woman and wanted other deaf or hard of hearing people to have access to the episode through a written transcript. It took a greater-than-average amount of time to edit the AI-produced transcript to achieve that purpose.

Generative artificial intelligence is not going anywhere, and it will become increasingly prevalent in modern society. While the benefits it offers are impressive and possibly life-altering, we must always keep in mind that AI is only as good as the human programming that goes into it. If programmers do not account for very real differences among groups of people based on gender identity, disability, race/ethnicity, language fluency or dialect, and many other categories when they create programs and algorithms, AI could further entrench the systemic inequities that pervade our lives and institutions. The old adage "garbage in, garbage out" is still accurate, so we must always be aware that bias in will certainly produce bias out.

--- Professor Kate Floros