The Turkish word o is a non-gendered pronoun that translates as either “he” or “she.” Yet for a long time, if you plugged the sentence O bir doktor into Google Translate, it would come back as, “He is a doctor.” Switch doktor to hemşire—nurse—and the translation would read, “She is a nurse.”
That was a bias in the Google Translate algorithm, and it stemmed from perceptions embedded in language and human minds. While this particular Google problem has been fixed, many others remain.
“Human beings are biased,” says SFI External Professor Mahzarin Banaji. “So if you use the output from human minds to train an artificial system, it will by necessity learn the biases inherent in the human data.”
It’s an issue up for discussion at a two-day SFI working group meeting titled, “Language as a window into mind and society.” Banaji, a Harvard psychologist, organized the meeting as an opportunity for computer scientists, psychologists, and linguists to learn from each other’s work.
The purpose of language is communication — but it’s also much more. “We can elevate our mental states by the poems and novels we read,” Banaji says. “We can also do terrible things with language. We can hurt people, we can lie and deceive.”
Thanks to databases as wide-ranging as the Internet, researchers can now quantify such biases and harms by analyzing billions of words and sentences to determine how society associates certain groups of people based on race, ethnicity, gender, and other characteristics. For example, men are widely associated with engineering, technology, power, religion, sports, war, and violence, whereas women are associated with sex, lifestyle, appearance, toxic language, and profanities.
“This poses a very challenging socio-technical problem,” says University of Washington computer scientist Aylin Caliskan, who will present her research on gender bias in word embeddings at the SFI meeting.
Machines use algorithms embedded with implicit bias to make crucial decisions that affect people’s lives — everything from job candidacy and university entrance to recidivism prediction.
Caliskan gives an example of a woman applying for a tech job. If her resume contains words that reflect gender — a reference to a women’s college or sports team, perhaps — machines may perceive her as a less-than-ideal fit for the job, which historically is associated with men.
“These are not very optimistic research findings,” Caliskan says, although awareness of the problem is increasing.
As Banaji says, there is an aspiration that one day we will design machines that make better decisions than humans do. After all, language is a reflection of humanity’s wondrous potential.
“Some of the gifts that evolution has given our species, such as language, are so basic and so familiar to us that we just fail to be gobsmacked by it as we should be,” she says. “We should be just astounded by the capacity, and its role in improving judgment and decisions.”