What happens when technology is unrestrained in its growth? With accelerating technological progress, the world may be a veritable utopia for those who have mastery over machines. But those on the other side of the digital divide will not be so fortunate.
We can look to science fiction for predictions on this. In the novel Altered Carbon, societies’ richest have the means to buy immortality. And in stories like 1984, Fahrenheit 451, and Brave New World, those who don’t have power are not allowed freedom of thought – they are bombarded with propaganda, distracted with mind-numbing “entertainment”, and conditioned through subliminal messaging to unquestioningly follow the command of the powers that be. In those stories, books are burned and academics are outlawed. Ideas are not valued, innovation is unheard of, and the status quo is reinforced and reinforced and reinforced. This creates a feedback loop – a recursion.
Do you recognise anything about this image? This is artwork that I created, which I generated using DeepDream (1), a computer vision program which uses image recognition technology to find patterns in images, and reinforce what it thinks it is seeing based on available reference data. This particular image was a relatively boring picture that I took in an airport… so how has the program been able to recognise eyes, and birds, and fish, and other creatures, and play them back over the image?
It’s similar to the human phenomenon pareidolia – whereby we see things we recognise among visual noise, like looking for animals in the clouds (2). And just like we’re able to recognise things that we’ve seen before, objects from our memories, this program is able to recognise things based on the images it was trained on: a data set created by researchers at Stanford and Princeton that contains 14 million human labelled images (3).
I’ve put this image through the generator a number of times, and the more iterations it goes through, the more it recognises patterns and reinforces what it is seeing. That circle looks like an eye. That eye-looking thing looks like an eye. That eye looks like an eye. That eye looks like an eye, looks like an eye. Its iterations are self perpetuating. The result of this feedback loop is perhaps more of a nightmare than a dream.
Like my nightmarish imagery, the technology we use every day contains elements that reaffirm themselves – affirmation based on limited data sets available, and limited worldviews of the people programming them.
Much image recognition software is incredibly biased. For example, an image of a person in a kitchen is most likely to be recognised as a woman (4). This is because we have such an extensive cultural history of women being in homemaker roles, and the images that are used to train neural networks in these data sets reflects our cultural history
On the flip side, most voice recognition software is trained on databases that are massively biased towards male voices, and Western accents (5). In 2016, a research fellow in linguistics at the University of Washington, found that Google’s speech-recognition software was 70% more likely to accurately recognise male speech (6). An article that quoted a woman who had bought a 2012 Ford Focus, only to find that its voice-command system only listened to her husband, even though he was in the passenger seat. Another woman called the manufacturer for help when her Buick’s voice-activated phone system wouldn’t listen to her: “The guy told me point-blank it wasn’t ever going to work for me. They told me to get a man to set it up.” (7).
Earlier this year I taught some intro to coding workshops to groups of primary school aged girls for the Adelaide Fringe. In my day job, I train adults to use enterprise software, and some of them barely know how to unlock their computers. Children, on the other hand, are fearless – and these little girls weren’t afraid to dive headfirst into learning coding. Not only that, but they weren’t afraid to interact with the computers in different ways – preferring to use the touchscreen over the keyboard and mouse, and using voice commands.
It scares me to think these girls might be left out – by implicit bias in design, and through recursion, machines with programmed bias that serve only those who programmed them.
How can we improve this?
- By diversifying the data sets to be as wide ranging as possible
- By diversifying the group of people that collate and label the data sets
- By diversifying the pool of the people that write the algorithms that teach the computer how to make use of the data sets.
Sources and Links: