How can we stop algorithms telling lies?
[Kip Currier: Cathy O'Neil is shining much-needed light on the little-known but influential power of algorithms on key aspects of our lives. I'm using her thought-provoking 2016 Weapons of Math Destruction: How Big Data Increases Inequality And Threatens Democracy as one of several required reading texts in my Information Ethics graduate course at the University of Pittsburgh's School of Computing and Information.]
"A proliferation of silent and undetectable car crashes is harder to investigate than when it happens in plain sight.
I’d still maintain there’s hope. One of the miracles of being a data sceptic in a land of data evangelists is that people are so impressed with their technology, even when it is unintentionally creating harm, they openly describe how amazing it is. And the fact that we’ve already come across quite a few examples of algorithmic harm means that, as secret and opaque as these algorithms are, they’re eventually going to be discovered, albeit after they’ve caused a lot of trouble.
What does this mean for the future? First and foremost, we need to start keeping track. Each criminal algorithm we discover should be seen as a test case. Do the rule-breakers get into trouble? How much? Are the rules enforced, and what is the penalty? As we learned after the 2008 financial crisis, a rule is ignored if the penalty for breaking it is less than the profit pocketed. And that goes double for a broken rule that is only discovered half the time...
It’s time to gird ourselves for a fight. It will eventually be a technological arms race, but it starts, now, as a political fight. We need to demand evidence that algorithms with the potential to harm us be shown to be acting fairly, legally, and consistently. When we find problems, we need to enforce our laws with sufficiently hefty fines that companies don’t find it profitable to cheat in the first place. This is the time to start demanding that the machines work for us, and not the other way around."
