Who, me? They warned you about me?
Thoughts on "We are the people they warned you about."
Caution sign on ski slope (source: D Coetzee on Flickr)
Chris Anderson recently published “We are the people they warned you
about,” a
two part
article
about the development of killer drones. Here’s the problem he’s
wrestling with: “I’m an enabler … but I have no idea what I should do differently.”
That’s a good question to ask. It’s a question everyone in technology
needs to ask, not just people who work on drones. It’s related to the
problem of
ethics at scale: almost everything we do has consequences. Some of those consequences are good, some are bad. Our ability to multiply our actions to
internet scale means we have to think about ethics in a different way.
Learn faster. Dig deeper. See farther.
The second part of Anderson’s article gets personal. He talks about
writing code for swarming behavior after reading
Kill Decision, a science fiction novel about swarming robots running amok. And he struggles with three issues. First (I’m very loosely paraphrasing Anderson’s words), “I have no idea how to write code
that can’t run amok; I don’t even know what that means.” Second, “If
I don’t write this code, someone else will—and indeed, others
have.” And third, “Fine, but my code (and the other open source code)
doesn’t exhibit bad behavior—which is what the narrator of Kill
Decision would have said, right up to the point where the novel’s
drones became lethal.”
How do we protect ourselves, and others, from the technology we
invent? Anderson tries to argue against regulatory solutions by saying
that swarming behavior is basically math; regulation is essentially
regulating math, and that makes no sense. As Anderson points out, Ben
Hammer, CTO of Kaggle,
tweeted that regulating artificial intelligence essentially means regulating matrix multiplication and derivatives. I like the feel of this
reductio ad absurdum, but neither Anderson nor I buy
it—if you push far enough, it can be applied to anything. The
FCC regulates electromagnetic fields; the FAA regulates the Bernoulli
effect. We can regulate the effects or applications of technology,
though even that’s problematic. We can require AI systems to be “fair” (if we can agree on what “fair” means); we can require that drones not attack people (though that might mean regulating emergent and unpredictable behavior).
A bigger issue is that we can only regulate agents that are willing to
be regulated. A law against weaponized drones doesn’t stop the
military from developing them. It doesn’t even prevent me from
building one in my basement; any punishment for violation comes after
the fact. (For that matter, regulation rarely, if ever, happens before
the technology has been abused.) Likewise, laws don’t prevent
governments or businesses from abusing data. As any speeder knows,
it’s only a violation if you get caught.
A better point is that, whether or not we regulate, we can’t prevent
inventions from being invented, and once invented, they can’t be put
back into the box. The myth of Pandora’s box is powerful and resonant
in the 21st century. The box is always opened. It’s
always already opened; the desire to open the box, the desire
to find the box and then open it, is what drives invention in the
first place.
Since our many Pandora’s boxes are inevitably opened, and since we
can’t in advance predict (or even mitigate) the consequences of
opening them, perhaps we should look at the conditions under which
those boxes are opened. The application of any technology is
determined by the context in which it was invented. Part of the
reason we’re so uncomfortable with nuclear energy is that it has been
the domain of the military. A large part of the reason we don’t have
Thorium reactors, which can’t melt down, is that Thorium reactors
aren’t useful if you want to make bombs.
If this is so, perhaps the solution is opening the box in an
environment where it does the least harm. Paradoxically, that means
opening the box in public, not in private. My claim is that
putting an invention into a public space inevitably makes that
invention safer. Military research in many countries is no doubt
building autonomous killer drones already. This being the case, does
developing open source drone software make us more or less safe? When
invention takes place in public, we (the public) know that it exists.
We can become aware of the risks; we have some control over the
quality of the code; just as many eyes can find the bugs, many
minds can think about the consequences. And many minds can think about
how to defend against those consequences.
That argument isn’t as strong as I’d like. We can make it stronger by
expanding our concept of an “invention” to include everything that
makes the invention work, not just the code. Cathy O’Neil has
frequently written about the danger of closed, opaque data models,
mostly recently in
Weapons of Math Destruction. Openness and safety are allies. Regulation is a useful tool, though a tool that’s not as powerful as we’d like to think.
Regulation or not, we won’t prevent the technology from being
invented. By inventing in public, inventions can get the scrutiny and
critical examination they need. I think Anderson would like to make
this point, but isn’t really comfortable with it. I share that
discomfort (whether it’s his or not), but I think it’s unavoidable.
That may be all the safety we get.