Volume 32 Number 2
[Don't Get Me Started]
By David Platt | February 2017
Thou shalt not make a machine in the likeness of a man’s mind.
—From the “Orange Catholic Bible,” in “Dune,” by Frank Herbert (Chilton, 1965)
Thou and I are first-class sinners then, aren’t we?
—From “Why Software STILL Sucks,” by David S. Platt (to be published in 2018)
Anniversaries are natural times for reflection. As I begin my eighth year fulminating in this space, I see today’s software industry crossing a watershed.
Software, up until now, has meant machines aiding humans in simple tasks. My favorite example is the auto-complete in Microsoft Word. When I type “hte,” instead of ringing an alarm bell and insisting that I fix it, Word simply corrects it to the “the” that I intended. I praised this behavior in my very first Don’t Get Me Started column in February 2010 (msdn.com/magazine/ee309884). “This feature uses the computer to do what computers do best, so that the human user can do what humans do best. It understands; it respects; it even enhances the humanity of the user.” Great.
Today’s artificial intelligences do not merely run code written by their developers (“case WM_LBUTTONDOWN:” and so on). Today’s machines are starting to evolve their own behavior via processing large data sets, modifying—and we hope improving—themselves in ways not directly controlled by their human creators. Perhaps these evolutions will be beneficial, or at least benign. I certainly go to work every day aiming for those results, and I know you do, too. But today we’re entering a realm of critical mass, where we plug stuff in, but we’re not quite sure what’s going to come out.
This isn’t just Clippy popping up, emitting his little mechanical fart and saying: “I see you’re writing a crude forgery. Can I help? Is this a business forgery or a personal forgery?” We’re handing over life-and-death responsibility to these new AIs.
Consider self-driving cars, now being tested on the open road and coming soon to a street near you. If the computer driving the car is boxed in and can’t avoid hitting something, does it choose the bicyclist wearing a helmet, because he’ll probably be hurt less? Or the one not wearing a helmet, because he’s obviously less intelligent? Does it matter if either is flouting the traffic signal? How about if the car’s passenger isn’t wearing a seat belt? You can game this out in the Moral Machine simulator at moralmachine.mit.edu.
Our software, and therefore we as its creators, is shouldering much more responsibility today than it ever has. What happens if we get it wrong?
In his 1965 masterwork “Dune,” Frank Herbert envisions a far-future universe in which interstellar travel is commonplace, but which doesn’t have any sort of computing machinery. The machines had gotten too smart, and humans had to band together to wipe them out in the Butlerian Jihad, set 10,000 years previously. The surviving humans then forbade the resurgence of computing machinery with the biblical commandment you see at the start of this column.
“Humanity lost its drive,” reads the prequel (“Butlerian Jihad” by Herbert’s son Brian Herbert, Tor Books, 2010). “With few ambitions, most people allowed efficient machines to perform everyday tasks for them. Gradually humans ceased to think, or dream … or truly live.” Is this where our Nest thermostats and, soon, our self-driving Uber cars are leading us?
As Reverend Mother Gaius Helen Mohiam instructs the young Paul Atreides: “Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them.” Does the alleged hack of the 2016 Democratic campaign ring any bells here?
If you and I can’t make the world work, the alternative isn’t pretty. In his acclaimed novel, “A Canticle for Leibowitz” (Lippincott, 1960), Walter E. Miller Jr. describes a post-apocalyptic purge of brainy people. “Nothing had been so hateful in the sight of those mobs as the man of learning … Joyfully the mobs accepted the name, took up the cry: Simpletons! Yes, yes! I’m a simpleton. … Anybody here not a simpleton? Get the bastard, if there is!”
We’d better get this new stuff right. Because if we get it wrong, they’re coming after us.
David S. Platt teaches programming .NET at Harvard University Extension School and at companies all over the world. He’s the author of 11 programming books, including “Why Software Sucks” (Addison-Wesley Professional, 2006) and “Introducing Microsoft .NET” (Microsoft Press, 2002). Microsoft named him a Software Legend in 2002. He wonders whether he should tape down two of his daughter’s fingers so she learns how to count in octal. You can contact him at rollthunder.com.