The unique ability of the human brain to create technologies has taken us far. The benefits of technology, however, are not guaranteed, yet we celebrate and pursue them with abandon. When we imagine new technological abilities, we tend to ask one question only: “Can we?” “Can we create such a thing?” However, we’re good at creating what we can but shouldn’t. “Should we?”, though rarely asked, is the more important question by far.
I recently read a book by Samuel Arbesman, entitled Overcomplicated. I found it intriguing, yet also utterly frightening. Arbeson is Scientist in Residence at Lux Capital, a science and technology venture capital firm. He is a fine spokesperson for his employer’s interests, for he gives the technologies that make venture capitalists rich free license to do what they will by calling it inevitable.
Many modern technologies are now complicated in ways and to degrees that place them beyond our understanding. Arbesman accepts these over-complications as a given. In light of this, he proposes ways to study them that might yield a bit more understanding, even though, in his opinion, they will forever remain beyond our full grasp. He argues that modern technologies are like biological systems—the result of evolution rather than design—sometimes a mishmash of kluges embedded in millions of lines of programming code and sometimes the results of computers generating their own code with little or no human involvement. At no point in the book does Arbesman ask the question that was constantly screaming in my head as I read it: “Should we?” Should we create technologies that exceed our understanding and can therefore never be fully controlled? The only rational and moral answer to this question is “No, we shouldn’t.”
Arbesman assumes that we often cannot design and develop modern technologies in ways that remain within the reach of human understanding. Even though he acknowledges several examples of technologies that have created havoc because they were not understood, such as financial trading systems and power grids, he accepts these over-complications as inevitable.
As a technology professional of many years, I see things differently. These technological monsters that we create today as the products of kluges are over-complicated not because they cannot be kept within the realm of understanding and our control but because of poor, sloppy, undisciplined, and shortsighted design. Arbesman and others who pull the strings of modern information technologies want us to believe that these technologies are inherently and necessarily beyond human understanding, but this is a lie. Those who create these technologies are simply not willing to do the work that’s required to build them well.
We have a choice. We could demand better design. We could and should set the limits of human understanding as the unyielding boundary of our technologies. We can choose to only build what we can understand. This is harder than quickly and carelessly throwing together kluges or trusting algorithms to manage themselves, but it is a path that we must take to avoid destruction.
Arbesman advocates humility in the face of technologies that we cannot understand, but this is an odd humility, for it’s wrapped in hubris—a belief that we have the right to unleash on the world that which we can neither understand nor control. We may have this ability, but we do not have this right, for it is an almost certain path to destruction. Along with most of the technologists that he admiringly quotes in the book, Arbesman seems to embrace all information technologies that can be created as both inevitable and good—a reverence for Technology with a capital “T” that is both irrational and dangerous.
I’m certainly not the only technology professional who is concerned about this. Many share my perspective and express it, but our concerns are not backed by the deep pockets of technology companies, which currently set the agendas and shape the values of cultures throughout the developed world. The fear that our technologies could do great harm if left uncontrolled has been around for ages. This is a reasonable fear. In his film Jurassic Park, Steven Spielberg poignantly expressed this fear regarding biological technologies. There’s a great scene in the movie when a scientist played by the actor Jeff Goldblum asks the questions that we should always ask about potential technologies before we create and unleash them on the world. The scene accurately frames the problem as one that results from the selfishness of those who care only about their own immediate gains, never raising their eyes to look further into the future and never doubting the essential goodness of their creations, despite the monsters we are capable of creating.
Although this concern about unbridled technological development is occasionally expressed, it has had little effect on modern culture so far. Each of us who cares about the future of humanity and understands that the arc of technological development can be brought into line with the interests of humanity without sacrificing anything of real value should do what we can to voice our concerns. In your own organization, when an opportunity to create, modify, or uniquely apply a technology arises, you can ask, “Should we?” This might not be the path to popularity—those who choose to do good are often unappreciated for a time—but it is the only path that doesn’t lead to destruction. Be courageous, because you should.