Are We Intelligent Enough?

I recently watched a program about the development of artificial intelligence that examined, among other things, the human labor behind it. Workers around the world have performed data labeling, annotation, content moderation, and other tasks necessary to help train systems that may eventually create extraordinary economic value.

There is nothing inherently exploitative about creating work or profiting from the capital, technology, risk, and ideas required to create it. The more difficult question is what happens when enormous differences in bargaining power determine how the resulting value is shared. At what point does opportunity become extraction?

This is hardly a new question.

Human beings have been extracting value from one another and the environment for thousands of years. Over the last several centuries, sugar, rubber, cotton, timber, minerals, oil, fisheries, manufacturing, and countless other industries have produced tremendous wealth and advancement while sometimes leaving remarkably little of that wealth where the resources and labor originated.

The pattern can resemble a swarm of locusts. The swarm arrives because the field contains something valuable. It consumes what benefits it and eventually moves on. What determines the morality of the relationship may not be simply what everyone received while the swarm was present, but what remains after it leaves.

There is a considerable difference between extraction and development.

If an industry enters a poor region, employs thousands of people, develops their skills, creates local businesses, builds infrastructure, produces experienced managers, generates accumulated capital, and leaves behind people better equipped to participate in whatever economy comes next, something has been developed. Everyone involved may have profited, including those who supplied the original capital.

But if the industry arrives because poverty makes labor inexpensive, extracts the value that is there, and leaves when cheaper labor or resources become available somewhere else, what remains when it leaves?

This is not an indictment of business. Business simply gives us an unusually visible example of a much older human tendency. When we discover something valuable, our appetite for that value can become more powerful than our consideration of what extracting it will leave behind.

The resource might be labor, land, minerals, information, attention, influence, political power, technology, or something we have not yet discovered. The person doing the extracting might be a corporation, a government, a nation, an institution, or an individual. Consumers participate as well. We routinely benefit from inexpensive products and extraordinary conveniences without knowing much about the people, resources, or systems required to provide them.

The larger question is not whether business is moral or immoral. It is what happens to human judgment when opportunity and appetite meet vulnerability.

This is where the Optessimist becomes important.

The Optessimist recognizes vulnerability as opportunity. Inexpensive labor can become cheaper labor. Weak regulation can become an advantage. Desperation can become negotiating leverage. Environmental protections can become expenses that might be avoided. Political influence can become another investment capable of producing a return.

Seeing vulnerability is not itself immoral. What matters is what we choose to do after recognizing it. One person sees vulnerability and asks how it might be strengthened. The Optessimist asks how it might be used.

Eventually, entire systems can begin rewarding the people who are particularly good at finding and exploiting these vulnerabilities. The Optessimist does not necessarily have to break the rules. The greater problem may occur when those who benefit from exploiting weakness also gain influence over the systems and rules that determine what is permissible. Exploitation can then become a rational path to success while remaining completely within the rules.

Human advancement has accomplished extraordinary things without requiring exploitation as its purpose. We have crossed oceans, learned to fly, cured diseases, transplanted organs, communicated across the planet almost instantaneously, explored space, and developed machines capable of participating in conversations like the one that inspired this essay. Human curiosity and ambition are among our greatest strengths.

The lines can blur, however, when we examine how success was achieved. Ultimately, humanity seeks the good, even while disagreeing about what the good is and how it should be pursued. Which is where Moral Science comes in.

I don’t believe the answer is for eight billion people to adopt the lifestyle of monks.

There is, however, something worth noticing about the monk. Across thousands of years and remarkably different philosophical and religious traditions, human beings repeatedly arrive at some version of restraint, moderation, temperance, simplicity, or non-attachment. Perhaps there is wisdom in recognizing that endlessly increasing our appetite does not necessarily increase our well-being.

We should explore our limits, not become consumed by them.

There is nothing wrong with discovering what is possible. Limits have always invited human curiosity, and many of our greatest advancements resulted from refusing to accept what previous generations believed could not be done. But there is a difference between exploring a boundary and developing an appetite that requires us to continually exceed it.

More wealth, more energy, more land, more computation, more influence, more products, and more consumption can all represent advancement when they serve a meaningful purpose. The danger comes when more itself becomes the purpose.

This brings us back to artificial intelligence.

We are investing extraordinary amounts of human labor, capital, electricity, water, minerals, infrastructure, and intellectual effort into creating something we hope will eventually possess intelligence equal to or greater than our own. Much of the discussion surrounding AI concerns whether increasingly capable systems could become dangerous. People wonder whether Artificial General Intelligence, or AGI, and whatever might eventually develop beyond it could threaten humanity.

Perhaps part of what we fear is ourselves. At our best and worst, we provide plenty of examples of what we have done when one person, organization, nation, or civilization acquired overwhelming power over another. We have used power to protect, educate, heal, liberate, and build. We have also used it to conquer, enslave, exploit, manipulate, extract, and destroy.

When we imagine something becoming vastly more intelligent and powerful than ourselves, perhaps it is natural to wonder whether it will treat us the way we have sometimes treated those with less power than ourselves.

That is where the next level of AI begins to resemble the nuclear situation.

If we do not build it, someone else might. If someone else builds it first, what will they do with it? Who will they allow to use it? What will they require in return? What happens when access to superior reasoning becomes one of the most valuable resources on Earth?

So we build our own and construct a fortress around it because unrestricted access could be dangerous. But the greater the capability behind the fortress, the greater the value of gaining access to it. Governments, corporations, competitors, researchers, and nations therefore have increasing incentives to develop their own systems rather than remain dependent upon whoever controls the first one.

The mechanism intended to create security can contribute to the competition creating the danger.

And eventually, if we actually succeed in creating something approaching general intelligence, it will encounter the same contradictions human intelligence has struggled with for thousands of years.

We value freedom and security, prosperity and preservation, competition and cooperation, privacy and convenience, individual choice and collective welfare, justice and mercy, ambition and contentment. These are not necessarily contradictions because one side is right and the other wrong. They are difficult because both sides can contain something worth preserving.

A sufficiently capable intelligence asked to determine what is best for humanity may eventually discover that many of the outcomes we claim to desire require something from us in return. A healthier planet may require different consumption. Preservation may require leaving valuable resources untouched. Greater equality may require some people accepting less advantage. Long-term prosperity may occasionally require sacrificing short-term gain.

And then we encounter another human value: freedom.

We may someday create an intelligence capable of understanding consequences more completely than we do and still reserve the right to ignore its advice. Not necessarily because it is wrong, but because freedom includes the ability to make choices, including choices that may not produce the mathematically optimal result.

Perhaps that is another contradiction an artificial intelligence will have to understand.

Intelligence cannot simply mean the ability to accomplish more. If it did, a swarm of locusts would become increasingly intelligent as it became increasingly efficient at consuming the field.

True advancement must account for what remains afterward.

Build the company. Invent the machine. Cure the disease. Explore space. Create artificial intelligence. Make money. Improve our lives and enjoy what human ingenuity makes possible. There is nothing inherently wrong with prosperity or ambition.

But perhaps greater intelligence requires us to ask more than whether something can be done.

What does it cost? Who bears that cost? What remains afterward? Is the damage reversible? Could the same advancement be achieved with less destruction? Are we developing something or merely extracting from it? Are we exploring our limits or becoming consumed by the appetite to continually exceed them?

Human beings have spent thousands of years trying to understand what intelligence should ultimately accomplish. Perhaps the development of artificial intelligence will force us to confront that question more directly than ever before.

We keep asking whether artificial intelligence will eventually become smarter than us.

Perhaps we should first ask whether we understand what becoming smarter actually means.

Are we collectively intelligent enough to see what is best overall, doing the right thing when it is the right thing to do?