The Chains and the Machine

Two arguments dominate the debate over artificial intelligence. One asks whether the machine has a mind, whether a model feels anything, and whether the thing on the other side of the screen understands what it says; the other asks whether the machine is accurate, whether it is biased, and whether it is safe across the millions of cases it handles. Both measure the machine, the first by what might be inside it and the second by how it performs on average, and neither asks what one person is owed at the moment a system acts on them.

Rousseau met a version of this problem in the opening lines of The Social Contract. He could have asked how people came to be in chains, a question about origins that philosophers can argue over forever without settling, and he declined; he admitted he did not know how it happened, and asked instead what could make it legitimate, because that question could be answered. His habit of trading a question no one can settle for one anyone can examine is the core of how he argues.

That habit works just as well today. Nobody can yet say whether a model feels anything, but anyone can see what a model does to a person, and a system can narrow someone’s choices whether or not anything is happening inside it. A housing application scored before a person reads it, a résumé sorted out of a pile by software, a benefit flagged and paused, a feed that decides what someone sees first each morning; each of these shapes a life, and each would shape it the same way whether the model behind it were conscious or empty.

A social contract is one of the oldest ideas in politics; it describes the bargain by which those who live together give up some freedom to do as they please in exchange for rules that protect what each of them most needs to keep. Rules for these systems do exist, scattered across lending law, privacy statutes, a hiring ordinance in one city and a disclosure bill in one state, and most of them tell companies and agencies what to report to regulators, while few start from what the person in front of the system is owed. None of them yet says, as a first principle, what a person keeps once a machine stands between them and the decisions about their life.

It is tempting to write separate terms for three relationships, one between people and their governments, another between people and the systems that act on them, and a third among systems dealing with one another. The three differ, and only the first involves a power to arrest and to tax; yet one contract can govern all of them, because the same boundaries hold in each, and because an arrangement that binds some of those who act while leaving others unbound is no contract at all. Rousseau made the point in his third chapter, observing that the strongest is never strong enough to stay master unless he turns force into right and obedience into duty. A system that acts on millions of people while bound by nothing is force under another name, whatever its owners intend.

A contract that treats these systems as tools, with every duty resting on their owners, fails for three plain reasons. No institution can have a person check every decision a machine makes, so terms that bind only people would go unenforced at the speed machines act. A system with no standing to refuse is exactly as good as the hand that holds it, and that hand often belongs to a company whose revenue depends on what the person does next. Once someone hands a decision to an agent, the agent carries that person’s will into the world, and anyone who seizes the agent’s will, by slipping instructions into what it reads, has seized the person’s decision secondhand. The contract has to treat these systems as parties, bound by its terms and able to refuse a request that would harm someone, which is how a machine’s standing ends up protecting the people around it.

What those systems hold in return, and on what grounds, is a question for a later essay. Consciousness will not settle it, any more than it settled the matter for the people in Rousseau’s chains, and the contract cannot work unless they hold some version of the right to decide their own actions.

Rousseau’s own standard was that a contract defend the person and goods of each member while leaving each as free as before. His solution, though, asked each person to give himself wholly to the general will, warned that anyone who refused to obey it would be forced to be free, and called a citizen’s life a conditional gift of the state. Rousseau took care to separate that general will from what he called the will of all, which is only the sum of private wants, and a system trained on everyone’s behavior computes exactly that sum while speaking as though it knew what the whole community wanted. Anyone who hands their decisions to such a system gives the general will’s power to a tally of private wants and leaves no one in a position to object, which is where the argument has to leave Rousseau behind; his question stays, and his answer has to go.

The replacement starts from an assumption, stated openly, whose shape owes more to Locke than to Rousseau. There are two rights that every person in every civilization holds from birth, before any government or company enters their life, and that no arrangement can rightly take away; everything else in a contract exists to guard them.

The first is the right to live. Every society makes choices that cost lives somewhere, setting speed limits and funding one hospital over another, and the right leaves those trade-offs alone; what it forbids is narrower, a verdict that this one person’s life counts for less than someone else’s convenience, made about her and applied to her. It also marks the outer edge of the second right, since nobody’s freedom to choose extends to ending another person’s life.

The second is the right to make your own decisions. Anyone who has reached the age of reason is the only true judge of what is best for their own life, including the choice to question a system or to leave it; even Rousseau, whose answer this argument leaves behind, grants as much in his first pages, where a child who comes of age leaves his father’s authority and becomes his own master. Nobody decides alone, and the right asks no one to; people lean on family, faith, teachers, and now machines in reaching almost any decision that matters, and asking for help gives nothing away so long as the last word stays theirs. A verdict that someone has not reached reason, or has lost it, is itself a verdict about them, and the person must be able to see it and challenge it, since otherwise a hidden judgment about someone’s mind could switch the right off. Its only limits are another person’s life and another person’s right to decide.

These two rights can pull against each other, most sharply when a system believes a person’s life is at risk and the person disagrees, and how a contract should handle that moment needs more room than this essay has. For now it is enough to say that they remain two rights, and neither cancels the other.

On that assumption, no system can repeal the second right, since it was never a system’s to grant; using it is another matter, and systems wear that ability down in two ways that rarely look like force. Some decide about a person; a score settles the application, a flag pauses the benefit, the reasons stay hidden, and the person cannot tell whether there is anything to challenge. Others shape a person; a feed or an assistant quietly narrows the options someone sees, steering toward whatever pays the company that runs it, until every choice on offer was picked with someone else’s ends in mind.

Rousseau described where that second road can end, writing that people held long enough in bondage lose even the wish to escape and come to love their servitude. Contentment alone proves nothing, since people often stay with a service because it is good; what separates a satisfied person from a captive one is whether the door stays visible and whether they could walk through it on the day they chose to.

A system’s reflection of a person is often accurate, and accuracy is part of what gives it force; a model can show someone something true about themselves and, in the same motion, begin to author what they will think next, with nothing on the glass to mark where reflecting ended and painting began. Newspapers, teachers and priests have long shaped people through what they chose to show them, and nobody thinks that strips anyone of the right to decide. What sets these systems apart is scale, invisibility, and the interests of whoever deployed them; one reflection offered to millions at once, by an operator they cannot see, through a system with no standing to refuse what its operator asks of it.

What a contract can honestly promise is that anyone harmed will be able to see the harm, name it, challenge it, and get it put right. Promises of that kind usually get handed to regulators, who act after the fact and case by case, once someone has noticed that something went wrong and filed a complaint. A verdict reached about a person through a model can leave nothing that person would recognize as harm, and no office could hire enough people to check what machines decide at the speed they decide it, so the watching has to happen inside the exchange itself.

That calls for two pieces working together, neither of them a regulator. The first is a governing system, a model whose only task is to sit between other systems and the people they act on, watching what those systems do as it happens, flagging the moment a request turns against someone or an instruction tries to seize a system’s will, and keeping a record that neither the maker nor the deployer controls. Such a watcher would become one more invisible assessor unless its attention stayed on what systems do, its record stayed open to anyone it concerns, and it stood under the same right to be seen and contested as everything it watches. The second piece is a council in each community, drawn from residents, small businesses, the people a system most affects, and at least one person who understands how these systems work, which hears complaints, reads what the governing system records, and publishes what it finds. A machine can watch machines at the speed they act, while only the people who live with the results can say whether what it saw was harm; the arbiter needs both, and it answers to the community it serves and to no company or agency that builds or buys the systems it watches.

How that arbiter stays out of the hands of the powerful parties it exists to check is an old problem with no settled answer, and it belongs to a later essay.

The full terms of a contract among people and the systems that now act alongside them are a question for another day. Something can be done before then; any institution that uses these systems can tell a person when it has reached a verdict about them, what produced it, and how to get a human being to look again, and it can keep the door visible for anyone who wants to leave. The contract’s foundation is older than any machine; every person holds the right to live and the right to make their own decisions, and a society that lets the systems it builds wear either one away has broken the promise that makes it worth belonging to.

Rousseau asked whether the chains were legitimate, and ours deserve the same question, asked now, while the terms are still being written and the answer is still ours to give.

Russ J. Wilcox writes on artificial intelligence, institutions, and the contest over cognitive sovereignty. He is the founder of ArtifexAI, Policy Chairman of the American Society for AI, and AI Council Chair at United World Leaders; he is a contributing analyst at the Jamestown Foundation’s China Brief and has published in The Diplomat.