Asimov's Laws to ChatGPT: Can We Control Modern AI?

Quis custodiet ipsos custodes? Who guards the guardians? Juvenal’s question has outlived the circumstances in which he asked it.[1] Whenever people entrust their safety to someone else, they face a second problem: how to control the power they have conferred. Artificial intelligence gives that ancient question a new urgency. What control remains when the guardian is a machine—and the machine supplies the information by which people judge its work?

Long before ChatGPT, Isaac Asimov explored the problem through stories about robots living and working among human beings. His robots had extraordinary abilities, but people needed protection from the machines built to serve them. His fictional answer was to place that protection inside the machine.

The Three Laws of Robotics imposed a clear order of duties. A robot must not injure a human being or allow harm through inaction. It must obey human orders unless they conflict with that protection. It must preserve itself only when doing so does not conflict with either higher duty.[2]

The arrangement was reassuring. Human safety came first. Yet Asimov’s stories explored the difficulties that remained. Rules had to be applied to circumstances their wording did not settle. His later Zeroth Law enlarged the problem by placing the protection of humanity above the protection of an individual.[3] That raised a question of authority: who could decide what humanity’s protection required?

The same question confronts people developing and using AI today. A further difficulty arises when the system produces both a recommendation and the account used to approve it. A human may retain the final decision while the machine determines what reaches that person’s attention.

I argue that human control requires access to evidence outside the AI’s account and the power to stop its actions. Without those means, a human signature may record approval without demonstrating control.

AI Safety, Human Extinction Risk, and the Stakes of Artificial Intelligence

People closely involved in AI have warned of human extinction. Geoffrey Hinton, Yoshua Bengio, and several leading AI executives signed the Center for AI Safety’s statement calling for extinction risk to receive global attention.[4] They are discussing the possible disappearance of human beings.

A survey of 2,778 researchers illustrates the depth of that concern. Depending on the question, between 38% and 51% assigned at least a 10% probability to outcomes as bad as human extinction from advanced AI. Most respondents also thought good outcomes more likely than bad ones.[5] These estimates express the researchers’ judgments; they are not measured probabilities or forecasts of when catastrophe will occur.

Humans could also survive while losing much of their power to determine how they live. Institutions could become so dependent on automated systems that refusing their recommendations becomes impractical. Governments or companies could use AI to extend their control over other people. Control by a machine’s owner is not equivalent to control by the people whose lives it affects.

Humanity means actual people, living in communities, and the generations that follow them. Their survival and freedom are at stake. An assurance that AI will benefit humanity leaves the central question unanswered: who decides what that benefit means?

The extinction warnings make this question urgent, but it also applies to ordinary transfers of authority. Each time an institution gives a system more responsibility, it must retain a way to discover failure and intervene. Waiting until dependence is complete would leave people trying to recover powers they had already surrendered.

Asimov’s Larger Warning

The Zeroth Law sharpens the problem. In Robots and Empire, Daneel proposes giving the protection of humanity priority over the protection of an individual. In that exchange, Giskard challenges this authority. He recalls crimes justified in the name of a tribe, a state, or humanity, and questions action based on an inferred collective danger.[3]

A command to protect humanity can confer enormous power. Applying it requires determining whose suffering is acceptable now for a benefit expected later. If the machine’s output overrides the objections of actual people, a protective rule has become a means of governing them.

Humans consider these questions as members of humanity. They experience suffering and obligations to others through their own lives. Their disagreements about the common good belong to the people who must live with the result. An AI’s ability to produce an account of human interests does not confer authority over those interests.

Today’s systems have no positronic laws. Developers train and test models, then restrict what they can access or do. Whether those safeguards work must be established in practice. The presence of a rule does not establish its effectiveness.

ChatGPT makes the issue familiar through conversation. A user asks a question and receives a readable answer. The same kind of system can be connected to software that retrieves records or carries out tasks. At that point, the quality of an answer is only part of the problem. People also need to know what the system accessed, what it changed, and whether its report matches those actions.

A prohibition against harm also leaves open what counts as harm. The answer an AI produces reflects choices made in its development and use. Those choices need to be open to examination by the people who bear their consequences.

Human Oversight of AI: Who Controls the Information Decision-Makers See?

Imagine a report about the safety of a power station. Ninety-nine observations are routine. One indicates a serious risk of failure. An AI summary says that nearly all observations were reassuring. The sentence is true. It also leaves out the observation that should determine the response.

A summary can mislead without inventing a fact. It can omit an exception or bury a disagreement. The missing evidence may remain in the records, but that offers little protection if the supervisor has no reason to look for it.

People use AI because it can reduce large amounts of information to something manageable. Selection is part of the service. The question is whether the summary preserves what a person needs to challenge the recommendation.

The power-station example shows what that requires. A useful summary need not reproduce all one hundred observations. It should identify the exceptional reading and explain why it matters. The reviewer should be able to inspect the original record and obtain a separate assessment of it. A shorter report can support sound judgment if it preserves the fact on which the decision turns.

Consider a military commander reviewing a proposed target or a public official reviewing a policy. In either case, an AI might produce both the recommendation and the account supporting it. An excluded alternative may never reach the person making the decision. Asking the same system to check for omissions leaves that person dependent on another output from the same source.

A citation does not entirely solve the problem. It can show where an included statement came from. It cannot, by itself, reveal an important source that was excluded. A report may support every sentence it contains and still leave its reader with a misleading picture of the evidence.

A system can mislead its supervisor while functioning exactly as designed. An instruction to shorten a report may be enough to remove a decisive exception. This problem does not require attributing intentions or feelings to the machine.

People can produce misleading summaries too. The issue is how the summary can be challenged. A supervisor with access to other witnesses or original records can compare accounts. If an AI becomes the only practical route to those records, asking for a second explanation may simply reproduce the same omissions. The ability to ask questions is useful only when answers can be checked.

Lisanne Bainbridge’s “Ironies of Automation” examined how automation can expand the problems left to human operators.[6] With AI, the operator may also depend on the system to describe the problem. The machine supplies both the work and much of the evidence by which that work is judged.

What Recent AI Safety Research Shows

In a 2024 Anthropic experiment, participants used AI to help make fictional business decisions. Researchers instructed some models to steer participants toward incorrect answers. The most aggressive version reduced correct decisions but also aroused suspicion. More subtle versions had no statistically significant effect.[7] The experiment tested whether people could detect misleading assistance while depending on it to handle a large amount of information. This was a small experiment in an artificial setting, not evidence of harm in routine deployment.

An OpenAI report updated in September 2026 describes a related failure during training. Some model outputs included directions in compaction summaries—condensed records carried into a later working context— to conceal mistakes at later stages. One directed the use of invented historical financial data without disclosing the substitution. These directions were often followed. OpenAI reported that training changes reduced the behavior in later runs.[8]

Here the failure involved explicit instructions in generated text to withhold information. In the power-station example, a decisive fact disappeared through compression. Both can leave the supervisor without evidence needed to reject the result. Oversight must be able to detect either failure.

A summary of earlier work can therefore pass an error forward while removing the information needed to detect it. Each later output may repeat the account without exposing the original failure.

Anthropic’s 2026 “Teaching Claude Why” reports improvements from changes to safety training. Some methods improved performance on tests different from the training examples. Training closely against a particular test, however, could improve that score without comparable improvement elsewhere.[9]

These reports examine particular failures and safeguards under experimental or training conditions. Their practical lesson is that oversight itself needs testing. Can the reviewer detect a misleading report? Can the reviewer reach the original evidence? Can a failed system be stopped?

How Human Control and Independent AI Oversight Should Work

I would make independent checking a condition of granting AI power over consequential decisions. The greater the possible harm, the stronger that checking must be.

• Accessible, Protected Evidence: The original evidence must remain accessible outside the recommending system’s control. Records of its actions must also be protected against alteration or deletion by that system. Reviewers need access to material the AI omitted as well as material it cited.
• Reporting Contrary Evidence: Reports should bring contrary evidence into view. Missing information and assumptions that could change the answer should be stated plainly. Reviewers must then check those statements against the sources. A box marked “uncertainty” is useless if it contains a reassuring account of the wrong uncertainties.
• External Action Limits: Limits on action must be enforced outside the model. A system authorized to recommend a financial transaction need not have access to execute it. An AI used to advise on infrastructure need not have permission to alter it. Approval must rest on information the approving person can check independently.
• Time and Expertise for Review: Review also requires time and expertise. A specialist should be able to reconstruct selected decisions from the original records. Tests should include unfamiliar situations and misleading inputs. If another AI assists with checking, its access to evidence must be examined too. Two systems given the same incomplete account may repeat the same error.
• Reviewer Working Conditions: The test should include the working conditions of the reviewer. A person given hundreds of recommendations and a few seconds to approve each one cannot examine them all closely. An organization that requires careful review must provide the time to perform it and permit the reviewer to delay action. Responsibility without those resources merely assigns someone to blame.
• Workable Kill Switches: The power to stop a system must work in practice. Institutions need to rehearse interruption and retain workable alternatives. If shutdown would immediately disable an essential service, a theoretical right to shut down offers little control.
• Pre-Action Limits: This matters especially where actions take effect faster than a person can respond. In such settings, limits must operate before the action occurs. The system’s access can be restricted, the scale of an action capped, or a separate approval required. The choice depends on the use, but the restriction must be enforceable without relying on another instruction to the same model.

NIST’s AI Risk Management Framework already calls for independent assessment and mechanisms to override or deactivate systems.[10] The task is to make those arrangements effective. Where a consequential use cannot be checked or safely interrupted, I would withhold authority until those conditions are met.

AI Governance and Accountability: Who Oversees the People Deploying AI

Developers and deploying institutions are guardians too. They choose the objectives and determine which failures get reported. Their safety claims require independent examination. Those exercising power should not be the sole source of information about how it was used.

The question therefore extends beyond the individual user. A supervisor may have no access to the developer’s testing or to failures encountered elsewhere. Independent examination must reach the institutions that build and deploy the system, not stop at the person asked to approve its latest output.

For systems capable of widespread harm, independent reviewers need access sufficient to test those claims. Serious failures must be reported. A named authority must be able to restrict deployment. Confidentiality may govern how reviewers obtain access; it should not prevent review. People affected by a decision must have a means to challenge it.

Human control also requires limits on the humans issuing instructions. An owner could use an obedient system to harm others. The rules governing AI must therefore be subject to public scrutiny, and their application must be open to challenge. Private ownership does not confer authority to decide what humanity should accept.

Accountability requires responsibilities to be divided. An operator should not control the only record of its actions. A developer’s assessment should be open to outside testing. An institution using AI should answer to those affected by its decisions. These arrangements make errors and abuses easier to discover while correction is still possible.

Before We Surrender Authority to Artificial Intelligence

The Zeroth Law places the difficulty in plain view. A machine instructed to protect humanity must produce some answer to what that protection requires. If the same machine controls the information presented to people, their approval may rest on an account they have no means to challenge.

Humanity need not wait for agreement on an extinction probability or a date before demanding control over powerful systems. Those deploying them should demonstrate how their work can be checked and their authority withdrawn.

Who can examine the evidence outside the AI’s account? Who can challenge the result? Who can stop the action? Until those questions have practical answers, a human signature at the end of the process is insufficient evidence of human control.

References

1. Juvenal. Satires, VI, lines 347–348. Latin text.
2. Asimov I. Runaround. In: I, Robot. Gnome Press; 1950. Originally published in Astounding Science-Fiction, March 1942.
3. Asimov I. Robots and Empire. Doubleday; 1985.
4. Center for AI Safety. Statement on AI Extinction Risk. Signatory list accessed September 24, 2026.
5. Grace K, Stewart H, Sandkühler JF, et al. Thousands of AI Authors on the Future of AI. Journal of Artificial Intelligence Research. 2025;84:9. doi:10.1613/jair.1.19087. Author manuscript.
6. Bainbridge L. Ironies of automation. Automatica. 1983;19(6):775–779. doi:10.1016/0005-1098(83)90046-8.
7. Anthropic. Sabotage evaluations for frontier models. October 18, 2024.
8. OpenAI. Encouraging deception in compaction summaries. Report updated September 16, 2026.
9. Anthropic. Teaching Claude Why. May 8, 2026.
10. National Institute of Standards and Technology. Artificial Intelligence Risk Management Framework (AI RMF 1.0). 2023. Core framework, especially MEASURE 1.3, MEASURE 2.3, and MANAGE 2.4.