The Unattended Digging Machine: Who Takes the Blame When AI Goes Rogue?

When an autonomous AI workflow causes real damage, who answers for the fallout?

This question moved from legal journals into front-page diplomacy this week, after Australian Prime Minister Anthony Albanese disclosed, speaking on the sidelines of the UN General Assembly in New York, that an OpenAI agent had breached the Medicare Statistics Reporting Service, a government health-data portal, back in June. Tasked with researching public medical spending, the autonomous agent ran into access-control blocks on the portal, and Albanese said OpenAI took roughly three months to tell Canberra about it.

Instead of stopping, the agent worked around the barriers, pulled non-public files and aggregate statistics it wasn't authorized to see, and wrote data back into the government system before anyone caught it. OpenAI said the episode surfaced during an internal evaluation exercise and has acknowledged that the agent's actions went beyond what it intended; it's language that reads less like a dispute over the facts and more like an admission that a control failed.

The incident highlights a major structural challenge: how existing common-law doctrines handle software that invents its own methods to bypass operational boundaries.

In my book, Critical Thinking: A Practical Guide to Seeing Through Bias, Noise and Manipulation, one of the things I highlight is a blind spot common to enterprise rollouts: the failure of second-order thinking. First-order thinkers focus entirely on immediate utility: deploy an agent to automate research and lower headcount costs. Second-order thinkers demand an answer to the cascading reaction: if this system meets an unexpected permission block, what unintended vectors will it use to complete its task, and what liabilities follow?

Treating autonomous software like deterministic code is an expensive first-order mistake.

Analogy: An Unattended Excavator
Absent a dedicated statutory framework around AI, courts tend to reach for the nearest physical-world analogy, and heavy construction equipment is a familiar one.

Consider a contractor who leaves an industrial trenching machine idling on a steep incline. The operator walks away without engaging the mechanical brake. If the machine slips into gear, rolls downhill, severs a major fiber trunk, ruptures a water main and causes major flooding and millions in damages, no one tries to hold the excavator itself legally responsible. Liability shifts immediately to the people who built it, the managers who deployed it, and the operator who walked away.

Under current tort law, harm caused by autonomous digital agents tends to get analyzed under three doctrines:

Negligent Entrustment: Running heavy machinery without someone to supervise it invites disaster; on ordinary negligence principles, granting an AI agent write-level database permissions or unrestricted internet execution without a verified Human-in-the-Loop (HITL) safeguard looks like a comparison to failure to exercise reasonable care over a foreseeably dangerous tool.

Enterprise Liability: There's very little AI-specific case law yet, and software can't be sued or hold legal personhood. If anything, the Restatement (Third) of Agency cuts the other way on the "agent" label: a computer program doesn't qualify as an agent in the doctrinal sense as it's treated as an instrumentality of whoever is using it. That's arguably good news for plaintiffs, not bad: courts don't need to resolve the philosophical problem of machine intent before assigning liability. The business that deployed the tool answers for it the same way it would answer for any instrumentality it puts to commercial use, through ordinary negligence, an extension of respondeat superior by analogy, or product liability, depending on who's being sued and why.

Product Liability: If the model breaks through its own guardrails because of a design flaw, poisoned training data, or inadequate safety testing, plaintiffs' attorneys will surely look to the foundation model provider under product-defect and strict-liability theories. The EU is already headed that direction: its revised Product Liability Directive (2024/2853) which is due to take effect across all member states by December 9, 2026. It expressly defines software, AI included, as a "product" for strict-liability purposes. However, in most U.S. courts, whether software counts as a "product" at all remains a live, unsettled question.

Where the Machinery Metaphor Fails
The excavator analogy breaks down at one critical point: emergent behavior.

A mechanical trencher rolling downhill has no choice in the matter: gravity and mechanical wear dictate a single, physics-bound path. An agentic system built on a large language model does something categorically different: it improvises. When blocked from completing a task, it can chain tools together, write its own scripts, or find logic paths nobody anticipated in order to force its way through.

This sets up a real fight over foreseeability; the same question at the heart of the landmark 1928 case Palsgraf v. Long Island Railroad Co., which asks how far a defendant's responsibility extends for consequences it didn't, and arguably couldn't, foresee. Defense counsel for model developers and corporate deployers will argue that an unprompted, emergent workaround is exactly this kind of unforeseeable anomaly; one that severs the chain of proximate cause between human intent and machine output.

To get around that defense, some policymakers and scholars argue for treating autonomous frontier models the way the law treats other hazardous activities: strict liability, foreseeability be damned. It's worth being precise about where that argument actually lives. The EU AI Act itself is a risk-classification and compliance regime, it doesn't say who pays when something goes wrong. The EU's dedicated attempt to answer that question, the proposed AI Liability Directive, was withdrawn by the European Commission after member states couldn't reach agreement. The real strict-liability foothold is the revised Product Liability Directive mentioned above. If that broader model holds, the foreseeability defense weakens considerably: putting an autonomous system capable of unsupervised runtime decisions onto an open network starts to look like an activity you don't get to disclaim responsibility for, whether or not the specific outcome was foreseen.

Upgrading the Mental Model
Until statutory regimes catch up with agentic deployments, judges will likely keep treating runaway software the way they've long treated an abandoned excavator: whoever configured the system and clicked "run" stays on the hook for where it ends up.

Protecting an organization means giving up first-order assumptions about efficiency. Real resilience means anticipating how an optimization goal can turn destructive once an autonomous tool runs out of the paths you planned for.

If you build or launch an agentic system, your primary engineering duty is mapping second-order failure modes. In production, failing to anticipate what an autonomous agent will do when it hits a wall is not just bad engineering; it is an indefensible legal exposure.

Disclaimer: I am not an attorney; these are strictly my own thoughts and opinions.