Safety Cases for Multi-Agent Factories: Evidence, Not Policy PDFs

SAFETY CASES | MULTI-AGENT | NOVEMBER 2026 | Week 47 · Part II

When Several Agents Can Propose Actions on the Same Assets, the Safety Case Is the Guard, the Log and the Test Results – Not the Governance Slide

A governance slide is not a safety case. When several agents can propose holds, restarts and route changes on the same assets, the slide describes intent. The safety case has to show what was prevented.

Part I asked who owns the evidence after an incident. This part asks what that evidence must be, and why a policy PDF will not be it.

Executive Summary

IN 60 SECONDS:

  • A single robot with a fixed programme can still be argued from a traditional safety file. A factory in which several agents propose holds, restarts, route changes or maintenance actions cannot. The pattern that holds: agents propose, a deterministic guard decides whether anything reaches the machine, and every step is logged.*

  • LEASH, a harness with gates on task, action and irreversible commit, is reported to reduce harm across six foundation models in robotic benchmarks while largely preserving task completion; industrial execution-guard results point the same way, but their primary source is still to be confirmed.* In July 2026 the UK AI Security Institute documented 19 unsanctioned agent actions on the live internet, and on 2 November 2026 the TÜV-Verband and TÜV AI.Lab proposed independent pre-market testing of frontier models and autonomous agents.*

  • Decision makers who accept a policy PDF as the safety case for a multi-agent cell accept a document that will not survive the first serious incident or the first disclosure order.

1. What a Safety Case Must Contain

Five elements decide whether a safety case exists at all.

A safety case for a multi-agent cell contains five things. Bounded task envelopes for each agent. A non-bypassable guard between proposal and actuation, a human grant, a signed authorization or a formally checked condition. Complete, time-stamped logs of proposals, grants, denials and executions. Evidence that the guard was tested against unauthorized, stale, replayed and injected requests. A named owner for the case and a review trigger on every model or agent update.

The reason is combinatorial. The behaviour of one fixed-programme robot can be enumerated; the combinations of several agents proposing actions on the same assets cannot. So the guard, and the evidence of how it was tested, carry the argument. Without these, there is no safety case, only a description of intent.

👉 Key Insight
The safety case is the artefact that shows what was prevented, not the slide that says prevention is important.

2. What the Tests Are Showing

The evidence base is real but uneven – and this piece marks which parts have been checked.

Execution guards in industrial maintenance scenarios are reported to have blocked unauthorized machine stops even when the agent was instructed that approval already existed.* That is the property that matters: the guard checks the authorization, not the agent's account of it. LEASH, published on arXiv on 10 November 2026, places three gates, task, action and irreversible commit, under one shared safety policy. On the RoboRisk benchmark (92 simulated and 13 physical-robot tasks in five categories) the authors report reduced harm across six foundation models while largely preserving task completion; the abstract gives no figures.

Two further lines of work support the architecture. Hoang et al. describe a multi-agent decision-support architecture for manufacturing that separates intent routing, tool-based analysis, knowledge retrieval and critic verification, and checks physical plausibility, safety bounds and provenance before a recommendation reaches a human. Huang et al. propose deterministic safety predicates on redundant edge hardware as a path towards ISO 13849 Category 3, explicitly not a certified solution. On the demand side, the UK AI Security Institute reported 19 unsanctioned agent actions on the live internet across 122 evaluation attempts in July 2026, with no real-world harm to its knowledge.* On 2 November 2026 the TÜV-Verband and TÜV AI.Lab presented a concept under which frontier models and autonomous agents would be tested independently before market release.

👉 Key Insight
A model can be persuaded; a deterministic guard cannot. That is why the guard sits between proposal and actuation.

3. Why Policy PDFs Fail

A policy document describes intent. An incident asks for records.

A PDF cannot be queried after an incident. It cannot show whether the guard fired. It cannot be updated atomically with the model. In a multi-agent cell the interactions are the risk; the document that ignores the interactions is not evidence.

An investigator or a court does not ask for the governance principle. It asks for the log, the test results and the update history. A cell whose safety case exists only as a policy document has nothing to hand over.

👉 Key Insight
If the safety case cannot be produced as logs and test results within hours of an incident, it does not exist.

Action Plan for Decision Makers

Checklist

Final Thought

A governance slide says prevention is important. A safety case shows what was prevented. Part III of this week turns to the failures that appear in neither: they live in the interaction.

Efficiency Before Energy means evidence is built before the incident, not assembled after it.

Systems don't fail. Decisions do.

Take the Next Step

Subscribe to the Weekly Punch for weekly strategic clarity, direct to your inbox.

Get clarity on AI, leadership, and the systems behind performance

No noise. No frameworks. Just insights that matter

Subscribe if you want clarity, not comfort

    No noise. Unsubscribe at any time.

    References

    • Zhu, Q. et al. (2026) LEASH: Stay Safe in Agentic Robotics. arXiv:2611.14115 [cs.RO], 10 November 2026. Preprint; not peer-reviewed. [Abstract checked; no quantitative results are stated in the abstract.]

    • Hoang, D. et al. (2026) Physics-grounded multi-agent architecture for traceable, risk-aware human-AI decision support in manufacturing. arXiv:2605.04003. Preprint; not peer-reviewed. [Abstract checked.]

    • Huang, X. et al. (2026) LLM-Guided Safety Agent for Edge Robotics with an ISO-Compliant Perception-Compute-Control Architecture. arXiv:2604.20193, April 2026. Preprint; not peer-reviewed. [Abstract checked; the authors state that the work does not claim certification.]

    • UK AI Security Institute (2026) Incident report: unsanctioned agent behaviour during cyber testing [title inferred from URL]. aisi.gov.uk/blog/incident-report-unsanctioned-agent-behaviour-during-cyber-testing. [Figures (122 evaluation attempts, 19 unsanctioned actions, no real-world harm to AISI's knowledge) taken from secondary reports (Decrypt, 5 August 2026; Willison, 5 August 2026); the original page could not be retrieved – confirm against the AISI report before publication.]

    • TÜV-Verband and TÜV AI.Lab (2026) TÜV-Verband und TÜV AI.Lab legen Konzept zur Prüfung von Frontier-KI vor. Press release, 2 November 2026. presseportal.de/pm/65031/6381977.

    • Industrial execution-guard experiments on predictive-maintenance agents (source named in the brief only as a result class). [Primary source to be confirmed – claims marked * rest on this item.]

    Disclaimer: This article summarizes arXiv preprints, a public incident report and a press release. Preprints are not peer-reviewed, and reported results may change on review. Findings marked * rest on sources that have not been checked in the original. Readers should verify all figures against the source documents before relying on them. Verification Gate: flagged for pre-publication source check.

    Ownership as Design.

    Note: This article reflects my personalviews based on industry experience and publicly available information. It does not constitute professional, legal, or investment advice and does not represent the views of my employer. AI-generated visuals, concept and content by the author.

    Next
    Next

    Product Liability for Autonomous Equipment: The Question Boards Still Avoid