What agents should be allowed to do in a plant

Autonomous action in a production environment is not a software question — it is a governance question. How approval loops, escalation paths, and audit trails turn agents from liabilities into assets that shift leads actually trust.

The discussion about AI agents in manufacturing tends to collapse into one of two positions: 'the AI should handle it automatically' or 'the AI should never touch production systems.' Both positions avoid the actual design question, which is considerably more nuanced and considerably more important.

The fault detection problem is largely solved. Modern anomaly detection on historian data, combined with pattern matching against prior incident records, can identify developing faults with reasonable accuracy on well-instrumented equipment. The hard question is not 'can the AI detect the fault.' It is: once the fault is detected, what should the system be allowed to do next, and under what conditions?

Getting this wrong in either direction is expensive. Too restrictive, and you have a very sophisticated alarm system that requires the same human response time as the SCADA it replaced. Too permissive, and you have autonomous changes to production systems without adequate human review — which, on a real production line, can cause damage, safety events, and quality escapes that dwarf the cost of the downtime you were trying to prevent.

The action spectrum

Agent actions in production exist on a spectrum defined by two axes: consequence magnitude (how bad is a wrong action) and reversibility (how easily can a wrong action be corrected). The appropriate level of autonomy at any point on that spectrum is a function of both.

→Read-only analysis: signal retrieval, pattern matching, historical comparison, anomaly flagging. No consequence from a wrong output other than wasted attention. Appropriate for full autonomy.
→Notification and recommendation: generating an alert with supporting evidence and a suggested action. No production impact from the recommendation itself; the human decides whether to act. Appropriate for full autonomy with quality controls on evidence grounding.
→Parameter suggestion with approval gate: recommending a specific setpoint change — 'increase barrel temperature Zone 3 by 6°C' — surfaced to the operator with source citations, pending approval before any change is made. Appropriate with a single-person approval gate and a clear approval interface.
→Approved setpoint adjustment: the agent executes a parameter change that a qualified person has reviewed and approved via an authenticated action. Appropriate with approval gate, audit trail, and rollback capability.
→Autonomous correction within bounds: the agent executes changes within pre-defined operating windows without per-action approval — for example, automatic PID retuning within ±10% of nominal setpoints during non-critical process steps. Appropriate only for well-understood, bounded, low-consequence adjustments with continuous monitoring and automatic rollback triggers.
→Autonomous shutdown or safety action: triggering a controlled stop or engaging a safety system. Appropriate for systems that are faster than human reaction time in safety-critical contexts, but requires IEC 61511 / IEC 62443 compliance, formal hazard analysis, and is distinct from process optimization.

Why the boundary is not fixed

The same action can be appropriate at one confidence level and inappropriate at another. An agent recommending a pressure setpoint adjustment based on strong evidence from three similar prior incidents, all resolved by the same adjustment on the same machine model, is in a different position from an agent recommending the same adjustment because a pattern-matching algorithm flagged nominal resemblance to a prior case from a different product line.

Confidence scores matter here — not as a single number, but as a structured statement of what evidence supports the recommendation and what would change the assessment. A recommendation with explicit evidence grounding — 'this setpoint corrected the same deviation on M-IMM-02 on three separate occasions; signal trace confirms identical Tag-ZB-042 deviation sequence; prior cases approved by senior process engineer J. Weber' — earns a different approval threshold than a recommendation with no cited evidence.

The practical implication is that the action boundary should be dynamic, not fixed. As the system accumulates evidence on a specific machine type and fault mode, the appropriate level of autonomy for well-characterized cases can increase. For novel situations — new equipment, new process, first-occurrence faults — the boundary should pull back toward recommendation-only regardless of what the algorithm's confidence score says.

Designing the approval loop

An approval gate is only as good as what the approver can see. The most common failure in industrial AI approval loops is that the approver is given a yes/no decision on a recommendation they cannot evaluate — either because the supporting evidence is not visible, the recommendation is expressed in terms the approver does not directly understand, or the approval interface is buried in a system the operator does not use during active fault conditions.

Effective approval loop design requires: the recommendation expressed in process-native terms ('increase Zone 3 barrel temperature setpoint from 228°C to 234°C'); the evidence that supports it ('Zone 3 running 5.8°C below setpoint for 3.2 hours; same deviation preceded pressure fault in INC-0712, INC-0834, INC-0901'); the confidence level and what would lower it; who is authorized to approve; and a clear rollback path ('setpoint can be returned to nominal in under 30 seconds via HMI Tag ZB-ZONE3-SP').

The approval record becomes part of the incident's structured evidence. Who approved what, when, with what information visible to them. This creates an audit trail that is legally and operationally important, and it also generates the feedback signal the system needs to improve its own calibration over time.

Trust is earned incrementally

Production teams are appropriately skeptical of automated systems that act on their processes. This is not conservatism or technophobia — it is rational risk management from people who have seen automated systems cause significant problems. The right response is not to override the skepticism but to design for incremental trust building.

Start with read-only analysis and evidence-grounded recommendations. Run the system long enough for operators to observe that its recommendations, when followed, produce the predicted results — and that when it expresses low confidence, the situation is genuinely ambiguous. Once the team has direct experience with the system's accuracy and honesty, extending the approval boundary to include parameter suggestions makes sense. Autonomous execution comes later, if at all, for specific well-characterized cases that meet explicit confidence and consequence thresholds.

The question is not whether the AI is capable of making the right decision. The question is whether the organization has sufficient evidence to trust it to make that specific decision, in that specific context, without human review.
— Design principle for governed production agents

Systems that earn trust incrementally stay in production. Systems that are deployed with broad autonomy before the organization understands their behavior get switched off after the first significant error — regardless of their long-term potential value.