What does it mean to supervise?
Monitoring an AI system means verifying its operation and results with the ability to intervene. In occupational safety and health (OSH), the goal is to prevent an incorrect or incomplete output from becoming a decision that affects safety and health. Monitoring must be integrated into the process where the tool is used.
The supervisor needs to understand the system’s objective and how its response will be used. Reviewing an informational draft is not the same as accepting a recommendation that modifies a control measure. Evaluating the use of AI helps determine where and how intensive intervention is needed.
Competencies and responsibilities
Competence combines job knowledge, training on the tool , and an understanding of its limitations. The individual must be able to recognize when information is missing, when a recommendation contradicts actual conditions, and when specialized support is needed. Knowing how to use an interface does not qualify one to validate any result.
The organization must assign roles and replacements, and define who can approve, correct, or stop each process. Oversight should not be used to shift all responsibility to whoever reviews the latest document. Those who design, select, and organize the use of the system also make decisions that affect its reliability and impact.
Enough information to review
The review requires sources, context, and criteria. In an accident report, records, time period, population, and calculations must be verifiable. In a preventive procedure, it is necessary to compare tasks, equipment, and measures. A clearly written text may still be inadequate if it is based on outdated information or information from another center.
Data quality and the presentation of uncertainty influence control. It is important to distinguish between recorded facts, inferences, and proposals. The reviewer must be able to access what is necessary without receiving so much information that verification becomes impractical. Adding more screens does not always make it easier to understand what supports the decision.
Avoid automatic approval
Over-reliance on automated systems can reduce critical verification. A quick response, a precise figure, or a professional interface can create an impression of reliability that exceeds the available evidence. The process should encourage the detection of errors and allow for the rejection of a proposal when it is not sufficiently substantiated.
It is helpful to define risk-related checks: consistency with the assignment, adequacy of sources, essential data, and consequences of implementing the result. The organization should allocate time for this review. If the volume of proposals exceeds available capacity, the process should be modified rather than turning validation into a routine confirmation.
Intervene and maintain an alternative
The ability to control an issue includes correcting an output, requesting additional information, and suspending its use when appropriate. It must be clear what happens after a discrepancy arises and who resolves it. An issue reporting channel that doesn’t produce a response leaves the user with a choice between accepting a dubious recommendation or halting work without support.
When AI is involved in critical tasks, a viable alternative procedure must exist. For AI agents, in addition to reviewing the final text, it may be necessary to monitor actions related to records or communications. Monitoring should occur before the effect that needs to be prevented; detecting an error after distribution serves a different purpose and may require further corrections.
Relationship with the regulatory framework
Article 14 of the European AI Regulation provides for human oversight of high-risk systems, proportionate to the risks, autonomy, and context. This includes understanding boundaries, interpreting results, and intervening. Its practical application requires verifying the classification, the organization’s role, and the relevant regulatory timeline.
These rules are coordinated with applicable preventive and professional obligations. Not every occupational safety and health (OSH) tool automatically receives the same legal classification. Even if a specific regulatory requirement is not applicable, the company must analyze the occupational risks associated with its use. Supervision is a control measure that must demonstrate its usefulness in the real-world context.
Practical example
In a hypothetical situation, a system drafts an assessment proposal based on previous documents. The reviewer notices that a manual operation that is now automated is described, and that a new maintenance task is missing. Simply correcting names and dates would have resulted in an incorrect preventive diagnosis.
The review requests information about the change, checks the task, and redoes the affected parts. It also reports the error to prevent reusing the document as a valid reference. The result is incorporated into the preventive process with the corresponding professional review. The tool provides a starting point, but the validation verifies the work that is actually performed.
Measure if the control works
The organization can analyze detected errors, corrections, escalated cases, and failures that passed review. It’s important to interpret this data in conjunction with workload and case complexity. A very low rejection rate can indicate good performance, but also a superficial review; the number alone doesn’t resolve the issue.
The evidence from the review should allow for the reconstruction of relevant decisions without generating disproportionate bureaucracy. The evidence and observations must be updated when the tools, users, or purpose change. The goal is to verify that human intervention improves decisions and has sufficient conditions to continue doing so.
