Back to Blog
Blog

How do you validate AI identified fatal risks against reviewed fact sheets?

How do you validate AI identified fatal risks against reviewed fact sheets?

AI can scan procedures and flag steps that look like they involve a fatal risk, but a flag on its own is not something a safety team can rely on. A…

AI can scan procedures and flag steps that look like they involve a fatal risk, but a flag on its own is not something a safety team can rely on. A model may recognize language associated with working at height, stored energy or confined spaces, but it can also miss risks that are described indirectly, or flag steps where the hazard has already been controlled earlier in the procedure.

The flag needs to be checked against the company’s own reviewed fatal risk standards or fact sheets, which describe what the risk is and what controls are required. Those documents represent decisions the company has already made about how it defines each fatal risk and what minimum controls apply, so they are the right reference for judging whether a flag is correct, rather than a general definition the model learned somewhere else.

The practical setup is to have the AI cite the specific fact sheet and the text in the procedure that triggered the flag, so a reviewer can confirm or reject it quickly. A reviewer looking at a flag should see the procedure step, the fact sheet the AI matched and the reason, and be able to accept it, reject it or reassign it to a different risk, with a short note explaining any rejection so the decision is clear to whoever looks at it later.

Over time, the confirmed and rejected flags show where the AI is reliable and where it needs tuning, and the final call always stays with someone qualified. Patterns in the rejections are especially useful, since a fact sheet that produces many false flags may need clearer wording, and steps that reviewers keep adding by hand point to risks the AI is not recognizing.

Reviewers should be people who know both the fatal risk standards and the work, and the review should be built into the normal procedure approval process rather than handled as a separate exercise. It also helps to sample a set of procedures with no flags at all, since checking what the AI missed is as important as checking what it found, and a clean result is only reassuring once it has been tested.