TLDR: A study explored using real-time operational data from control room operators to predict their responses to critical alarms. Researchers used a formaldehyde plant simulator with 92 participants across four support configurations (baseline, alarm rationalization, on-screen procedures, AI decision support). They found that data like alarm counts, response times, and interactions with systems can predict operator errors. While advanced systems generally improved performance, alarm prioritization with traditional paper-based procedures surprisingly showed the highest success in the most complex scenarios. The research highlights the value of operational data and the need to tailor support systems to task complexity.
In the complex world of industrial control rooms, operators are the frontline defense against accidents and unexpected events. They monitor plant processes, manage alarms, diagnose issues, and execute critical control actions. However, these vital roles come with significant challenges, especially as plants become more automated, sometimes leaving operators ‘out of the loop’ until a critical alarm demands immediate attention.
Traditional methods for assessing operator performance, such as human reliability analysis or intrusive physiological measurements like eye-tracking and EEG, often fall short. They can be time-consuming, require expert assessment, or are simply not practical for daily operations. This highlights a crucial need for a predictive approach – one that can anticipate factors hindering correct alarm responses and enable timely support for operators without disrupting their work.
A recent research paper, titled “Managing the unexpected: Operator behavioural data and its value in predicting correct alarm responses” by Chidera W. Amazu, Joseph Mietkiewicz, Ammar N. Abbas, Gabriele Baldissone, Davide Fissore, Micaela Demichela, Anders L. Madsen, and Maria Chiara Leva, delves into this very challenge. The study explores the potential of using real-time data from process and operator-system interactions – information that can be readily recorded and retrieved from a plant’s distributed control system (DCS) or process logs. This includes metrics like alarm acknowledgements, response times, the number of alarms, how often operators open mimic displays (visual representations of the plant), and how frequently they access procedures.
The researchers conducted an experiment using a formaldehyde production plant simulator with 92 participants. These participants, mostly junior process engineers, were divided into four groups, each with a different level of support: a baseline group without alarm rationalization, a group with alarm rationalization, a group with on-screen procedures, and a group integrating an AI-based decision support system (DSS).
The study’s findings offer valuable insights into how different support systems impact operator performance across varying levels of task complexity. Alarm prioritization, for instance, showed clear benefits, especially in scenarios with a high intensity of alarms. It led to improved alarm management and better overall performance, suggesting its effectiveness in managing cognitive load and focusing operator attention on critical issues.
The transition from paper-based to digital procedures yielded mixed results. While digital procedures improved response times and potentially reduced alarm clutter in less complex scenarios, their impact was less significant in highly complex situations. In some cases, they even led to less efficient alarm management, indicating that careful consideration and additional training might be needed for their full potential to be realized.
Perhaps the most intriguing findings came from the integration of the AI-based DSS. In low to moderate complexity scenarios, the AI system showed promising results, improving decision accuracy, reducing reaction and response times, and facilitating faster problem resolution. However, its effectiveness was limited in highly complex scenarios. Surprisingly, in the most challenging conditions, the group with alarm prioritization and *paper-based* procedures often showed the highest probability of success, outperforming both digital procedures and AI-based DSS. This suggests that in highly complex situations, the familiarity and simplicity of traditional systems, combined with effective alarm prioritization, might offer advantages over more technologically advanced solutions.
To predict operator errors, the researchers employed two machine learning algorithms: step-wise Logistic Regression (LR) and Bayesian Networks (BN). Both models demonstrated strong predictive capabilities, identifying key behavioral metrics such as the number of alarms, scenario complexity, acknowledgements, mimics opened, and response time as significant predictors of errors. The study confirmed the importance of situational awareness and workload in predicting correct alarm responses.
Also Read:
- Human-Robot Teams Boost Concrete Crack Detection in Nuclear Facilities
- Are LLMs Truly Collaborative Partners? A Study’s Insights into Human-AI Interaction
In conclusion, this research highlights the immense value of exploring operational data to understand operator behavior and its impact on safety. It underscores that while advanced technologies offer significant benefits, their effectiveness is not uniform across all situations. The findings emphasize the importance of tailoring support systems to specific operational contexts and complexity levels, rather than assuming that more advanced technology will always lead to better outcomes. This study paves the way for future research into dynamic models and a wider range of variables to achieve even more robust predictions. You can read the full paper here.
