Measuring Human Performance through Normal Operations Monitoring

Most high hazard industries know that human performance is important to safety outcomes but few routinely measure the standard of human performance in their operations. Most managers recognise it is difficult to manage what... [ view full abstract ]

Most high hazard industries know that human performance is important to safety outcomes but few routinely measure the standard of human performance in their operations. Most managers recognise it is difficult to manage what you cannot measure, however many organisations only collect data about human error by exception, through incident reports.
Normal Operations Monitoring (NOM) provides a tool for measuring a baseline of human performance in everyday operations. NOM data can provide many benefits such as: proactively investigating where human risks may arise; highlighting problems with procedures or working environments; providing data about adaptive and maladaptive behaviours, evaluating training effectiveness, informing evidence based safety interventions; and measuring the effectiveness of interventions through behaviour change and risk reduction.
This paper outlines the development and application Normal Operations Monitoring based on an adaptation of the Line Operation Safety Audit (LOSA) approach from aviation (Klinect, Murray Merritt, & Helmreich 2003). The development of a new method for observation, coding and data analysis described within the context of aviation ground handling. A new model for interpreting human performance is proposed. Results from over 1200 observations are presented and discussed.
To demonstrate the benefits of NOM approaches within the rail context, a case study from an Australian rail signalling centre is presented. In this example an adapted version of a NOM method is applied in signaller communications. A coding system was developed based on previous research into safety critical communications (RSSB 2004 TO14). Over 200 recording of communications were sampled and coded to establish a baseline of human communications performance, inform interventions and monitor their subsequent effectiveness. Results of the baseline data and potential benefits are discussed.
Finally the case is made for the adoption of Normal Operations Monitoring more broadly within the rail industry in order to: deepen our understanding of human performance in rail; improve risk modelling with human risk data; inform future research priorities, and to develop targeted, evidence based, improvement initiatives. Periodic ongoing monitoring of human performance will also help to measure the effectiveness of interventions and demonstrate return on investment of human factors through safety improvement in the rail industry.