Eye-tracking analysis during crisis management simulations: focusing on the leader's visual trajectory in emergency care settings, through the use of special glasses.
In a groundbreaking study, researchers have delved into the world of eye-tracking analysis to explore the non-technical skills of healthcare professionals (HCPs) in emergency care settings. The focus of this study is on situational awareness (SA), a crucial skill for leaders in critical situations.
The SA framework, consisting of three levels - perceiving an event, understanding what is being perceived, and making predictions - serves as a reference in this study. By analysing gaze patterns at team members' faces and bodies during requests, the study aims to provide insights into HCPs' SA, particularly at the third level, projection of future status.
The research is based on the technology of eye-tracking to analyse HCPs' gaze behaviours. Methods for integratively analysing gaze and utterances involve a multimodal approach, combining eye-tracking, audio/video recordings, natural language processing (NLP), and visualization tools.
Data Collection & Preprocessing is the first step, capturing multimodal data such as gaze (via eye-tracking), audio of spoken utterances, and video of interactions. Transcriptions of speech are processed with NLP techniques to prepare data for analysis.
Multimodal Data Fusion follows, combining gaze data with utterance transcripts and video to analyse behavioural cues in context. This fusion enables capturing not only what is said but where attention is directed, which is critical for assessing SA level 3 (anticipating future events).
Visualization Techniques, such as heatmaps of gaze distribution over time, help identify areas of attention and neglect. Correlating these with spoken content segments in time-aligned multimodal visualizations aids interpretation of focus and intention during emergency interactions.
NLP & Topic Modeling methods classify and segment spoken content to understand task-related versus non-task-related dialogue components. These methods help interpret SA based on the semantic content of utterances aligned with gaze patterns.
Frameworks for Collaborative Interaction Analysis, like reCAPit, provide interactive interfaces integrating video, audio, gaze, and notes, supporting experts to derive virtual artifacts that represent sequences, attention focus, and topic engagement relevant for SA.
Multimodal Emotional and Cognitive State Recognition uses sensor fusion and large language model integration to infer emotional/cognitive states from combined gaze and speech features, reflecting the anticipatory reasoning involved in SA level 3.
In summary, the study combines eye-tracking with NLP-based utterance analysis, fused in time-synchronized platforms, and augmented by advanced visualization like heatmaps and streamgraphs to capture and analyse the anticipatory cognitive state of emergency care professionals in situ. This multimodal, integrative approach provides insights into how healthcare professionals monitor, interpret, and project future status in complex clinical environments.
The study's findings could potentially improve our understanding of how leaders maintain SA during critical situations, contributing to the development of training programs for HCPs to improve their SA and team communication skills.
