Patient Safety in Operating Rooms
Overview & Problem
Operating rooms are one of the most complex clinical environments, and errors during surgeries can negatively impact patient outcomes, such as longer hospital stay and fatality rate. Existing analysis of surgical performance focuses heavily on adverse events reported after the surgery, typically through forms. This approach emphasizes on events that resulted negative outcomes. However, most of surgeries are successful because the surgical team are able to respond to a wide variety of situations. The existing approach do not provide insights into the resilience of the surgical teams.
Approach
This project sought to understand resilient behaviours that contribute to the success of surgeries by conducting data analysis on surgical recordings after operations. The analysis focused on uncertain situations where the surgical team may be unclear about the current situation, the best choices, and future outcomes of the surgery.
The goal of the project is two-fold. First, we would like to gain a deeper understanding on both the causes of uncertainties and the resilience factors (e.g., technological, environmental, technical skills, and non-technical skills) that allowed clinicians to appropriately and quickly respond. Furthermore, the manual review of the surgical videos is extremely time consuming which delays the feedback sent to the surgical team. Thus, the second aim of the project was to improve the analysis efficiency of recordings by applying machine learning algorithms to identify the transition from routine work to uncertain situations.
A specialized recorder, OR Black Box, was installed in one of the operating rooms. It collects video, audio, vital signs, and laparoscopic view of the patient during laparoscopic surgeries.
The goal of the project is two-fold. First, we would like to gain a deeper understanding on both the causes of uncertainties and the resilience factors (e.g., technological, environmental, technical skills, and non-technical skills) that allowed clinicians to appropriately and quickly respond. Furthermore, the manual review of the surgical videos is extremely time consuming which delays the feedback sent to the surgical team. Thus, the second aim of the project was to improve the analysis efficiency of recordings by applying machine learning algorithms to identify the transition from routine work to uncertain situations.
A specialized recorder, OR Black Box, was installed in one of the operating rooms. It collects video, audio, vital signs, and laparoscopic view of the patient during laparoscopic surgeries.
Method
Protocol Design and Recruitment Strategy
There are several challenges in the design of the research protocol and participant recruitment. For example, the research design needs to fit into the clinician’s workflow and medical data flow. I worked with all the clinical departments involved to understand their priorities and concerns, e.g., surgeons, nurses, anesthetists, patient registration, and operating room. I also studied how the medical data and related information (e.g., appointment dates) flow across departments. I presented the research design to the clinicians and revised based on their feedback to ensure a smooth workflow.
Another major concern of the project is participant anonymity. For example, only participating clinicians should be assigned to the room with the OR Black Box, but we need to keep participation anonymous to avoid coercing them into participation. I designed the mechanism for sharing participant data so that only the OR coordinator who is in charge of scheduling has that information. The figure below shows one example of visuals presented to the clinicians to explain the workflow regarding tracking clinician participant consent.
There are several challenges in the design of the research protocol and participant recruitment. For example, the research design needs to fit into the clinician’s workflow and medical data flow. I worked with all the clinical departments involved to understand their priorities and concerns, e.g., surgeons, nurses, anesthetists, patient registration, and operating room. I also studied how the medical data and related information (e.g., appointment dates) flow across departments. I presented the research design to the clinicians and revised based on their feedback to ensure a smooth workflow.
Another major concern of the project is participant anonymity. For example, only participating clinicians should be assigned to the room with the OR Black Box, but we need to keep participation anonymous to avoid coercing them into participation. I designed the mechanism for sharing participant data so that only the OR coordinator who is in charge of scheduling has that information. The figure below shows one example of visuals presented to the clinicians to explain the workflow regarding tracking clinician participant consent.
Part of the workflow for tracking participant consent to ensure anonymity.
Study Design
We conducted a modified field study to maximize the realism of data collected. First, I conducted three initial field studies to understand the operating room environment and surgical workflow. Once the study officially kicked off, the surgeries were recorded by the surgical team and I conducted observation and data analysis on the recordings. All laparoscopic surgeries took place in the operating room with the OR Black Box installed was recorded.
We conducted a modified field study to maximize the realism of data collected. First, I conducted three initial field studies to understand the operating room environment and surgical workflow. Once the study officially kicked off, the surgeries were recorded by the surgical team and I conducted observation and data analysis on the recordings. All laparoscopic surgeries took place in the operating room with the OR Black Box installed was recorded.
Data Analysis
I led a junior researcher to conduct the video coding analysis to understand resilience in surgeries during uncertain situations. The figure below illustrates the analysis process.
Overview of the data analysis process
We started with a literature review to come up with an initial coding analysis framework (a set of codes to analyze the videos). For example, we looked for instances where the surgical team members demonstrated leadership and communication skills. We also looked for behaviours that may indicate uncertainty such as saying “I’m not sure”, asking questions, and discovering atypical anatomy.
By collaborating with a surgery analyst, we coded the videos iteratively and refined the coding analysis framework continually. The iterations continued until the video analysis framework was stable and no new codes were added. The videos previously coded with older versions of the framework were then recoded.
By collaborating with a surgery analyst, we coded the videos iteratively and refined the coding analysis framework continually. The iterations continued until the video analysis framework was stable and no new codes were added. The videos previously coded with older versions of the framework were then recoded.
Results
Based on the data analysis, I produced two deliverables: 1) an analysis framework for uncertain situations, and 2) coded data for developing a machine learning algorithm to automatically flag potential uncertain situations during surgeries.
The analysis framework contains two primary sets of elements: uncertainty trigger and cue. The uncertainty triggers are the causes of an uncertain situation, such as atypical anatomy, nurse shift change during surgeries, miscommunication, and technical experiences. The cues are machine identifiable instances that point towards a potential for uncertainty, such as excessive bleeding, long pauses during surgery, and questions asked by team members.
The analysis framework contains two primary sets of elements: uncertainty trigger and cue. The uncertainty triggers are the causes of an uncertain situation, such as atypical anatomy, nurse shift change during surgeries, miscommunication, and technical experiences. The cues are machine identifiable instances that point towards a potential for uncertainty, such as excessive bleeding, long pauses during surgery, and questions asked by team members.
Sample codes in the analysis framework
Impacts
My work at North York General Hospital has created many impacts in improving patient safety in the operating rooms:
- The project is ongoing at North York General Hospital to provide feedback to clinicians and improve surgery outcomes.
- The analysis framework is used to conduct further manual analysis of surgeries and to provide feedback to the surgical team.
- The analysis framework and coded data guide the development of a machine learning algorithm for automatically flagging events that need manual analysis
My Roles
- Launched the research project by securing funding for the project, collaborating with a cross-functional team, and designing the research protocol, including recruitment strategy, consent process, information flow across multiple departments.
- Increased buy-in, participation, and project visibility through verbal and written communications with clinicians, patients, and admin staff
- Conducted observations and surgical video analysis to understand supporting and hindering factors (e.g. technological and environmental) related to surgery performance
- Designed a data analysis framework for uncertain situations during laparoscopic surgeries by collaborating with domain experts and conducting iterative video analysis
- Coached junior researchers in grant writing, literature review, and video analysis
News
Collaborators
- Bijan Dastgheib – St. Michael’s Hospital
- Mark Fan – North York General Hospital
- Alistair Forsyth – North York General Hospital
- Teodor Grantcharov – St. Michael’s Hospital, Surgical Safety Technologies
- Usmaan Hameed – North York General Hospital
- Darryl Irwin – North York General Hospital
- Donna McRitchie – North York General Hospital
- Sonia Pinkney – University of Toronto
- Adrian Ragobar – North York General Hospital
- Michell Richardson – North York General Hospital
- Lloyd Smith – North York General Hospital
- Patricia Trbovich – University of Toronto, North York General Hospital
- Julie Zen – North York General Hospital
Acknowledgement
This project was funded through Ontario Centres of Excellence VIP I + NSERC Engage and Steinberg Fund at the North York General Hospital with support from the Surgical Safety Technologies. This work was completed as part of my postdoctoral fellowship funded through the Health System Impact Fellowship sponsored by Canadian Institutes of Health Research.