Observational teamwork assessment for surgery: feasibility of clinical and nonclinical assessor calibration with short-term training.

Journal article

OBJECTIVES: To assess the feasibility of training clinical and nonclinical novice assessors to rate teamwork behavior in the operating room with short-term structured training using the observational teamwork assessment for surgery (OTAS) tool. BACKGROUND: Effective teamwork is fundamental to the delivery of optimal patient care in the operating room (OR). OTAS provides a comprehensive and robust measure of teamwork in surgery. To date, assessors with a background in psychology/human factors have been shown to be able to use OTAS reliably after training. However, the feasibility of observer training over a short timescale and accessibility to the wider clinical community (ie, OTAS use by clinicians) are yet to be empirically demonstrated. METHODS: Ten general surgery cases were observed and assessed using OTAS in real-time by an expert in rating OTAS behaviors (100+ cases rated) and 4 novices: 2 psychologists and 2 surgeons. Assessors were blinded to each other's scores during observations. After each observation, scores were compared and discussed between expert and novice assessors in a debriefing session. RESULTS: All novices were reliable with the expert to a acceptable degree at rating all OTAS behaviors by the end of training (intraclass correlation coefficients ≥0.68). For 3 of the 5 behaviors (communication, cooperation, and leadership), calibration improved most rapidly across the first 7 observed cases. For monitoring/situational awareness, calibration improved steadily across the 10 observed cases. For coordination, no significant improvement in calibration over time was observed because of high interrater reliability from the outset (ie, a ceiling effect). There was no significant difference between surgeons and psychologists in their calibration with the expert. CONCLUSIONS: It is feasible to train both clinicians and nonclinicians to use OTAS to assess teamwork behaviors in ORs over a short structured training period. OTAS is an accessible tool that can be used robustly (ie, reliably) by assessors from both clinical and nonclinical backgrounds.

Authors: Russ, S; Hull, L; Rout, S; Vincent, C; Darzi, A

• Journal: Annals of surgery
• Volume: 255
• Issue: 4
• Pages: 804-809
• Publication date: 2012-04-01
• DOI: 10.1097/sla.0b013e31824a9a02
• EISSN: 1528-1140
• ISSN: 0003-4932
• Language: English
• Keywords: Single-Blind Method; Clinical Competence; Teaching; General Surgery; Observer Variation; Prospective Studies; Interprofessional Relations; Feasibility Studies; Humans; Process Assessment (Health Care); Cooperative Behavior; Patient Care Team; Learning Curve; Calibration