DEVELOPMENT OF COMPETENCY ASSESSMENT TOOLS FOR NEONATAL BAG MASK VENTILATION AND INTUBATION: A DELPHI STUDY

Abstract BACKGROUND As paediatric training programs shift to a competency-based education model, there is a growing need for tools with strong evidence of validity to teach and assess procedural skills. To date, there are no competency-based assessment tools for bag mask ventilation or neonatal intubation that are widely accepted in the field of paediatrics. OBJECTIVES We aimed to develop a neonatal bag mask ventilation competency assessment tool (BMVCAT) and neonatal intubation competency assessment tool (NICAT) to assess proficiency in these skills for application in both the clinical and simulation-based training environments. Delphi methodology was used to determine expert consensus regarding critical items to be included. DESIGN/METHODS Systematic literature reviews were performed to generate potential items to include in the assessment tools, consisting of two parts: a checklist of specific actions required to complete the procedures competently and global ratings reflecting overall competence on general aspects of the skill. Checklist items were grouped into 3 domains: pre-procedure, intra-procedure, and post-procedure. A Delphi panel of North American neonatal experts was established to determine expert consensus regarding critical items required to objectively assess the competence of individuals performing neonatal bag mask ventilation and intubation. Panelists completed iterative surveys to rate the importance of checklist and global rating items using a 7-point Likert scale. Responses were evaluated and items were removed after each round if the mean rating was <5.5 until consensus was achieved. RESULTS Thirty-four experts from 26 centres in Canada (N=23) and the United States (N=11) participated in the Delphi process: 18 neonatologists, 9 neonatal nurses or nurse practitioners, 4 respiratory therapists, 2 paediatricians, and 1 paediatric anesthesiologist. Systematic literature reviews generated 48 checklist items and 23 global rating items for the BMVCAT and 67 checklist items and 24 global rating items for the NICAT. The first Delphi round reduced the BMVCAT to 43 checklist items and 20 global rating items and the NICAT to 63 checklist items and 23 global rating items. The second Delphi round reduced the BMVCAT to 27 checklist items and 16 global rating items and the NICAT to 50 checklist items and 22 global rating items. The Delphi process was continued until expert consensus was achieved to generate the BMVCAT and NICAT. CONCLUSION Delphi methodology allowed for the determination of consensus regarding essential items to be included in tools designed to measure competence in performing neonatal bag mask ventilation and intubation. Further studies are planned to prospectively validate the BMVCAT and NICAT in clinical and simulated settings.