As part of the Evaluating Healthcare Simulation tools, the Facilitator Competency Rubric (FCR) was developed by Leighton, Mudra, and Gilbert (2018) based on the Healthcare Simulation Standards of Best Practice and Patricia Benner’s (1984) Novice to Expert Theory. The goal of this instrument was to differentiate the varying levels of competency of the healthcare simulation facilitator. There are five constructs: preparation, prebriefing, facilitator, debriefing, and evaluation. The FCR is also available for download below in English and as well in German and Brazilian Portuguese Version.

FCR Downloads Include:

• Download the Facilitator Competency Rubric in English
  • Sample FCR. Practice using this with your team to identify areas in need of improvement and create an improvement plan, then prioritize it! Consider how you might use these results to provide coaching or conduct an evaluation session with a simulationist. Prior to using the FCR, you should use the information below to establish inter-rater reliability.
  • Preparation Construct*
  • IRR Prebriefing Construct*
  • Facilitator Construct*
  • Debriefing Construct: Please watch Simulation Instructor Course – Bad Debrief and Simulation Instructor Course – Good Debrief (Using D.E.B.R.I.E.F. Method)
  • Evaluation Construct
  • *Note: Click on sound icon for slides with this option. For others, respond to questions on slide. Please note that the slides are large files and there may be longer download times. For audio to work, download file to your computer and open in Slide Show mode.
• Facilitator Competency Rubric (German Translated Version).
• Facilitator Competency Rubric (Brazilian Portuguese Version).

Permission to Use FREELY: General use is already permitted by posting the statement: I understand that I have been granted permission by the creators of the requested evaluation instrument to use it for academic, clinical and/or research purposes. I agree that I will use the evaluation instrument only for its intended use, and will not alter it in any way. I will share findings as well as publication references with the instrument creator(s). I am allowed to place the evaluation instrument into electronic format for data collection. If official ‘Permission to Use’ letter is required, please contact the primary author. Include the purpose of the official request (research, grant), the intended use of the tool and with what population.

Why the Facilitator Competency Rubric was Developed: Members of the development team implemented a multi-site simulation education program for an undergraduate nursing program but realized they had no way to evaluate whether the program was effective. While the clinical simulation educators had the standard evaluation form that asks if the learning objectives were met and if the instructor was knowledgeable, they did not have a way to observe for behavior and practice change. The authors searched the literature and asked community experts, but a comprehensive facilitator evaluation tool did not exist. The team wanted to look at the overall abilities of a facilitator, rather than selected parts, such as debriefing. Therefore, the authors set off to create the Facilitator Competency Rubric (FCR).

How the Facilitator Competency Rubric was Developed: The INACSL Standards of Best Practice: Simulation now the Healthcare Simulation Standards of Best Practice were used as the foundation for the FCR, which was built upon Benner’s (1984) Novice-to-Expert theory, with the goal of creating a tool that could differentiate between varying levels of competency. The authors wanted to know who was competent, but also wanted to identify experts who we could enlist to mentor the identified novice facilitators.

The authors five major constructs were identified, and sub-components to define each construct were selected. The team presented the tool at six conferences over the course of one year to gain feedback as well as to socialize the concept of observational evaluation of facilitators. A workshop was also held for the purpose of developing the rubric concepts in alignment with Benner’s theory. The 4-member expert panel evaluated content validity and the content validity index (CVI) ranged from .75 to 1.0. Items with scores below .80 were revised for clarity.

Reliability and Validity of the FCR: The study was conducted at four sites, with a total of 7 raters and 18 facilitators volunteering participation in the study. Three sites were located in the US and one in the Middle East. Data analysis was completed with 81 observations. Test-retest reliability, determined by the interclass correlation coefficient, ranged from .43 to .92 across institutions (good to excellent at 3 of 4 sites). Inter-rater reliability ranged from .62 to .90 (good in 3 of 4 sites). G coefficients ranged from .80 to .99 (good to excellent) for the four institutions. Variance attributable to the FCR items appeared to be at acceptable levels not exceeding 35% at any one institution.

Using the FCR: The FCR is designed for evaluation of simulation facilitators. While tested in the educational environment, the concepts are broad; feedback has indicated that the tool is appropriate for the clinical and academic simulation learning environments.

Inter-rater Reliability: When using the FCR, educators must establish inter-rater reliability be prior to using the FCR to evaluate facilitators to ensure that observers are interpreting the tool in the same way. You may also find that an item or two is not applicable for your learning environment. For example, scheduling (Preparation construct) is not the responsibility of the facilitator at many institutions. In that case, the item should be excluded from your evaluation. Examples that can be used to help establish inter-rater reliability will be provided if you decide to consider use of the tool .

Evaluation: Gaining agreement from facilitators to be observed and evaluated may present difficulties. The FCR can also be used for self-evaluation by facilitators. The information gained from use of the FCR can be used to individually evaluate facilitators as well as to identify development needs and prioritize facilitator development activities.

Scoring: A scoring guide has been added to each section of the FCR to help you determine who is considered competent, who requires mentoring (beginner/advanced beginner score) and who is suitable to mentor and coach for improvement (proficient/expert scores). This work was contributed by a team at Chamberlain University, led by Vickie Mudra, in response to requests by users who wanted to quantify the results of each section. This method has not been tested, but it is provided as a way for you to identify the levels of competence amongst your faculty. We suggest that you focus on the lowest scoring construct areas first, and prioritize the lowest item scores in that construct area. Use these low scoring items as a needs assessment for creating a facilitator development plan. An example is provided if you decide to consider use of the tool.

FCR Downloads Include:

• Download the Facilitator Competency Rubric in English
  • Sample FCR. Practice using this with your team to identify areas in need of improvement and create an improvement plan, then prioritize it! Consider how you might use these results to provide coaching or conduct an evaluation session with a simulationist. Prior to using the FCR, you should use the information below to establish inter-rater reliability.
  • Preparation Construct*
  • IRR Prebriefing Construct*
  • Facilitator Construct*
  • Debriefing Construct: Please watch Simulation Instructor Course – Bad Debrief and Simulation Instructor Course – Good Debrief (Using D.E.B.R.I.E.F. Method)
  • Evaluation Construct
  • *Note: Click on sound icon for slides with this option. For others, respond to questions on slide. Please note that the slides are large files and there may be longer download times. For audio to work, download file to your computer and open in Slide Show mode.
• Facilitator Competency Rubric (German Translated Version).

Note Regarding FCR Inter-rater Reliability: Simulation educators should understand the vital importance that the inter-rater reliability be established prior to using the FCR to evaluate facilitators to ensure that observers are interpreting the tool in the same way. You may also find that an item or two is not applicable for your learning environment. For example, scheduling (Preparation construct) is not the responsibility of the facilitator at many institutions. In that case, the item should be excluded from your evaluation. Following are examples used during conference presentations in which audience members practiced using the FCR. It was evident that participants interpreted statements differently based on their previous experience and the expectations at their own organizations. This further evidenced the need to establish inter-rater reliability among those using the tool. It is also important that those being evaluated are aware of expectations for their success as facilitators. The INACSL Standards of Best Practice: SimulationSM are a valuable resource to guide facilitation of simulation-based experiences. Example Process:

1. Print copies of the FCR for each observer.
2. Observers should individually score each example.
3. Compare scores with other observers.
4. When scores are not the same, discuss rationale for why each person scored as they did.
5. Review example and rescore. If scores remain different, continue discussion until agreement attained.
6. For more than two observers or to establish inter-rater reliability for research purposes, consult the literature for various methods of statistically establishing inter-rater reliability.

References:

• For additional detail regarding the development and psychometric analysis of the FCR: Leighton, K., Mudra, V., & Gilbert, G. E. (2018). Development and psychometric evaluation of the Facilitator Competency Rubric. Nursing Education Perspectives, 39(6), E3-E9. doi: 10.1097/01.NEP.0000000000000409 https://tinyurl.com/ya2tepcv
• Webpage suggested citation: Leighton, K, Mudra, V., & Gilbert, G. E. (2018). Facilitator Competency Rubric. Retrieved from https://www.healthysimulation.com/tools/evaluating-healthcare-simulation/facilitator-competency-rubric/
• If you need an official ‘Permission to Use’ letter, or have other questions regarding this tool, please contact Dr. Kim Leighton.

German Translation

How the FCR (German Version) was translated by Theresa A. Forbrig (M.Sc.): The Facilitator Competency Rubric was translated into two German versions by a professional translation agency by two independent translators. These two versions were translated by Theresa A. Forbrig (M.Sc.) – a research assistant in the bachelor’s degree program in nursing at the Alice Salomon University of Applied Sciences Berlin (lecturer in nursing education, state-certified nurse) and a panel of experts from the Alice Salomon University of Applied Sciences. They checked for any linguistic differences, depending on the context, so that a consolidated German version of the FCR (German Version) was created. The expert panel consisted of four further experts: Prof. Dr. Johannes Gräske: Professor of Nursing Science, Maria Biniok (M.Sc.), Louise Koppe (M.Sc.), Johannes Haas (M.Sc.) – experienced teachers in nursing education and simulation teaching as well as state-certified nurses.

This merged German version was in turn translated back into English by a native English speaker, within a contract with a professional translation agency. The back-translated version was submitted to Dr. Kim Leighton for review to capture any differences in conceptual content.

Download the Facilitator Competency Rubric (German Version)

Reference:

• Webpage suggested citation: Forbrig, T. (2022). Translation of the Facilitator Competency Rubric to German. Retrieved from https://www.healthysimulation.com/tools/evaluating-healthcare-simulation/facilitator-competency-rubric/
• If you need an official ‘Permission to Use’ letter or have questions about this German version please contact Theresa Forbrig.

Brazilian Portuguese Version Translation

Comnpleted by team which includes: Daniel Gonçalves Campos Coordenador de Atividades Complementares da Faculdade de Medicina SLMandic Campinas Enfermeiro e Docente da Faculdade de Medicina SLMandic Campinas Mestre em Ciências da Saúde – Universidade Estadual de Campinas – Unicamp Instrutor em BLS pela AHA.

Objetivo: traducir y adaptar la Facilitator Competency Rubric a la lengua portuguesa y a la cultura brasileña, y analizar las propiedades de medición. Método: estudio metodológico que completó las etapas de traducción, síntesis de las traducciones, back translation, revisión por un Comité de Expertos compuesto por 7 profesionales, prueba de la versión pre-final con 33 facilitadores de simulación y presentación a la autora del instrumento original. Para la validación de contenido se calculó el Índice de Validez de Contenido y el Coeficiente Kappa modificado. Para determinar la confiabilidad, 52 y 15 facilitadores de simulación evaluaron el α de Cronbach y el Coeficiente de Correlación Intraclase, respectivamente. Resultados: se realizaron dos rondas de evaluación de contenidos, con modificaciones de 19 ítems en la primera evaluación y 3 ítems en la segunda. La escala general alcanzó α de Cronbach de 0,98 y Coeficientes de Correlación Intraclase de 0,95 a 0,97. Conclusión: la Facilitator Competency Rubric fue traducida y adaptada culturalmente a la realidad brasileña y presentó validez de contenido, confiabilidad y estabilidad, con resultados seguros para su uso en la enseñanza y la investigación.

Download the Facilitator Competency Rubric (Brazilian Portuguese):

—

Return to the Evaluating Healthcare Simulation tools webpage.