Simulationists and administrative leaders of simulation programs benefit from understanding how to demonstrate and report the return on investment (ROI) of their simulation activities. This is crucial for the growth and sustainability of their programs, as well as for securing new technology, virtual reality headsets, simulators, task trainers, and software subscriptions. This HealthySimulation.com article by Melissa Jo Tully, MHPE, RN, CPHN, will identify the common pitfalls in most simulation investment proposals, review critical indicators for a sound program evaluation, and outline the steps to conduct an ROI analysis.
Healthcare decisions affect the investments made in improving the education of future healthcare providers, preventing patient harm and injury, and enhancing the efficiency and quality of care. Financial and administrative decision-makers in healthcare organizations must make smart investments that reduce the risk to their organizations, especially when the healthcare system (and the educational system that feeds the workforce) faces pressures to lower healthcare costs and improve quality. Educational leaders face pressures to improve test scores and pass rates, meet the clinical requirements, increase throughput, decrease cost, improve instruction, and produce quality trained healthcare professionals which directly impacts the quality of patient care.
Simulationists and administrative leaders of simulation programs need to demonstrate and report the return on investment (ROI) of their simulation activities. This article aims to empower simulationists with information on how to present proposals for simulation programs and simulation technology in a way that shows their value and impact.
Return On Investment (ROI)
To make the case for investments in simulation as a tool to assess risk, test systems, and train clinicians is often a challenge for simulationists, who usually come from clinical or educational backgrounds with little experience in business and financial thinking. Several things often missed in the ROI argument are:
- Simulationists do not clearly articulate the ROI of their proposals when asking for simulation technology. Proposals should compare two or more approaches to training (e.g., computer-based training versus simulation-based training).
- Simulationists do not measure and report the ROI of their simulation programs alongside their program evaluations to their financial stakeholders.
- Simulationists have difficulty making the objective case for simulation technology due to intangible benefits. However, proposals should include and state the intangible benefits as part of the ROI.
View the LEARN CE/CME Platform Webinar How to Encourage Administration to Hire a Sim Tech to learn more!
The first step to start the simulation investment proposal with an ROI analysis is to complete a plan for a comprehensive program evaluation of the technology that will be used for professional development. Collecting any preliminary or pilot data on this matter, even if anecdotal, is key to the planning and measurement reporting to the decision-makers. The ability to capture the current state and gaps is critical here. While Kirkpatrick is often the model used for program evaluation, a closer look at Moore et al. (2009) article “Achieving Desired Results and Improved Outcomes: Integrating Planning and Assessment Throughout Learning Activities“.
Conceptual Model
Moore et al. (2009) captures the key aspects of program evaluation for education geared toward healthcare professional development. Most importantly, Moore et al. drives us to measure the improvement of healthcare quality and community health as the ultimate outcome of the use of simulation technology. This outcome is the goal of improving patient safety and quality care. The facets of Moore’s version of program evaluation:
- Based on the premise that the ultimate goal of continuing education in the health professions is to improve healthcare quality and community health.
- Proposes seven levels of outcomes for evaluating educational activities, ranging from participation to patient health.
- Emphasizes the importance of integrating planning and assessment throughout the learning activities, rather than conducting them as separate steps.
- Provides a framework for identifying the desired results and improved outcomes of educational activities, as well as the strategies and methods to achieve them.
- Applies to various types of educational activities, such as simulation, and can be used to guide the design, implementation, and evaluation of simulation programs.
An analogy to understand this concept is to think of Moore’s model as a roadmap for planning and assessing simulation programs. Moore’s framework for program evaluation is a comprehensive model that guides the planning and assessment of educational activities in the health professions. The framework proposes seven levels of outcomes for evaluating educational activities, ranging from participation to community health. These levels are:
- Level 1 – Participation: The number and characteristics of the learners who participate in the educational activity.
- Level 2 – Satisfaction: The degree to which the learners are satisfied with the educational activity and its delivery.
- Level 3 – Learning: The change in knowledge, skills, or attitudes of the learners as a result of the educational activity. This level is further divided into two sublevels:
- Level 3A – Declarative knowledge: The factual information that the learners acquire from the educational activity.
- Level 3B – Procedural knowledge: The ability to apply the factual information to perform tasks or solve problems.
- Level 4 – Competence: The ability of the learners to apply their knowledge, skills, or attitudes in simulated or controlled settings, such as laboratories or standardized patients.
- Level 5 – Performance: The ability of the learners to apply their knowledge, skills, or attitudes in their actual practice settings, such as clinics or hospitals.
- Level 6 – Patient health: The impact of the learners’ performance on the health outcomes of their patients, such as morbidity, mortality, or quality of life.
- Level 7 – Community health: The impact of the learners’ performance on the health outcomes of the communities they serve, such as public health indicators, health disparities, or health policies.
The framework emphasizes the importance of planning and assessment integration throughout the learning activities versus separation into individual steps. Planning and assessment should be aligned with the desired results and improved outcomes of the educational activities, as well as the strategies and methods to achieve them.
The determination of two or more approaches is essential to close the performance gap or solve the problem which includes the current approach with or without simulation or different simulation approaches. This method of comparison shows the stakeholders and financial decision-makers the current approaches and possible alternatives to explore the potential benefit and yield for low effective interventions versus effective interventions.
ROI Calculation with Tangible and Intangible Benefits
To calculate the return on investment (ROI) for healthcare simulation programs, it is important to conduct an ROI analysis for all the methods. The standard ROI is calculated by collecting the costs of training. The standard ROI formula is as follows: ROI = (net benefits/costs) X 100, where net benefits = benefits – costs.
Tangible benefits are those measured in costs and intangible benefits cannot be measured directly, but they do have a significant business impact. Tangible benefits include making processes and personnel more efficient and effective, as well as reducing the cost of care and services. Intangible benefits are those things that are difficult to measure directly, such as factors related to:
- Patients: personalized healthcare leads to improved outcomes and HCAHPS scores.
- Professionals: increased engagement and experiential learning opportunities.
- Families: enhanced understanding, trust, and confidence.
- Proceduralists: simulation rehearsal, preplanning, and preparation lead to greater confidence and understanding. Opportunities to improve processes and/or techniques through innovation. Improved communication among multi-disciplinary teams.
- Students/Residents/Clinical Staff: improved procedure, technique, processes, and disease education.
- Researchers/Innovators: provides the tools and biomedical engineering that allows the “bench to bedside” discovery.
- Educators: provides the instructional scaffoldings that lead to improved educational outcomes.
- Marketing: increases the college or hospital’s brand awareness and image.
- Financial Stakeholders: decreases medical errors, safety events, and malpractice lawsuits. Builds a case for malpractice insurance reduction, insurance reimbursement, and common standards.
- Public branding and relations: Simulation indirectly impacts a college or hospital’s reputation with the addition of another structural resource that leads to an improved process of care delivery, which leads to improved outcomes. Patient safety is also improved as it is related to the process of healthcare delivery.
Clinical Simulation Proposal Presentations
Robust simulation proposals involves the comparison of choices, reporting and intangible benefits. Here is an outline for a professional proposal for investment in simulation technologies to improve patient care that is clear, concise, and effective:
- Introduction: A brief overview of the proposal, including the purpose and goals of the project.
- Background: A description of the current state of healthcare and the challenges faced by healthcare organizations in improving patient care. This section should highlight the importance of investing in simulation technologies to address these challenges.
- Objectives: A clear statement of the objectives of the project, including the specific outcomes that the project aims to achieve.
- Methods: A detailed description of the methods that will be used to achieve the objectives of the project. This section should include a description of the simulation technologies that will be used, the training protocols that will be implemented, and the metrics that will be used to evaluate the effectiveness of the project.
- Timeline: A timeline for the project, including key milestones and deliverables.
- Budget: A detailed budget for the project, including the costs of simulation technologies, training, and personnel.
- Expected Outcomes: A description of the expected outcomes of the project, including the impact on patient care, healthcare costs, and healthcare outcomes.
- Conclusion: A summary of the proposal and its potential benefits, as well as a call to action for the audience to support the project.
Learn More About Return on Investments in Clinical Simulation Programs!
References
- Asche, C. V., Kim, M., Brown, A., Golden, A., Laack, T. A., Rosario, J., … & Okuda, Y. (2018). Communicating Value in Simulation: Cost–Benefit Analysis and Return on Investment.
- Eppich, W., & Reedy, G. (2022). Advancing healthcare simulation research: innovations in theory, methodology, and method. 2
- Twigg, D., & McCullough, K. (2014). Nurse retention: A review of strategies to create and enhance positive practice environments in clinical settings. International Journal of Nursing Studies, 51(1), 85-92
- Boggs, S., M.D., & Okuda, Y. (2014). Cutting costs while maintaining quality: How the VA has leveraged simulation. Physician Executive, 40(2), 38
- Moore Jr., D. E., Green, J. S., & Gallis, H. A. (2009). Achieving desired results and improved outcomes: Integrating planning and assessment throughout learning activities. Journal of Continuing Education in the Health Professions, 29(1), 1-155
