Mixed and Augmented Reality Can Facilitate Seamless Medical Communication

ar-healthcare-simulation

MedicalResearch.com recently interviewed Birmingham City University Associate Professor Dr. Ian Williams PhD about the work of the DMT lab on mixed and augmented reality for healthcare simulated training. Make no mistake, VR and AR are the future of healthcare simulated training:

Dr. Williams: Our work at the DMT Lab (dmtlab.bcu.ac.uk) focuses on developing a novel Mixed Reality (MR) medical presentation platform which allows practitioners to interact with patient data and virtual anatomical models in real time. The system enables the presentation of medical data, models and procedures to patients with the aim of educating them on pending procedures or the effects of lifestyle choices (for example the effects of smoking or excessive alcohol consumption).

The system employs an exocentric mixed reality environment which can be deployed in any room. It integrates a medical practitioner in real time with multimodal patient data and the corresponding result is a real time co-located visualisation of both the practitioner and the data, which they can interact with in real time.  We implement a natural interaction method into the system which improves a user’s level of direct interaction with the virtual models and provides a more realistic control of the data.

The system can also be used in a fun educational setting where patients, students, children or any naive user, can learn about medical anatomical information via a real-time interactive mixed reality “body scanner”. This fun system overlays the MR information onto their own body in real-time and shows them scaled and interactive virtual organs, anatomy and corresponding medical information. We are aiming for this system to be used not only in patient education but also in engaging and informing people on lifestyle choices.


Sponsored Advertisement:


MedicalResearch.com: What types of medical or surgical problems do you envision can be enhanced with the use of free hand gestures to manipulate patient data?

Dr. Williams: Mixed reality has enormous potential within the medical field, with healthcare being profoundly affected by some recent developments. Mixed reality technology can also provide the platform for facilitating a seamless doctor-patient communications in real time. The system we are developing can provide a real time augmented view of the patient’s data which can be overlaid onto the patient, or interacted with via freehand interaction without the use of complex wearable devices.

Many current mixed reality systems rely on bespoke sensors and cumbersome wearable devices (for example haptic gloves) whereas we work in freehand interaction without the need for a medical practitioner or patient to wear any complex wearable device. This interaction method enables a more natural virtual interface and via the use of naturally inspired physical interaction models (for example common real grasping types) we bridge the gap between users and technology. This form of natural interaction can also enable an interaction which can be perceived as more realistic to the observer.


Sponsored Advertisement:


Serious Games Conference Features Future of AR and VR in Healthcare – July 18-20 VA

ar in healthcare simulation

Is your program or institution looking specifically at VR and AR for healthcare applications? Check out this Serious Play Conference July 18-20 at the George Mason University!

Use of VR for Game-Based Learning Growing; Healthcare, Govt, Education Apps at Serious Play Thought leaders leading sessions at Serious Play Conference in July will share their experience using Virtual Reality to create education and training games. Speakers at the gathering, hosted by the Virginia Serious Game Institute (VSGI) at George Mason University’s Science and Technology Campus outside DC include many speakers and topics such as these presentations focusing on healthcare:

  • Mobile VR & AR Games for Healthcare by David Metcalf Institute for Simulation and Training, UCF
  • Clinical Tools VR for Complicated 3D Healthcare Structures Bradley Turner
  • How VR and AR Will Revolutionize Healthcare by Walter Greenleaf Stanford School of Medicine

Serious Play is a gathering where creators and learning professionals can have critical conversations about game design requirements and share their knowledge with peers. The focus of the conference is exploring opportunities, challenges and the potential of game-based learning. Their goal is to provide a forum for visionary educators, chief learning officers and heads of training programs in health care, government/military or other fields that want to learn how to improve the effectiveness of their program, and use the data collected to do even better.

Healthcare Track Sessions Include:

  1. Thomas Talbot USC Institute for Creative Technologies: Time to Leave the Lab, What Will it Take to Make Useful Games Viable for People and Businesses?
  2. David Metcalf Institute for Simulation and Training, UCF: Mobile Games Developed for Military Healthcare Training
  3. Scott Simpkins Applied Physics Laboratory Johns Hopkins: Using Games to Improve Clinical Practice and Healthcare Administration
  4. Alexander Libin Georgetown University: Predictive Analytics for an Embedded Assessment Framework: Developing Data-based Multimedia Technologies
  5. Students From Univ of Washington iSchool with Ran Hinrichs: Serious Play in Government Leadership Training
  6. Kevin Holloway Center for Deployment Psychology Uniformed Services Univ Of the Health Sciences: Virtual Professional Training in Evidence Based Psychotherapies, Gaming for Behavioral Health Providers
  7. Ran Hinrichs 2b3d Studios: Using Games to Study the Psychological Impact of Military Deployment
  8. Walter Greenleaf Virtual Human Interaction Lab Stanford University School of Medicine: How Virtual and Augmented Reality Technology will Revolutionize Healthcare
  9. Brad Tanner: Clinical Tools 3D Virtual Reality Using Oculus to Teach Complicated 3D Structures in Healthcare
  10. Doris Rusch DePaul University: Integrating Academia, Healthcare Professionals and Patients to Create a Learning Game for Chronically Ill Patient Diseases
  11. Dmitriy Babichenko Lorin Grieve, Jonathan Velez University of Pittsburgh: To Scope or Not To Scope: Challenges of Gamifying Clinical Procedures Training
  12. Kimberly Hieftje Yale Center for Health & Learning Games: Re-purposing Serious Games: Making the Development Time Count Twice (or More)
  13. Kenneth Bibbins PrepWorld Philliph Mutisya NC Central University: Trauma Informed Game Based Learning for Kids
  14. Beth Rogozinski Pear Therapeutics: The Challenges of Creating Mobile Games for Regulated Health Situations
  15. Lisa Marriott OHSU/PSU School of Public Health: Working with Local Schools on Nutrition Education
  16. David Wortley GAETSS, UK: Trends in Serious Games for Health and Well-Being

Learn more and Register on the Serious Games Website today!

 

CAE Healthcare Leads Innovation at IMSH with New Games, Eyes, and AR Technologies | IMSH 2017 Video Interview

augmented reality ultrasound trainer

Furthering HealthySim’s coverage of the IMSH 2017 trade show floor, today we are sharing our exclusive interview with CAE Healthcare to learn more about their latest innovative healthcare simulation technologies including simulated eyes and augmented reality ultrasound trainer! Today we are seeing three new innovative products from CAE Healthcare: simulation-based anesthesiology training in a gaming-style online learning environment, the future of healthcare patient simulator eyes, and the first holographic computer that allows you to freely move and interact with your healthcare simulation learning environment. Watch our video interview below to see it all!

Screen-Based Anesthesiology Simulation Product Highlights

The American Society of Anesthesiologists (ASA) and CAE Healthcare have agreed to co-support the design and development of an interactive/gaming-style training product which will help learners enhance their medical skills. The screen-based simulation training product is intended to assist with improving performance in the management of anesthesia emergencies. The ASA screen-based simulation product will allow flexible, asynchronous training in a self-paced, just-in-time environment. This means users may engage with simulated, online interfaces in a totally immersive, 3D setting and progress at their own pace. Imagine the benefits to training when a student, or new learner, can “enter” an operating room in virtual reality, treat a virtual patient with varied symptoms and conditions, and have that patient respond as if it were all real. There is no risk of harm to a living person. The learner can stop and start the self-paced instruction at will, and develop progressive skills mastery according to their own ability.

The possibilities for improved learning and competency acquisition are endless. Other features include:

  • learning can take place in the comfort of the user’s own home, via their own computer
  • as learners enhance their clinical and decision-making skills, they can also see how they score in comparison to their peers
  • the product will include five (5) medical scenarios with realistic medical instruments and equipment for medical diagnostics and monitoring

See the Future of Simulation with SymEyes

The quality of the digital screen eyes was quite remarkable! Standard with CAE Healthcare’s Lucina High-Fidelity Maternal/Fetal Training simulator, you can now display even more realistic patient symptoms and conditions, including jaundice, hemorrhage, keyhole pupil, cataracts and bloodshot or droopy eyes. (Note: The flicker of the eyes was only due to the frame rate of our camera.) Our team was very impressed by the innovative thinking with these new eyes and believes they are the new standard for patient pupil display!

Augmented Reality Ultrasound Training

The expanded learning opportunities in healthcare with augmented reality are becoming increasingly clear. CAE Healthcare has reset the bar for augmented reality technology in healthcare simulation by combining Microsoft’s Hololens with their leading Vimedix Ultrasound trainer, designed to make learning more engaging and intuitive. The manikin-based system allows for the development of the psychomotor skills needed to handle ultrasound probes. The innovative software tools of Vimedix accelerate the development of the cognitive skills needed to interpret ultrasound images, make diagnoses and clinical decisions. With the Hololens addition, you can now gather your learners for a shared, immersive experience that will inform and delight. Our HoloLens-based solutions will inspire deeper understanding from the start, and awaken their imaginations to better treatments and tools to improve patient care.

Freed from its two-dimensional environment inside a monitor, our Vimedix ultrasound simulator leaps to life, displaying anatomy you can enlarge, turn, rotate or command to return into its manikin body so you can view the interrelatedness of all of its structures. Witness in real time how the ultrasound beam cuts through the human anatomy.

See all of these simulation technologies and more at the next CAE Healthcare HPSN event!


Sponsored Advertisement:


You Won’t Believe the Future Tech of Healthcare Education

new tech in healthcare

Chris Merritt, recently wrote an article for McKnights News on the evolution of healthcare education through modern day technologies — which should be forwarded along to simulation discomfiters, or naysayers, in your simulation program! Chris theorizes that online gaming is the future of healthcare education!

The rapid advancements in technology continuously impact our lives on a daily basis and each new week brings a critical update to our attention. This has significantly changed the ways in which we receive and process information such as current events, the daily news, industry updates, association content, medical journals and even our educational materials. You do not need to look any further than a grade-school classroom in which personal tablets have replaced pencil and paper for our youngest generation. This transformation has occurred not because it is the cheapest alternative, in fact this migration can often times be more expensive on the front-end.

We are changing our educational delivery mechanisms due to improved learner experience, retention and overall knowledge outcomes. A 2008 study by the Ewing Marion Kauffman Foundation reported that a great lecture can improve learning outcomes by 17%, while switching to a different delivery mechanism such as serious gaming can improve learning outcomes by 108%.

Many have acknowledged the explicit need for our healthcare professionals to have a safe environment in which to practice, make mistakes and increase their proficiency in the many critical decisions they make on a daily basis without putting a patient at risk. A 2015 study in The Journal of Clinical Nursing reported in 2015 that, “Finding a new platform to allow all nurses to practice difficult clinical decisions is key. A virtual immersive environment…can provide simulation for nurses to practice making such difficult decisions.”

The evidence is present that these online simulations increase learner engagement and retention while also resulting in improved patient outcomes and a positive impact on healthcare economics. One chronic disease specific online simulation called SiMCare Diabetes has published data which reports: improved glycemic control in patients with A1C >7%, a 60% reduction in the prescription of contraindicated medication and reduced cost by $71 per patient versus those professionals that did not train with the simulation.

Advances in medical education, clinical content, guidelines and standards of care now have a new and improved medium for dissemination. Online training simulations and educational games are continually updated in real-time and given the nature of the platform, content can be rapidly deployed around the globe with the click of a button.

Esper Augmented Reality Trainer from 3D4Medical Will Change the Way We Learn Anatomy

augmented reality virtual learning anatomy

3D4Medical Labs makes award-winning medical and fitness software. Their applications are used daily by education and clinical organizations around the globe. They have over 12 million downloads, numerous prestigious awards, and appearances on stage at major industry events. Their anatomical models are the most detailed available on consumer devices, which they build by studying real anatomical structures, and combining them with world-class medical knowledge.

Straight from the 3D4Medical Lab, discover how you will interact with the anatomy using augmented and mixed reality with their new Project Esper. Take a look at the future of medical learning with the video above about this world-class education tool that utterly transforms the way people learn about the human body.

Learn more on the 3D4Medical Labs website!


Supported Organization:


Microsoft Improves Healthcare Education with Launch of HoloLens Augmented Reality Glasses

hololens

Have you heard about the release of the Hololens from Microsoft yet? This oculus-like device will enable healthcare educators with a plethora of new tools to educate learners with the latest in virtual and augmented reality programs. Navigate anatomy, workspaces, and educational programs in 3d spaces. Track motion and spatial mapping to better learn how learners interact with learning programs. Watch this Microsoft demonstration by School of Medicine Dean Pamela Davis who shows how using holograms to teach anatomy dramatically enhances and accelerates learning:

About Microsoft HoloLens:

Microsoft HoloLens is the first fully untethered holographic computer running Windows 10. It is completely untethered–no wires, phones, or connection to a PC needed. Microsoft HoloLens allows you to place holograms in your physical environment and provides a new way to see your world.

Microsoft HoloLens generates a multi-dimensional image visible to a user so that he or she perceives holographic objects in the physical world. Holographic objects seen with Microsoft HoloLens can be placed in physical locations you choose, move according to their own rules, or remain in a specific location regardless of where you are or in which direction you are looking.

The holograms you’ll see with Microsoft HoloLens can appear life-like, and can move, be shaped, and change according to interaction with you or the physical environment in which they are visible. Use gestures to create, shape, and size holograms. Use your gaze to navigate and explore. Use your voice to communicate with your apps. Microsoft HoloLens understands your movements, gaze, and voice, enabling you to interact with content and information naturally. Using holograms, you can place your digital content, such as apps, information, and even multi-dimensional videos, in the physical space around you, so you can interact with it.

Learn more on the Microsoft HoloLens website!

Medicine Meets Virtual Reality (MMVR22) Launches in Los Angeles

mmvr 22

Today at the Sheraton Hotel in downtown Los Angeles the 22nd annual Medicine Meets Virtual Reality meeting launched to an international audience. HealthySim is on hand to provide updates throughout the day on our @HealthySim twitter account – and stay tuned tomorrow for a blog post covering more activities from the event. For now, learn more about MMVR below:

About MMVR

In 1992, Medicine Meets Virtual Reality first presented a daring vision of patient care and medical education transformed by computer technology. The evolving NextMed / MMVR engages researchers committed to intelligent healthcare—engineers, physicians, scientists, educators, students, industry, military, and futurists—with its creative mix of unorthodox thinking and validated investigation. Conference topics include:

    • Medical simulation and modeling
    • Data visualization and fusion
    • Virtual and augmented reality
    • Imaging devices and methods
    • Robotics, haptics, sensors
    • Human-computer interfaces
    • Data and decision networks, AI, mobile health
    • Wearable and implantable electronics
    • Projection systems
    • Learning and technology
    • Simulator design and validation
    • Physical and mental rehabilitation tools
    • Serious games
    • Surgical registration and navigation
    • Peri-operative guidance
    • Remote and battlefield care
    • Patient and public health monitoring and education

NextMed / MMVR promotes the creation and adoption of IT-enabled tools for patient care and medical education that support better precision, efficiency, and outcomes. The curriculum combines traditional assessment methods with unorthodox problem-solving to stimulate forward-thinking solutions to healthcare problems. Presentations are chosen to educate participants on:

    • Advances in simulation, modeling, and haptics that are upgrading medical education, skills  training, psychotherapy, and physical rehabilitation
    • Novel imaging, visualization, and data fusion methods that make clinical diagnosis and therapy more precise and personalized
    • Robotics and sensors that extend the caregiver’s reach and provide richer patient data
    • Medical intelligence networks that promote a collaborative healthcare environment and enhance decision-making
    • Broader goals, accomplishments, and challenges in the development and application of emerging healthcare technologies

Learn more at the Medicine Meets Virtual Reality Website!

Case Western Reserve, Cleveland Clinic Collaborate with Microsoft for Mixed-Reality Technology for Education

microsoft medical simulation

Shared from Case Western Reserve University:

Case Western Reserve University Radiology Professor Mark Griswold recently shared how “HoloLens” can transform learning across countless subjects, including those as complex as the human body. Speaking to an in-person and online audience at Microsoft’s annual Build conference, he highlighted disciplines as disparate as art history and engineering—but started with a holographic heart. In traditional anatomy, after all, students like Ghodasara cut into cadavers to understand the body’s intricacies. With HoloLens, Griswold explained, “you see it truly in 3D. You can take parts in and out. You can turn it around. You can see the blood pumping—the entire system.”

In other words, technology not only can match existing educational methods—it can actually improve upon them. Which, in many ways, is why Cleveland Clinic CEO Toby Cosgrove contacted then-Microsoft executive Craig Mundie in 2013, after the hospital and university first agreed to partner on a new education building. “We launched this collaboration to prepare students for a health care future that is still being imagined,” Cleveland Clinic CEO Delos “Toby” Cosgrove said of what has become a 485,000-square-foot Health Education Campus project. “By combining a state-of-the-art structure, pioneering technology, and cutting-edge teaching techniques, we will provide them the innovative education required to lead in this new era.”

Because the technology is relatively easy to use, students will be able to build, operate and analyze all manner of devices and systems. “[It will] encourage experimentation,” Buchner said, “leading to deeper understanding and improved product design.”

In truth, HoloLens ultimately could have applications for dozens of Case Western Reserve’s academic programs. NASA’s Jet Propulsion Laboratory already has worked with Microsoft to develop software that will allow Earth-based scientists to work on Mars with a specially designed rover vehicle. A similar collaboration could enable students here to take part in archeological digs around the world. Or astronomy students could stand in the midst of colliding galaxies, securing a front-row view of the unfolding chaos. Art history professors could present masterpieces in their original settings—a centuries-old castle, or even the Sistine Chapel.

“The whole campus has the potential to use this,” Griswold said. “Our ability to use this for education is almost limitless.”

Read the full Hololens article here!

 

SimX Augmented VR Glasses Win IMSH Serious Games Showcase | Video Interview

augment reality

At the IMSH 2015 HealthySim had a chance to interview Ryan Ribeira, CEO of SimX, and practicing resident physician in emergency medicine at Stanford. Ryan was showcasing SimX at the 5th Annual Serious Games and Virtual Environments Showcase and Arcade which took place at IMSH 2015 in New Orleans, LA and provided over 300 attendees demonstrations of more than 21 entries. The event, started by Dr. Eric B. Bauman, continues to attract exceptional talent from around the world and did a fantastic job of highlighting some serious technology advances in our field such as SimX – which walked away as the WINNER of the Small Business or Corporation category.  (Click here to visit TheClinicalPlayground.com for a full showcase awards recap).

More about SimX:

Virtual Patients: SimX’s software replaces your physical simulation mannequins with a customizable, high-definition, 3D virtual patient, that can be projected onto any empty hospital bed. Whether obese, pregnant, young, old, vomiting, missing limbs, bleeding, or expressing any number of other physical signs and symptoms, SimX’s software allows you to reproduce patient presentations with unprecedented visual fidelity.

Case Builder: Build complex cases in minutes using the powerful visual case building system. Drag & drop events onto the field, determine the environment, and set patient data with just a few clicks. Use SimX’s powerful case monitoring and feedback system to see the case from each trainee’s perspective, and adjust case parameters on the fly.

Global Case Marketplace: SimX allows you to access thousands of cases from top hospitals across the globe! Let your trainees learn from specialists at the cutting edge of their field. Tap into the expertise of your own simulation specialists. Market your cases to the world and turn your expertise into revenue.

Direct Integration: SimX is built so that your trainees can learn using the tools you already have right at your institution. You can use your own beds, monitors, ultrasounds, stethoscopes, even your existing simulation manniquins! It’s easy to ensure that SimX software will recognize your tools, and allow your trainees to use them in the simlation.

Cloud Based Software: SimX’s marketplace, case authoring tools, and moderator interface are all located on the cloud, so you don’t have to worry about downloads or compatibility issues. Write and run cases from your desktop, laptop, or even your mobile device! If it has a browser, it works with SimX.

Reporting:  SimX’s case authoring and moderating tools come with powerful reporting features built right in. Every order, request, and event is recorded on the case timeline, so your team can debrief and see how cases might have gone if their decisions had been different. Create powerful reports that can help you pinpoint where trainees need the most help, and can track the improvement of teams or individuals over time.

Learn more at SimXAR.com!