Neurosurgery simulation just saw a revolution in realism and application, with a real-time brain simulator which can help neurosurgeons train for and perform better during clinical practice. Out of the Igarashi Laboratory in the Graduate School of Information Science and Technology at The University of Tokyo, Dr. Seo and team have developed a cutting edge application for the very complicated task of looking at brain anatomy during an actual surgical intervention. Utilizing the Unreal Engine, this project is supported by Epic Games who we also shared about recently with their support of Precision OS: Orthopedic Surgery Training in VR. Let’s take a deeper look at Sébastien Lozé’s recent article on this innovative brain surgery simulator!
Brain surgery remains one of the most intricate and challenging tasks a surgeon might have to perform, and requires extensive training and practice. While the medical field has long used CGI to visualize the human body inside and out, many applications are limited to opaque shaders on rigid objects, which don’t accurately depict the many translucent, thin, or flexible parts of the brain’s structure.
In their 2018 paper Enhancement Techniques for Human Anatomy Visualization, Hirofumi Seo and Takeo Igarashi state that “Human anatomy is so complex that just visualizing it in traditional ways is insufficient for easy understanding…” To address this problem, Seo has proposed a practical approach to brain surgery using real-time rendering with Unreal Engine.
As part of the User Interface Research Group, Igarashi Laboratory, Graduate School of Information Science and Technology at The University of Tokyo, Seo and his team are working on a real-time visualization and training application for brain surgery that more accurately portrays the brain’s structure and how it deforms during surgery. The software prototype, made possible with a grant (Grant Number JP18he1602001) from Japan Agency for Medical Research and Development (AMED), helps surgeons visualize a patient’s unique brain structure before, during, and after an operation.
“Neurosurgeons all over the world performing aneurysm surgery want some kind of pre-surgical simulation, practice, or check, because the actual surgical view is very limited and the surgery itself is very difficult,” says Seo. “They also know that the brief blood-vessel-branch dominant region of each blood vessel is easily predictable if they can see the whole brain and the blood vessels. So many neurosurgeons have wanted to use 3D CG for a long time, but they don’t know how to implement it.”
With a real-time 3D visualization, the surgeon can load a model of a patient’s brain from the individual’s MRI and 3D Rotational Angiography (3DRA) data, look at it from any angle, pull apart the CG sulcus to see inside, and even make the lobes invisible to better see the blood vessels. The user controls everything via simple mouse cursor movements or via multi-touch, making the app easy and accessible for surgeons without technical experience.
In developing the application, Seo’s team chose Unreal Engine as the underlying real-time technology because of its graphics and programming tools. “Unreal Engine has powerful mathematical C++ APIs such as FVector, FMath, and UKismetMathLibrary, so we find it to be a suitable platform for research on 3D CG geometry,” says Seo.
