Editor’s Blog – VR in the ER 

Virtual and augmented reality headsets are helping medical students prepare for routine operations and harrowing emergency admissions without a cadaver in sight, says Caroline Hayes

Medical students are used to donning surgical mask, scrubs and gloves for  observing resident surgeons perform real-life operations but now they are donning VR headsets to take part in a practical element to their medical training.

Market analyst firm, Allied Market Research, estimated that the VR for medical training market was estimated to be worth $204 billion in 2019 and would experience a compound annual rate (CAGR) of 13.7 per cent to reach $601 billion in 2027. 

Surgery without the patient

It is used to simulate operating theatres and routine operations. Realistic and medically accurate VR models can help students make minimal incisions and navigate typical parts of the human anatomy to perform operations. This helps them to practise surgical techniques and discover typical stumbling blocks, such as navigating blood vessels and perform keyhole or minimally invasive surgery. There are many advantages to pursing less invasive surgery techniques, not least quicker recovery time for the patient but also shorter hospital stays and lower costs to the healthcare provider. Educationally, the benefits are that the student can become familiar with handling, manipulating and using professional medical equipment so that the first they use it as a newly qualified medical professional, they can begin immediately.

Practise makes perfect

For non-surgical medical students, VR is also used to simulate major emergencies and accidents where there are multiple cases rushed into hospital. This allows groups to practise emergency response systems and how to assess these distressing scenarios. Some software modelling, for example, may include small children and their anxious parents alongside adults with injuries of varying seriousness which have to be prioritised and dealt with calmly and efficiently. Thankfully, it is rare for a medical student to be involved in such an incident in real life but they need to be prepared for the worst case scenarios. Even if they were to be the hospital when a full scale emergency was underway, training on the spot is not practical when everyone is in a heightened state of alert.

Oculus, a division of Meta (formerly Facebook) partnered with the Children’s Hospital Los Angeles (CHLA) to build a VR simulation for paediatric trauma cases. The student uses VR to place themselves in the training scenario complete with audio of paramedics reeling off the child’s symptoms and other staff waiting to hear the decision which has to be issued in earshot of one or more of the child’s parents.

CHLA students learn how to deal with stressful emergency cases using Oculus headsets. (Picture: Oculus, Meta Quest)

The VR, powered by VFX production company, AiSolve is based on actual case studies. Doctors report that VR has reduced the time and cost of mannequin based training session but also better prepares students for real-world cases.

Augmented reality 

Augmented reality also has a role in medical training. Brainlab, a digital medical technology company based in Munich, combines patient-specific, AI-driven anatomical segmentation visualisation software with spatial computing from Magic Leap, to provide surgeons an increased understanding of the patient’s anatomy.

Brainlab offers a version of Magic Leap’s technology to visualise and access medical imaging data to prepare for surgical procedures.

The latest headset from the Florida-based Magic Leap, the Magic Leap 2, will use 3D depth sensing technology developed by Infineon and pmdtechnologies. The headset is intended for the enterprise market, rather than gamers and will be one of the most immersive enterprise AR headsets available, claims the company. It is designed for collaboration among teams of staff with 3D indirect ToF depth sensing technology co-developed by the semiconductor company and the 3D ToF CMOS-based digital imaging company. 

It creates a 3D mapping of the environment as well as 3D images of faces, hand details or objects, in real time to enable accurate interaction. The ToF sensor developed by Infineon and pmd, is also claimed to enable enhanced gesture control while minimising power consumption, reducing heat and increasing battery life of the Magic Leap 2 headset.

Infineon and pmdtechnologies have developed the 3D ToF sensor used in the Magic Leap 2.

“Sensing the environment precisely and in real-time is key for augmented reality applications,” says Magic Leap’s CTO, Julie Green. “Magic Leap 2 will provide an even more immersive user experience,” she adds. 

Andreas Urschitz, division president, power and sensor systems, Infineon, believes the 3D ToF technology will enable new augmented and mixed reality applications for healthcare. “It’s about to change the way we live and work fundamentally.”

Explaining the role of the ToF sensor technology, Bernd Buxbaum, CEO of pmd, says it helps Magic Leap 2 “detect precisely, to the mm, the location of objects in a physical environment. Virtual objects can be placed in the real world, and stay in place when the user walks around a room and be obscured, as other real objects appear in front of them,” says Bernd Buxbaum, CEO of pmd. “It also works reliably in bright sunlight or complete darkness, where other depth-sensing solutions quickly reach their limits.”