Medical Education Gets Experiential

November 22, 2021

It’s been almost 30 years since the first virtual reality (VR) headsets were released for a general audience and adoption by the public of head-mounted displays (HMDs) for augmented reality (AR) experiences—including VR and mixed reality (MR)—has been slow. However, incredible technological innovations in the last half decade have begun making the promise of AR a reality. In fact, just recently Mark Zuckerberg, CEO of Meta Platforms (formerly Facebook, Inc), announced plans to begin building what he called the next iteration of the Internet: the metaverse—”a convergence of physical, augmented, and virtual reality in a shared online space¹.” This has only been made possible by those remarkable advances. Grandiose technological visions aside, AR presents groundbreaking opportunities for medical education.

Demand for this technology has grown rapidly over the last several years, across numerous industries, but especially in an industry like healthcare, which requires highly skilled operators who have increasingly diminished opportunities for hands-on experience². In the realm of human anatomy and invasive procedures, the use of AR is proving to be transformative. It can allow students to look inside the body to see the physiological processes at play, integrate biomedical data like MRIs and CT scans, and allow “trainees to prepare, practice, and review their performance repeatedly in an environment with no real-world consequences².” We’ve seen great success in augmented reality at institutions like Stanford, the University of California, San Francisco, Case Western, the University of Illinois College of Medicine, and the Children’s Hospital of Los Angeles.

Mayer’s Cognitive Theory of Multimedia Learning

AR is great for learners at all stages of their career, from beginner medical students to late-stage professionals. Training rooted in AR provides a great potential to efficiently prepare learners for real world practice because of both the safe learning environment it provides and how it enhances the learner’s overall experience. Learning using AR can be a highly positive experience—both fun and easy to use. We need to look no further than games like Pokemon Go, to see how successful, and fun, using AR can be. But most importantly, AR “can enhance learning delivery, presentation and the utilisation of sensory systems, which are three crucial elements of Mayer’s cognitive theory of multimedia learning³.”

Mayer’s theory is grounded on three main assumptions: there are two channels (auditory and visual) for processing information; these channels have limited capacity; and learning is “an active process of filtering, selecting, organizing, and integrating information⁴.” According to Mayer’s theory, there is a principle known as the “multimedia principle” stating that “people learn more deeply from words and pictures than from words alone⁴.” There’s only so much information humans can process at a time and the incoming information is made intelligible by creating mental representations. Mayer emphasizes three different memory stores:

Sensory—Receiving stimuli and storing it for a short time
Working—Actively processing information to create mental constructs, or schema
Long-term—The stores of everything learned

Mayer’s theory posits that “the brain does not interpret a multimedia presentation of words, pictures, and auditory information in a mutually exclusive fashion; rather, these elements are selected and organized dynamically to produce logical mental constructs⁴.” Education using AR is an engagement in what is called “experiential learning” and we know that experiential learning can boost retention by up to 90%⁵. By working with the three different memory stores and thus allowing the creation of more well-defined mental constructs, education using AR can be more effective at stimulating information retention than other more traditional mediums such as lectures. This has been our experience using AR at Forefront Collaborative.

Forefront Collaborative and the Microsoft HoloLens

Back in 2019, we identified the Microsoft HoloLens, an AR head-mounted display, as the best device for which we could develop a mixed-reality program on educational gaps in migraine. The challenge for HCPs who treat this disease is the varied neurologic approaches through which therapies target migraine. Mixed reality presents itself as the optimal means to address this need given the complexity of neurologic phenomena, the varied nature of therapies and benefit of interpersonal interaction to the education.

With our partners and collaborators, we developed 3D representations of neurons, receptors, and other relevant features to explore the pathophysiology of migraine and the mechanisms of action (MOAs) of five different classes of therapies. The biological features were developed at three levels of magnification: whole brain, cellular and receptor. Using the Unity platform, the 3D representations and animations were transformed into an interactive, mixed-reality experience.

We incorporated the migraine mixed-reality experience into two satellite symposia that were conducted in conjunction with national conferences. Here the results:

Ninety-five percent of respondents reported that the HoloLens presentation met the associated learning objective, resulting in 90% of all respondents feeling more confident in developing treatment plans in migraine.
Eighty-five percent of respondents specifically reported that the HoloLens feature enhanced their learning experience.
Ninety-three percent and 94% of respondents reported that the technical/scientific level of the HoloLens presentation and feature, respectively, was “appropriate.”
Ninety-four percent of respondents reported that the HoloLens activity was applicable to improving their clinical practice.
Sixty-five percent of respondents reported they would make changes in their practice as a result of the HoloLens activity.

This is just the beginning of our foray into AR. Recently, we conducted AR activities with two goals: to summarize the mechanisms of action, effectiveness, and safety of conventional and biologic DMARDs for treatment of psoriatic arthritis (PsA); and apply virtual strategies and tools, such as AR apps and patient decision aids, to remotely monitor PsA progression and manage patients with PsA. 89% of the learners surveyed said the educational activities improved their knowledge and 89% said they improved competency.

We believe the opportunities are limitless when it comes to this experiential learning technology. We look forward to the opportunity to bring this incredible development into more therapeutic areas where it will have an impact on learner retention and lead to an even greater understanding of all the remarkable innovations happening within the world of life sciences.