Precise Planning for Heart Surgery

Heart valve operations are often very complex and have to be prepared with great precision. The Fraunhofer Institute for Digital Medicine MEVIS, the German Heart Center Berlin, and the Charité are developing an assistance system that will ease planning interventions by using virtual reality. In the future, patients are also likely to benefit from this new technology.

Surgery is often necessary when a heart valve no longer closes properly. In these operations, special rings or threads, for example, are sewn into the inlet or outlet of the heart chamber that the valve is supposed to close. They enable the valve leaflets to connect again. Some operations are still performed on an open heart, but patient-friendly minimally invasive procedures are increasingly being used.

“Heart valve surgery can be very complex, often combining several measures,” says MEVIS researcher Anja Hennemuth, a professor at the Institute for Computer-Assisted Cardiovascular Medicine (ICM), a joint facility of Charité and the German Heart Center Berlin. “Often, the team has to make many quick, ad hoc decisions during surgery.” Thorough planning of the procedure must take place to avoid potential mistakes and minimize risk to patients.

This is where Fraunhofer MEVIS aims to support medical professionals. The institute is developing an AI-supported assistance system that adheres to guidelines to optimize the planning and execution of heart valve surgery and thereby improve operation quality. Specifically, the team will be able to simulate the surgery realistically on the computer even during preliminary discussions. To choose the best possible strategy, the experts can test various alternatives. They can help discover, for example, which type of ring should be used and at which exact location it should be inserted.

An AI that analyzes heart valve movement

The new assistance system is based on moving images, for example, from a CT or MRI scanner. These portray the heart of a patient in action and can uncover, for example, when the mitral valve, through which blood flows into the heart, no longer closes correctly. “We can extract an anatomical model of the patient from this image data using computer graphics methods,” Hennemuth explains. “This digital model doesn’t just display the heart; it also shows how the valves open and close again, like in a 3D movie.” Artificial intelligence plays a crucial role in creating this model. Learning algorithms can recognize individual structures in the image data, analyze the heart valve movement quickly, and transfer them to the digital heart model.

The digital model can then be observed using AR/VR glasses: the beating heart floats in 3D directly in front of the user and can be seen from different angles. It can also be displayed on a conventional screen and manipulated using a mouse. This is practical, for example, for experts joining the meeting using teleconferencing. The team can use this model during a preliminary meeting to simulate the intervention in advance. Specialists can, for example, insert virtual sutures at different locations or attach rings of different sizes. They can determine which of the actions would have the most beneficial effect on heart valve movement.

Support in the operating room

The new MEVIS concept, however, goes even further – right into the operating room, where images of the virtual and real heart can be displayed side by side or even superimposed during the procedure. This lets the team see the planned operation literally before their very eyes.

MEVIS scientists have already developed and tested building blocks for this innovative concept alongside their project partners. They generated a database of currently available valve rings and compared their model with results from real interventions. A follow-up project called MINIMAKI began in March 2021. “Here, we aim to merge the individual elements,” explains Anja Hennemuth. “We’re also working on streamlining 3D interactions so that medical professionals can use the new assistance system as intuitively as possible.”

One of the challenges: Some personnel and patients find using AR/VR glasses to be annoying and disturbing. The glasses can, for example, obscure the facial expressions of those in front of them. Fraunhofer MEVIS is researching these aspects in collaboration with medical ethicists as part of the new project. “Fortunately, AR/VR glasses are becoming lighter and more comfortable to wear,” Hennemuth emphasizes. “We’re also working on a version that can hand off a session to a tablet or computer.” The first complete version of the new assistance system should be ready in two to three years. Soon afterwards, it could start being used in clinics and make it easier for both doctors and patients to prepare for heart valve surgery.