Open Access

Nora Dimitrova, Armin Teubert, Tim Klopfer, Anna Manawapat-Klopfer, Thomas Notheisen, Heiko Baumgartner, Christoph Gonser, Ramy ZeineldinORCiD, Oliver BurgertORCiD

• This paper discusses the development and application of an augmented reality (AR) system for assisting in nail implantation procedures for complex tibial fractures. Traditional procedures involve extensive x-ray usage from various angles, leading to increased radiation exposure and prolonged surgical times. The study presents a method using pre- and post-operative computed tomography (CT) data sets and a convolutional neural network (CNN) trained on segmented bone and metal objects. The augmented reality system overlays accurate 3D representations of bony fragments and implants onto the surgeon's view, aiming to reduce radiation exposure and intervention time. The study demonstrates successful segmentation of bone and metal objects in cases of heavy metal artifacts, achieving promising results with a relatively low number of training sets. The integration of this system into the clinical workflow could potentially improve surgical outcomes, significantly reduce radiation time, and therefore improve patient safety.