Open Access

Johanna BrandenburgORCiD, André SchulzeORCiD, Alexander JenkeORCiD, Nithya BhaskerORCiD, Noelle BleserORCiD, Denise JungerORCiD, Antonia Stern, Dominik Rivoir, Hamid NaderiORCiD, Fleur Fritz-KebedeORCiD, Oliver BurgertORCiD, Lena Maier-Hein, Lars Mündermann, Sebastian BodenstedtORCiD, Stefanie Speidel, Vladimir LozanovskiORCiD, Peter GrimmingerORCiD, Adrian BilleterORCiD, Rosa KlotzORCiD, Jürgen Weitz, Marius DistlerORCiD, Beat Müller-Stich, Martin WagnerORCiD

• Introduction Robot-assisted minimally invasive esophagectomy (RAMIE) is a complex procedure that may benefit from workflow analysis for context-aware assistance and surgical data science. This study aimed to model the RAMIE workflow, validate the applicability of the obtained workflow model in the operating room (OR) and retrospectively assess its generalizability across three academic centers using video data and automated workflow analysis with machine learning (ML). Methods A RAMIE workflow model was developed based on currently available literature, participatory OR observation, and expert interviews. This model was formalized to be included into a checklist tool to document the workflow live in the OR. To investigate generalizability of the workflow model, the surgical phases of 36 RAMIE videos from three different academic hospitals were retrospectively annotated. Based on this data set, a ML model was trained and tested within a six-fold cross validation. Results Ten surgical phases with 60 underlying steps were identified for RAMIE. The applicability of the workflow model was validated with live documentation in the OR. Multicenter video annotations revealed significant inter-institutional differences in the duration of all ten RAMIE phases. The ML model for automatic phase recognition showed an accuracy of 0.872 ± 0.091 and an f1-score of 0.872 ± 0.082 over all videos. The center with the best performing videos achieved a mean accuracy of 0.919 ± 0.036. Conclusion The RAMIE workflow was successfully modeled and validated in a retrospective multicenter setting. Despite high variability in phase duration between surgical centers, ML-based phase recognition achieved highly promising results.