(125) Deep Learning-Based Scene Reconstruction from Monoscopic Videos in Otologic Surgery

Introduction: Recent advances in computer vision have enabled accurate reconstruction of 3D objects in monoscopic video, which can be used to augment endoscopic approaches in temporal bone surgery. This study presents a preliminary framework for recreating the surgical scene from monoscopic video using state-of-the-art neural surface reconstruction techniques.

Methods: Monocular video of a Phacon temporal bone phantom was recorded after a completed cortical mastoidectomy. A corresponding post-operative cone-beam CT scan was also obtained, and bony structures were manually segmented. Using the open-source structure-from-motion algorithm COLMAP, we obtained the 3D location and intrinsic parameters of the recording camera. Video images were then input to a neural surface reconstruction algorithm, which encodes positional information into corresponding pixels. A 3D mesh of the drilled phantom was extracted from this algorithm using marching cubes algorithm. The extracted mesh and the segmented CT were then co-registered and compared using average Hausdorff distance analyses.

Results: 3D information was recovered from 299516 surface points of the temporal bone. We achieved submillimeter reconstruction accuracy with an average Hausdorff distance of 0.98±1.76 mm. Among Hausdorff distances, 72.37% were less than 1 mm, 85.68% were less than 2 mm, and 92.33% were less than 3 mm.

Conclusions: This preliminary study has successfully demonstrated feasibility of a novel surface reconstruction technique for monocular video that leverages camera motion to generate 3D structures. Potential applications of this technique include video-based intra-operative navigation, integration with virtual surgical simulation platforms, and surgical education on patient-specific anatomy.