Adjusting videoendoscopic 3D reconstruction results using tomographic data

Abstract

Videoendoscopic and tomographic research are the two leading medical imaging solutions for detecting, classifying and characterizing a wide array of pathologies and conditions. However, source information from these types of research is very different, making it hard to cross-correlate them. The paper proposes a novel algorithm for combining results of videoendoscopic and tomographic imaging data based on 3D surface reconstruction methods. This approach allows to align separate parts of two input 3D surfaces: surface obtained by applying bundle adjustment-based 3D surface reconstruction algorithm to the endoscopic video sequence, and surface reconstructed directly from separate tomographic cross-section slice projections with regular density. Proposed alignment method is based on using local feature extractor and descriptor algorithms by applying them to the source surface normal maps. This alignment allows both surfaces to be equalized and normalized relative to each other. Results show that such an adjustment allows to reduce noise, correct reconstruction artifacts and errors, increase representative quality of the resulting model and establish correctness of the reconstruction for hyperparameter tuning.