Geometric deep learning for medical image processing problems

Medical image processing provides an expanding set of methods and applications to improve clinical diagnosis, decision-making, and treatment planning through specific computational methods. Recent advances in deep learning (DL) have led to enhanced processing techniques that, in effect, increased the quality of medical image analysis. The success of DL models in medical imaging is often accompanied by the dependence on the quality and quantity of training data. However, available data is often sparse in medical imaging because of the cost of acquisition, the rarity of certain diseases, and the requirements for advanced imaging hardware. Furthermore, since medical images are derived from intricate anatomical structures and often depend on specific physical phenomena (e.g., water diffusion in magnetic resonance imaging), they reside in geometric domains or obey structural relationships that standard DL models do not necessarily respect. Geometric deep learning (GDL) is a family of DL methods designed to address both the data sparsity and the geometric complexity of medical images.

This thesis consists of four studies, each utilising appropriate GDL methods to develop a problem-specific medical image processing pipeline. The first study focuses on predicting stiffness tensors from micro-CT (μCT) trabecular bone scans. Trabecular bone involves complex structures, and the task requires learning relationships between input bone volumes and output stiffness tensors while operating under limited data availability. We project and learn the data in the spherical domain, extending established stiffness tensor prediction models. The second study investigates the prediction of the lung cancer survival rate based on tumour shape. We propose training a spherical convolutional neural network (SphCNN) model to infer survival rates from segmented CT images of non-small cell lung cancer. Our method is benchmarked against existing models, including radiomics feature-based approaches for image-based survival rate prediction. The third and fourth studies explore the structural tractography of diffusion MRI, addressing different parts of the connectivity pipeline. The third study deals with the challenge of rotational equivariance in reinforcement learning-based tractography algorithms. We propose integrating an SE3-equivariant transformer model into the tractography framework to improve performance under rotational transformations. The fourth study is centred around structural connectivity, combined with subsequent classification, where we apply graph neural networks (GNNs) in addition to other brain network-specific analyses to identify group differences in brain connectivity related to Parkinson’s disease.

Together, these four studies demonstrate how GDL methods can be adapted to different medical imaging problems, from biomechanics to oncology and neurology.

List of scientific papers

I. Fabian Sinzinger, Jelle van Kerkvoorde, Dieter H. Pahr, Rodrigo Moreno. Predicting the trabecular bone apparent stiffness tensor with spherical convolutional neural networks. Bone Reports. Volume 16, - 2022. https://doi.org/10.1016/j.bonr.2022.101179

II. Fabian Sinzinger, Mehdi Astaraki, Örjan Smedby, Rodrigo Moreno. Spherical Convolutional Neural Networks for Survival Rate Prediction in Cancer Patients. Frontiers in Oncology. Volume 12, - 2022. https://doi.org/10.3389/fonc.2022.870457

III. Fabian Sinzinger, Antoine Théberge, Rodrigo Moreno. Leveraging Rotational Equivariance for Reinforcement Learning in Tractography. [Manuscript]

IV. Fabian Sinzinger, Marvin Köpff, Joana Pereira, Rodrigo Moreno. Impact of Tractogram Filtering and Graph Creation for Structural Connectomics in Subjects with Parkinson’s Disease. [Manuscript]