Christoph Kolbitsch

Casper da Costa-Luis, Matthias J. Ehrhardt, Christoph Kolbitsch et al.

Introduction: We describe the foundation of PETRIC, an image reconstruction challenge to minimise the computational runtime of related algorithms for Positron Emission Tomography (PET). Purpose: Although several similar challenges are well-established in the field of medical imaging, there have been no prior challenges for PET image reconstruction. Methods: Participants are provided with open-source software for implementation of their reconstruction algorithm(s). We define the objective function and reconstruct "gold standard" reference images, and provide metrics for quantifying algorithmic performance. We also received and curated phantom datasets (acquired with different scanners, radionuclides, and phantom types), which we further split into training and evaluation datasets. The automated computational framework of the challenge is released as open-source software. Results: Four teams with nine algorithms in total participated in the challenge. Their contributions made use of various tools from optimisation theory including preconditioning, stochastic gradients, and artificial intelligence. While most of the submitted approaches appear very similar in nature, their specific implementation lead to a range of algorithmic performance. Conclusion: As the first challenge for PET image reconstruction, PETRIC's solid foundations allow researchers to reuse its framework for evaluating new and existing image reconstruction methods on new or existing datasets. Variant versions of the challenge have and will continue to be launched in the future.

Felix F Zimmermann, Christoph Kolbitsch, Patrick Schuenke et al.

Quantitative Magnetic Resonance Imaging (qMRI) enables the reproducible measurement of biophysical parameters in tissue. The challenge lies in solving a nonlinear, ill-posed inverse problem to obtain the desired tissue parameter maps from acquired raw data. While various learned and non-learned approaches have been proposed, the existing learned methods fail to fully exploit the prior knowledge about the underlying MR physics, i.e. the signal model and the acquisition model. In this paper, we propose PINQI, a novel qMRI reconstruction method that integrates the knowledge about the signal, acquisition model, and learned regularization into a single end-to-end trainable neural network. Our approach is based on unrolled alternating optimization, utilizing differentiable optimization blocks to solve inner linear and non-linear optimization tasks, as well as convolutional layers for regularization of the intermediate qualitative images and parameter maps. This design enables PINQI to leverage the advantages of both the signal model and learned regularization. We evaluate the performance of our proposed network by comparing it with recently published approaches in the context of highly undersampled $T_1$-mapping, using both a simulated brain dataset, as well as real scanner data acquired from a physical phantom and in-vivo data from healthy volunteers. The results demonstrate the superiority of our proposed solution over existing methods and highlight the effectiveness of our method in real-world scenarios.

Andreas Kofler, Christian Wald, Tobias Schaeffter et al.

Sparsity-based methods have a long history in the field of signal processing and have been successfully applied to various image reconstruction problems. The involved sparsifying transformations or dictionaries are typically either pre-trained using a model which reflects the assumed properties of the signals or adaptively learned during the reconstruction - yielding so-called blind Compressed Sensing approaches. However, by doing so, the transforms are never explicitly trained in conjunction with the physical model which generates the signals. In addition, properly choosing the involved regularization parameters remains a challenging task. Another recently emerged training-paradigm for regularization methods is to use iterative neural networks (INNs) - also known as unrolled networks - which contain the physical model. In this work, we construct an INN which can be used as a supervised and physics-informed online convolutional dictionary learning algorithm. We evaluated the proposed approach by applying it to a realistic large-scale dynamic MR reconstruction problem and compared it to several other recently published works. We show that the proposed INN improves over two conventional model-agnostic training methods and yields competitive results also compared to a deep INN. Further, it does not require to choose the regularization parameters and - in contrast to deep INNs - each network component is entirely interpretable.

Andreas Kofler, Christian Wald, Tobias Schaeffter et al.

The concept of sparsity has been extensively applied for regularization in image reconstruction. Typically, sparsifying transforms are either pre-trained on ground-truth images or adaptively trained during the reconstruction. Thereby, learning algorithms are designed to minimize some target function which encodes the desired properties of the transform. However, this procedure ignores the subsequently employed reconstruction algorithm as well as the physical model which is responsible for the image formation process. Iterative neural networks - which contain the physical model - can overcome these issues. In this work, we demonstrate how convolutional sparsifying filters can be efficiently learned by end-to-end training of iterative neural networks. We evaluated our approach on a non-Cartesian 2D cardiac cine MRI example and show that the obtained filters are better suitable for the corresponding reconstruction algorithm than the ones obtained by decoupled pre-training.

Christiane Posselt, Mehmet Yigit Avci, Mehmet Yigitsoy et al.

Purpose: To provide a simulation framework for routine neuroimaging test data, which allows for "stress testing" of deep segmentation networks against acquisition shifts that commonly occur in clinical practice for T2 weighted (T2w) fluid attenuated inversion recovery (FLAIR) Magnetic Resonance Imaging (MRI) protocols. Approach: The approach simulates "acquisition shift derivatives" of MR images based on MR signal equations. Experiments comprise the validation of the simulated images by real MR scans and example stress tests on state-of-the-art MS lesion segmentation networks to explore a generic model function to describe the F1 score in dependence of the contrast-affecting sequence parameters echo time (TE) and inversion time (TI). Results: The differences between real and simulated images range up to 19 % in gray and white matter for extreme parameter settings. For the segmentation networks under test the F1 score dependency on TE and TI can be well described by quadratic model functions (R^2 > 0.9). The coefficients of the model functions indicate that changes of TE have more influence on the model performance than TI. Conclusions: We show that these deviations are in the range of values as may be caused by erroneous or individual differences of relaxation times as described by literature. The coefficients of the F1 model function allow for quantitative comparison of the influences of TE and TI. Limitations arise mainly from tissues with the low baseline signal (like CSF) and when the protocol contains contrast-affecting measures that cannot be modelled due to missing information in the DICOM header.

Felix Frederik Zimmermann, Patrick Schuenke, Christoph S. Aigner et al.

We introduce MRpro, an open-source image reconstruction package built upon PyTorch and open data formats. The framework comprises three main areas. First, it provides unified data structures for the consistent manipulation of MR datasets and their associated metadata (e.g., k-space trajectories). Second, it offers a library of composable operators, proximable functionals, and optimization algorithms, including a unified Fourier operator for all common trajectories and an extended phase graph simulation for quantitative MR. These components are used to create ready-to-use implementations of key reconstruction algorithms. Third, for deep learning, MRpro includes essential building blocks such as data consistency layers, differentiable optimization layers, and state-of-the-art backbone networks and integrates public datasets to facilitate reproducibility. MRpro is developed as a collaborative project supported by automated quality control. We demonstrate the versatility of MRpro across multiple applications, including Cartesian, radial, and spiral acquisitions; motion-corrected reconstruction; cardiac MR fingerprinting; learned spatially adaptive regularization weights; model-based learned image reconstruction and quantitative parameter estimation. MRpro offers an extensible framework for MR image reconstruction. With reproducibility and maintainability at its core, it facilitates collaborative development and provides a foundation for future MR imaging research.

Andreas Kofler, Dongyue Si, David Schote et al.

Recent innovations in Magnetic Resonance Imaging (MRI) hardware and software have reignited interest in low-field ($<1\,\mathrm{T}$) and ultra-low-field MRI ($<0.1\,\mathrm{T}$). These technologies offer advantages such as lower power consumption, reduced specific absorption rate, reduced field-inhomogeneities, and cost-effectiveness, presenting a promising alternative for resource-limited and point-of-care settings. However, low-field MRI faces inherent challenges like reduced signal-to-noise ratio and therefore, potentially lower spatial resolution or longer scan times. This chapter examines the challenges and opportunities of low-field and ultra-low-field MRI, with a focus on the role of machine learning (ML) in overcoming these limitations. We provide an overview of deep neural networks and their application in enhancing low-field and ultra-low-field MRI performance. Specific ML-based solutions, including advanced image reconstruction, denoising, and super-resolution algorithms, are discussed. The chapter concludes by exploring how integrating ML with low-field MRI could expand its clinical applications and improve accessibility, potentially revolutionizing its use in diverse healthcare settings.

Andreas Kofler, Marc Dewey, Tobias Schaeffter et al.

In this work, we propose an iterative reconstruction scheme (ALONE - Adaptive Learning Of NEtworks) for 2D radial cine MRI based on ground truth-free unsupervised learning of shallow convolutional neural networks. The network is trained to approximate patches of the current estimate of the solution during the reconstruction. By imposing a shallow network topology and constraining the $L_2$-norm of the learned filters, the network's representation power is limited in order not to be able to recover noise. Therefore, the network can be interpreted to perform a low dimensional approximation of the patches for stabilizing the inversion process. We compare the proposed reconstruction scheme to two ground truth-free reconstruction methods, namely a well known Total Variation (TV) minimization and an unsupervised adaptive Dictionary Learning (DIC) method. The proposed method outperforms both methods with respect to all reported quantitative measures. Further, in contrast to DIC, where the sparse approximation of the patches involves the solution of a complex optimization problem, ALONE only requires a forward pass of all patches through the shallow network and therefore significantly accelerates the reconstruction.

Andreas Kofler, Markus Haltmeier, Tobias Schaeffter et al.

In this paper we present a generalized Deep Learning-based approach for solving ill-posed large-scale inverse problems occuring in medical image reconstruction. Recently, Deep Learning methods using iterative neural networks and cascaded neural networks have been reported to achieve state-of-the-art results with respect to various quantitative quality measures as PSNR, NRMSE and SSIM across different imaging modalities. However, the fact that these approaches employ the forward and adjoint operators repeatedly in the network architecture requires the network to process the whole images or volumes at once, which for some applications is computationally infeasible. In this work, we follow a different reconstruction strategy by decoupling the regularization of the solution from ensuring consistency with the measured data. The regularization is given in the form of an image prior obtained by the output of a previously trained neural network which is used in a Tikhonov regularization framework. By doing so, more complex and sophisticated network architectures can be used for the removal of the artefacts or noise than it is usually the case in iterative networks. Due to the large scale of the considered problems and the resulting computational complexity of the employed networks, the priors are obtained by processing the images or volumes as patches or slices. We evaluated the method for the cases of 3D cone-beam low dose CT and undersampled 2D radial cine MRI and compared it to a total variation-minimization-based reconstruction algorithm as well as to a method with regularization based on learned overcomplete dictionaries. The proposed method outperformed all the reported methods with respect to all chosen quantitative measures and further accelerates the regularization step in the reconstruction by several orders of magnitude.

Jackie Ma, Maximilian März, Stephanie Funk et al.

High-resolution three-dimensional (3D) cardiovascular magnetic resonance (CMR) is a valuable medical imaging technique, but its widespread application in clinical practice is hampered by long acquisition times. Here we present a novel compressed sensing (CS) reconstruction approach using shearlets as a sparsifying transform allowing for fast 3D CMR (3DShearCS). Shearlets are mathematically optimal for a simplified model of natural images and have been proven to be more efficient than classical systems such as wavelets. Data is acquired with a 3D Radial Phase Encoding (RPE) trajectory and an iterative reweighting scheme is used during image reconstruction to ensure fast convergence and high image quality. In our in-vivo cardiac MRI experiments we show that the proposed method 3DShearCS has lower relative errors and higher structural similarity compared to the other reconstruction techniques especially for high undersampling factors, i.e. short scan times. In this paper, we further show that 3DShearCS provides improved depiction of cardiac anatomy (measured by assessing the sharpness of coronary arteries) and two clinical experts qualitatively analyzed the image quality.