Patrick Virtue

Jiayun Wang, Patrick Virtue, Stella X. Yu

Deep neural networks can be effective means to automatically classify aerial images but is easy to overfit to the training data. It is critical for trained neural networks to be robust to variations that exist between training and test environments. To address the overfitting problem in aerial image classification, we consider the neural network as successive transformations of an input image into embedded feature representations and ultimately into a semantic class label, and train neural networks to optimize image representations in the embedded space in addition to optimizing the final classification score. We demonstrate that networks trained with this dual embedding and classification loss outperform networks with classification loss only. %We also study placing the embedding loss on different network layers. We also find that moving the embedding loss from commonly-used feature space to the classifier space, which is the space just before softmax nonlinearization, leads to the best classification performance for aerial images. Visualizations of the network's embedded representations reveal that the embedding loss encourages greater separation between target class clusters for both training and testing partitions of two aerial image classification benchmark datasets, MSTAR and AID. Our code is publicly available on GitHub.

Patrick Virtue, Stella X. Yu, Michael Lustig

The task of MRI fingerprinting is to identify tissue parameters from complex-valued MRI signals. The prevalent approach is dictionary based, where a test MRI signal is compared to stored MRI signals with known tissue parameters and the most similar signals and tissue parameters retrieved. Such an approach does not scale with the number of parameters and is rather slow when the tissue parameter space is large. Our first novel contribution is to use deep learning as an efficient nonlinear inverse mapping approach. We generate synthetic (tissue, MRI) data from an MRI simulator, and use them to train a deep net to map the MRI signal to the tissue parameters directly. Our second novel contribution is to develop a complex-valued neural network with new cardioid activation functions. Our results demonstrate that complex-valued neural nets could be much more accurate than real-valued neural nets at complex-valued MRI fingerprinting.

Patrick Virtue, Michael Lustig

In Fourier-based medical imaging, sampling below the Nyquist rate results in an underdetermined system, in which linear reconstructions will exhibit artifacts. Another consequence of under-sampling is lower signal to noise ratio (SNR) due to fewer acquired measurements. Even if an oracle provided the information to perfectly disambiguate the underdetermined system, the reconstructed image could still have lower image quality than a corresponding fully sampled acquisition because of the reduced measurement time. The effects of lower SNR and the underdetermined system are coupled during reconstruction, making it difficult to isolate the impact of lower SNR on image quality. To this end, we present an image quality prediction process that reconstructs fully sampled, fully determined data with noise added to simulate the loss of SNR induced by a given under-sampling pattern. The resulting prediction image empirically shows the effect of noise in under-sampled image reconstruction without any effect from an underdetermined system. We discuss how our image quality prediction process can simulate the distribution of noise for a given under-sampling pattern, including variable density sampling that produces colored noise in the measurement data. An interesting consequence of our prediction model is that we can show that recovery from underdetermined non-uniform sampling is equivalent to a weighted least squares optimization that accounts for heterogeneous noise levels across measurements. Through a series of experiments with synthetic and in vivo datasets, we demonstrate the efficacy of the image quality prediction process and show that it provides a better estimation of reconstruction image quality than the corresponding fully-sampled reference image.