Ph.D. Research · 2021–2024
MultiRecon
Machine Learning for Multimodal Medical Image Reconstruction. Funded by the French National Research Agency (ANR).
The Research
PET and CT scans capture complementary information about the body — PET reveals metabolic activity while CT provides anatomical structure. Reconstructing them jointly (synergistically) can improve image quality beyond what either modality achieves alone.
My Ph.D. work developed multi-branch generative models — variational autoencoders (VAEs) and generative adversarial networks (GANs) — that serve as deep priors in the reconstruction process. These learned priors capture the joint distribution of PET and CT images, enabling information to flow between modalities during reconstruction.
The models were integrated into penalized weighted least-squares (PWLS) reconstruction using the ASTRA toolbox, and evaluated on clinical data in collaboration with physicists and clinicians at Brest University Hospital.
Key Contributions
Multi-Branch Generative Priors
Designed VAE and GAN architectures with separate encoder branches for PET and CT, sharing a joint latent space. This allows the model to learn cross-modal correlations while preserving modality-specific features.
Integration with PWLS Reconstruction
Embedded the learned generative priors as regularization terms in the iterative reconstruction framework. The deep prior guides the solution toward realistic image pairs without requiring paired training data at test time.
Clinical Validation
Evaluated reconstructed images with clinicians and physicists at Brest University Hospital, demonstrating improved image quality metrics (SSIM, PSNR) and diagnostic utility compared to conventional reconstruction methods.
Publications
Multibranch generative models for multichannel imaging with an application to PET/CT synergistic reconstruction
N. J. Pinton, A. Bousse, C. Cheze-Le-Rest, and D. Visvikis
IEEE Transactions on Radiation and Plasma Medical Sciences, vol. 9, no. 5, pp. 654–666, 2025
DOIMRI-guided PET reconstruction using a convergent plug-and-play approach
M. Savanier, V. Gautier, N. Pinton, C. Comtat, and F. Sureau
2024 IEEE Nuclear Science Symposium (NSS), Medical Imaging Conference (MIC) and Room Temperature Semiconductor Detector Conference (RTSD), IEEE, 2024
Joint PET/CT reconstruction using a double variational autoencoder
N. J. Pinton, A. Bousse, C. Cheze-Le-Rest, and D. Visvikis
IEEE Nuclear Science Symposium Medical Imaging Conference and Room Temperature Semiconductor Conference, 2023
Synergistic PET/CT reconstruction using a joint generative model
N. J. Pinton, A. Bousse, Z. Wang, C. Cheze-Le-Rest, et al.
International Conference on Fully Three-Dimensional Image Reconstruction in Radiology and Nuclear Medicine, 2023
Synergistic multi-energy reconstruction with a deep penalty "connecting the energies"
Z. Wang, A. Bousse, F. Vermet, N. J. Pinton, et al.
IEEE Nuclear Science Symposium Medical Imaging Conference and Room Temperature Semiconductor Conference, 2022
Python · TensorFlow · Keras · ASTRA Toolbox · OpenCV · numpy · Singularity · LaTeX