Convolutional neural network transformer (CNNT) for fluorescence microscopy image denoising with improved generalization and fast adaptation.

janelia7_blocks-janelia7_biblio_header | block
Sci Rep. 2024 Aug 06;14(1):18184. doi: 10.1038/s41598-024-68918-2
View Article on
Convolutional neural network transformer (CNNT) for fluorescence microscopy image denoising with improved generalization and fast adaptation. Shroff Lab
Rehman A, Zhovmer A, Sato R, Mukouyama Y, Chen J, Rissone A, Puertollano R, Liu J, Vishwasrao HD, Shroff H, Combs CA, Xue H
janelia7_blocks-janelia7_biblio_abstract | block

Abstract

Deep neural networks can improve the quality of fluorescence microscopy images. Previous methods, based on Convolutional Neural Networks (CNNs), require time-consuming training of individual models for each experiment, impairing their applicability and generalization. In this study, we propose a novel imaging-transformer based model, Convolutional Neural Network Transformer (CNNT), that outperforms CNN based networks for image denoising. We train a general CNNT based backbone model from pairwise high-low Signal-to-Noise Ratio (SNR) image volumes, gathered from a single type of fluorescence microscope, an instant Structured Illumination Microscope. Fast adaptation to new microscopes is achieved by fine-tuning the backbone on only 5-10 image volume pairs per new experiment. Results show that the CNNT backbone and fine-tuning scheme significantly reduces training time and improves image quality, outperforming models trained using only CNNs such as 3D-RCAN and Noise2Fast. We show three examples of efficacy of this approach in wide-field, two-photon, and confocal fluorescence microscopy.

PMID: 39107416 [PubMed - indexed for MEDLINE]

node:body | entity_field

Previous arXiv preprint: https://doi.org/10.48550/arXiv.2404.04726

janelia7_blocks-janelia7_biblio_authors | block

Janelia Authors

Hari Shroff
janelia7_blocks-janelia7_biblio_tools | block