PrimeEnhance™
Introduction
There is currently a revolution in computational imaging, pairing the power of a cutting-edge scientific camera, such as the Prime Family of CMOS cameras from Teledyne Photometrics, with powerful graphics processors from the latest generation of GPUs. Machine learning, data processing, and the camera-PC interface are all going to help imaging evolve, through methods such as denoising and localization.
In this article, we explore our 2D active image denoising capability for the Prime Family of sCMOS cameras, PrimeEnhance™. Prime's computational power matches the latest generation of GP GPU's while making the technology easy to deploy and accessible by embedding computational power inside the camera.
Photon Shot Noise
While camera and sensor manufacturers have made steady progress towards perfecting photometric imaging (such as minimizing read noise with technology such as Correlated Multi Sampling), low light imaging remains difficult due to the presence of another source of noise, photon shot noise.
Photon shot noise is an inherent property of light. There is always a statistical variation in the number of photons (or photoelectrons) detected in a given time period. This uncertainty is dependent on the number of signal photoelectrons being measured and has the statistical property of a Poisson distribution. For N signal, there will be √N shot noise, meaning that while shot noise increases as signal increases, it does so more slowly (due to the square root), meaning that shot noise has a more significant impact at low light levels, as outlined with Table 1.
| Signal Level (e-) | Shot Noise (e- RMS) | Percent of signal (%) |
| 5 | 2.23 | 44.8 |
| 10 | 3.16 | 31.6 |
| 50 | 7.07 | 14.1 |
| 100 | 10 | 10 |
| 500 | 22.36 | 4.5 |
| 1000 | 31.62 | 3.2 |
Table 1: The effect of photon shot noise at varying signal levels.
In order to improve the signal-to-noise ratio (SNR) at lower signal levels, researchers could increase exposure or light level, but the former affects the camera acquisition speed, and both affect the sample. A remaining technique to reduce the effects of photon shot noise is the use of a denoising algorithm.
Denoising algorithms can dynamically examine acquired images in order to separate and remove noise. The Prime™ family of cameras from Photometrics introduces a new real-time method for dynamic noise reduction called PrimeEnhance™.
PrimeEnhance™
When processing data to reduce noise, there are many challenges to consider, such as preserving the quantitative nature of the recorded pixel intensities, preserving key features (edges, textures, details with low contrast), and processing has to be accomplished without introducing new image artifacts like ringing, aliasing or blurring. Many denoising algorithms are currently available, and some are incompatible with certain image types, resulting in artifacts. Additionally, because noise tends to vary with the level of signal, it is difficult for many denoising algorithms to distinguish signal from noise, and as a consequence, small details tend to be removed.
For PrimeEnhance, we have used an algorithm invented at the French National Institute for Research in Computer Science and Automation (INRIA) by Jérôme Boulanger et al. This algorithm has been optimized for fluorescence microscopy in collaboration with the Institute Curie, and works with the fact that while shot noise is a fundamental property of light, fluctuations caused by shot noise behave in a predictable fashion governed by Poisson statistics. PrimeEnhance implements a 2D denoising process that evaluates and processes incoming images in real-time to reduce the effects of photon shot noise at low signal levels. The algorithm also preserves the finer details and features of biological samples and does not introduce image artifacts. In addition, PrimeEnhance is a quantitative algorithm, ensuring that intensity values remain unchanged. The implementation of PrimeEnhance has a significant impact on photon shot noise, winning the Vision Systems Innovator Award 2016.
PrimeEnhance works by being aware of each camera's characteristics and specifications. It uses this knowledge to first evaluate the image data and perform a variance stabilization transform, which removes the dependency between the mean intensities and their noise characteristics. Then a small patch of pixels is compared to similar sized patches in iteratively increasing areas of surrounding pixels (neighborhoods). The pixels within the neighborhood are selectively weighted based on their similarity to the intensity values of the original patch and using these weighted corrections, the original patch is updated. This process is repeated throughout the entire image, updating each patch and reducing the impact of shot noise. Once this process has been completed, the inverse variance stabilizing transform is applied to ensure that the quantitative nature of the pixel values is maintained.
PrimeEnhance Evaluation
In Figure 1 two images are shown, acquired without (left) and with (right) PrimeEnhance. Both images are of the same sample and with the same exposure. The mean, minimum, and maximum intensity values remain unchanged between both images, demonstrating that all measurements made remain quantitative and are relatable to each other.
In the PrimeEnchance image, the standard deviation has been reduced, indicating the removal of noise. Within the PrimeEnhance image photon shot noise has been reduced, while preserving image features, not generating artifacts, and extracting features that were previously indistinguishable from the noise.
Figure 1: The effect of PrimeEnhance. The top image is raw, the bottom image has PrimeEnhance activated.
By increasing the effective signal-to-noise in each frame, it is possible to acquire high-quality images at lower exposure times, reducing the effects of phototoxicity and photobleaching on samples.
By way of demonstration, Fig.2 shows three imaging conditions: raw images taken with 100 ms and 800 ms exposure, and a PrimeEnhance image taken with 100 ms exposure. All images are accompanied by line profiles to show the presence of noise and how clear the signals are.
Figure 2: PrimeEnhance and exposure time. The raw image at 100 ms has a low signal and is more significantly impacted by photon shot noise, this can be improved by either increasing the exposure by 8x (see 800 ms raw image) or by using PrimeEnhance.
The raw 100 ms image contains considerable noise and features are unclear, increasing the exposure to 800 ms results in 8x more signal and a clearer image. However, increasing the exposure limits the camera speed and exposes the sample to more light, which can lead to bleaching and photodamage, and also requires more time per image acquisition, adding inefficiency to experimentation. The alternative is keeping the exposure at 100 ms and using PrimeEnhance, which also removes noise, as seen from the line profile and clearer features, resulting in high image quality while maintaining low exposures.
Summary
PrimeEnhance provides a real-time quantitative increase in signal-to-noise ratio by reducing the effects of photon shot noise at low light levels, which improves the quality of images and data. The finer features within images are preserved and no unwanted processing artifacts are generated. A comparison between a 100 ms denoised image and an 800 ms standard fluorescence image shows equivalent results in image quality, indicating the ability to significantly reduce exposure times while maintaining the quality of captured data.
Reference
Jérôme Boulanger, Charles Kervrann, Patrick Bouthemy, Peter Elbau, Jean-Baptiste Sibarita, Jean Salamero (2010) Patch-Based Nonlocal Functional for Denoising Fluorescence Microscopy Image Sequences, IEEE Transactions on Medical Imaging, Institute of Electrical and Electronics Engineers, 29 (2), pp.442-453. ⟨10.1109/TMI.2009.2033991⟩. ⟨inria-00541082⟩
Further Reading
Back To Exclusive Features
Book A Demo Today