Department Of Physics, IIT Delhi

Course Code: PYD561

Scope of Deconvolution and Single Image

Akashdeep (2022PHS7217)
Akash Naskar (2022PHS7180)

Adviser: Prof Kedar Bhalchandra Khare

Signature of student 1:

Signature of student 2:

1|Page
Abstract
The study different image deblurring techniques. Specifically, deconvolution and ESRGAN,
SRGAN. And comparing the outcomes from these methods. 1st we used DeconvolutionLab 2,
an open-source software, employed for correcting blurring and distortions in images. It is freely
accessible and can be linked to well-known imaging software platforms, such as ImageJ Fiji.
Thereafter we used ESRGAN (Enhanced Super-Resolution Generative Adversarial Network)
and SRGAN (Super-Resolution Generative Adversarial Network). They are both deep learning
models primarily designed for Single Image Super-Resolution (SISR), focusing on enhancing
image resolution by generating high-quality, high-resolution images from low-resolution
inputs.
The comparison will be drawn between deconvolution, SISR, and real microscope
measurements. Initial deconvolution results show a promising modification of images, and the
study intends to delve into Super-Resolution techniques next. Comparing these methods and
real microscope data aims to establish relationships between them, potentially improving
microscopic imaging techniques.

2|Page
1. Motivation and background
Imaging systems play an essential role in scientific and technological investigations. However,
images obtained through these systems, such as microscopes, often suffer from blurring due to
optical imperfections and other factors. To address this issue, deconvolution and image super-
resolution techniques have been developed, which hold the potential to significantly enhance
the clarity and detail of images.

2. Deconvolution
When we have an image that has been convolved with something, and we want to undo the
effect of the convolution, it is known as deconvolution.
Deconvolution techniques help in restoring the original clarity of images distorted by optical
imperfections, motion blur, or noise. These techniques are increasingly employed to enhance
image quality. DeconvolutionLab 2 is an open-source software package for image
deconvolution in microscopy. It is freely accessible and can be linked to well-known imaging
software platforms, such as ImageJ Fiji.

2.1 Methods of deconvolution

3|Page
Before delving into another technique like Single Image Super-Resolution (SISR), it's
important to grasp the concept of deep learning and its significance. Let's explore this through
a few fundamental questions.
What is deep learning?
It is a type of machine learning that uses neural networks with multiple layers to understand
and analyse data. Deep learning allows computers to learn patterns and make decisions without
explicit programming. It uses large amounts of data to train mathematical entities named deep
neural network to approximate complex functions whose inputs and outputs seems far apart.
What is Deep Neural Network (DNN)?
Deep Neural Network (DNNs), specific architecture within deep learning, are composed of
multiple levels of nonlinear operations, such as neural nets with many hidden layers. Deep
learning methods aim at learning feature hierarchies, where features at higher levels of the
hierarchy are formed using the features at lower levels. The network is trained in a supervised
mode using back-propagation algorithm to adjust weights.
By noting the ⅈ 𝑡ℎ layer of the network and the hidden unit of the layer, we have:
[𝑖] [𝑖]𝑇 [𝑖]
𝑧𝑗 = 𝑤𝑗 𝑥 + 𝑏𝑗

where the weight, bias, and output respectively.

4|Page
3. SISR Techniques
Single-image super-resolution (SISR) is a computer vision task that reconstructs a high-
resolution (HR) image from a low-resolution (LR) image. It could be used in a variety of
applications such as medical imaging, security, and surveillance imaging. The quality of the
reconstructed HR image depends on how to extract and use the information from LR image.
Since there are multiple HR images that can be down sampled to the same LR image and this
is a one-to-many mapping relation to recover HR images from a LR image, SISR is an
ill-posed and still challenging problem. Numerous SISR algorithms have been recently
proposed with different assumptions and evaluation criteria. Broadly speaking, SISR
algorithms can be categorized based on their tasks. ESRGAN and SRGAN are pivotal
advancements in the realm of Single-Image Super-Resolution (SISR) algorithms.

3.1 SRGAN
SRGAN (Super-Resolution Generative Adversarial Network) is a deep learning model
specifically designed for Single Image Super-Resolution (SISR), including the task of image
deblurring. It uses the power of Generative Adversarial Networks (GANs) to generate detailed,
realistic high-resolution images from low-resolution inputs.
The SRGAN consists of two main components: a generator and a discriminator. The generator
is responsible for generating a sharp image from a blurred image, while the discriminator is
responsible for distinguishing between real and fake images.
The SRGAN uses a perceptual loss function to train the generator . This loss function
measures the difference between the features of the blurred image and the features of the
generated image. The features are extracted from a pre-trained convolutional neural network,
such as VGG-19. SRGAN's training involves two main losses:
The perceptual loss function:

Adversarial Loss:

The total loss function is defined as follows:

3.2 ESRGAN
Enhanced Super-Resolution Residual Network (ESRRGAN) is built upon the foundation of
SRGAN but introduces significant improvements. It employs a Residual in Residual Dense
Block (RRDB) architecture, which combines a multi-level residual network and dense
connections, eliminating Batch Normalization. This architecture enables ESRRGAN to capture
intricate relationships within data and produce sharper images with reduced artifacts.
ESRRGAN outperforms SRGAN in terms of image deblurring performance, as:

5|Page
Peak Signal-to-Noise Ratio (PSNR): ESRRGAN consistently achieves higher PSNR values
across various upscaling factors, indicating sharper and less artifact-prone images.
Visual Quality: ESRRGAN-deblurred images exhibit superior sharpness and detail
preservation compared to their SRGAN-deblurred counterparts.

ESRGAN utilize the same standard perceptual loss function.

Adversarial Loss This is where the main difference between SRGAN and ESRGAN lies.

The total loss function is defined as follows:

3.3 Sample

Original Blur Image ESRGAN SRGAN

6|Page
Original Blur Image ESRGAN SRGAN

4. Test model
We used an infinity corrected brightfield microscope to observe some human tissues at 10X,
20X and 40X magnifications. Our main idea was to take the snapshot at different magnification,
say 40X, another at 20X snapshot of the same sample at the same configuration, everything
else unchanged and untouched. Now, we apply our SRGAN and ESRGAN models on the lower
resolved (in this example, 20X) image and try to de-blur it. Then we compare the resolved
picture and observe up to what extent these algorithms can go.

7|Page
For this, we have used an infinity corrected brightfield microscope, glass slides with inked
sample (fig.1) and 10X,20X,40X range lenses. Infinity corrected microscope works by
employing infinitely corrected objectives. Unlike traditional microscope, here the image is
formed at infinity (by the objective lens) and is resolved by other optical elements before being
magnified by the eyepiece.

5. Results and Discussion

8|Page
PSNR (Peak Signal-to-Noise Ratio) is a metric used to measure the quality of an image
reconstruction or restoration process. PSNR = 10 * log10(max^2 / MSE), calculates the ratio
between the maximum possible power of a signal and the distortion introduced by Mean
Squared Error (MSE) between the original and reconstructed images. A higher PSNR value
indicates lower perceived distortion and better image quality. Calculated PSNR values
PSNR value of the 1st pair: 28.036 dB
PSNR value of the 2nd pair: 24.549 dB
PSNR value of the 3rd pair: 25.091 dB
PSNR value of the 4th pair: 21.234 dB

1st pair of images is the best resolved for our sample since high PSNR values guarantees
better resolution. This proves that although the algorithm is same but it depends of he image
as well.

9|Page
6. Current Status and Future Plans of Work
From the results, we got a rough idea of how the image will get modified by using
deconvolution and SISR. We want to train the model based on our own data set. That will help
us to get better results.

7. Acknowledgement
We would like to express our sincere gratitude to Prof Kedar B Khare for his invaluable
guidance, support, and mentorship. He helped us to get familiar with software and basic
concepts.

8. References
a) Butola, M., Khare, K., Rajora, S.(2022). “Fourier optics and computational imaging”
b) Szeliski, R.(2021). “Computer vision: Algorithms and Applications”
c) Sage, D., Donati L., Soulez F.(2017). “DeconvolutionLab2: An open-source software for
deconvolution microscopy”
d) Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data.
Xintao Wang1 Liangbin Xie∗2,3 Chao Dong2,4 Ying Shan1 1Applied Research Center (ARC),
Tencent PCG 2 Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences
e) Deep Learning for Single Image Super-Resolution: A Brief Review Wenming Yang,
Xuechen Zhang, Yapeng Tian, Wei Wang, Jing-Hao Xue, Qingmin Liao

10 | P a g e