Noise Reduction
NOISE REDUCTION IN IMAGES
A project report submitted in partial fulfilment of the requirements
for the award of the degree of
Bachelor of Technology
In
Department Of Computer Science And Engineering
By
Akash Roy Choudhury -)
Aniket Patra -)
DEPARTMENT OF COMPUTER SCIENCE AND
ENGINEERING
JALPAIGURI GOVERNMENT ENGINEERING
COLLEGE
JALPAIGURI, WEST BENGAL-735102.
FEBRUARY 2017
BONAFIDE CERTIFICATE
This is to certify that the project titled NOISE REDUCTION IN
IMAGES is a bonafide record of the work done by
Akash Roy Choudhury -)
Aniket Patra -)
in partial fulfilment of the requirements for the award of the degree of
Bachelor of Technology in Department of Computer Science and
Engineering of the JALPAIGURI GOVERNMENT ENGINEERING
COLLEGE during the year-.
NAME
Guide
DR. DIPAK KUMAR KOLE
Head Of Department
Project Viva-voce held on _____________________________
Internal Examiner
External Examiner
i
ABSTRACT
The given problem falls under the domain of IMAGE PROCESSING.
In the world of photography and machine vision, blurry images can spell
disaster. They can ruin an otherwise perfect photo or make it impossible
for a computer to recognize the image or certain components of it for
processing. The best way to counter this without taking another, clearer
picture is to utilize deconvolution techniques to remove as much blur as
possible. That is the design of this project. Our plan was to first design a
program that takes an image, blurs it using a known blur kernel, then
deblurs it to reproduce the original image. Then we created a program to
take the noisy deblurred image and smooth it using noise reduction. We
used Python as our programming language and the .pgm uncompressed
image format. Our success was measured simply by how much the output
(deblurred) image matches the input (original) image.
Keywords: deblurring, image processing, noise reduction.
ii
ACKNOWLEDGEMENT
We deeply express our sincere thanks to our Head of Department Dr. Dipak
Kumar Kole, for encouraging and allowing us to present the project on the
topic “Noise Reduction in Images” using Python. It is a privilege to express
our sincerest regards to our project coordinator and respected senior,
Somnath Rakshit for his valuable inputs, able guidance, encouragement,
whole-hearted cooperation and constructive criticism throughout the
duration of our project. We would like to take this opportunity to thank all
the lecturers who have directly or indirectly helped this project. We pay our
respects and love to our parents and all other family members for their love
and encouragement throughout. Last but not the least we would like to
express our gratitude to our friends for their cooperation and support.
iii
TABLE OF CONTENTS
Title
Page No.
Abstract………..………………………………………………….ii
Acknowlegement………...……………………………………….iii
Table of Contents………………………………………………...iv
List of Figures…………….……………………………………...vi
Chapter 1 Introduction
1.1
Project Objective..…………………………………………1
1.2
General Introduction…………………….……….………..1
CHAPTER 2
Literature Review
2.1
Noise………………………………….…………….……..2
2.2
Image Processing………………………………………….2
2.3
Deblurring……………………………………………....…3
2.4
Noise Reduction…………………………………………...3
2.5
Other Research………………………………………….…4
CHAPTER 3
Methodology
3.1
Language Used…………………………………………….5
3.2
Project Design……………………………………...……...5
3.3
Testing………………………………………………...…..6
3.4
Theory……………………………………………………..6
3.5
Noise Reduction Algorithm…………………………….…9
iv
CHAPTER 4
Results and Discussion
4.1
Experimental Result…………………………….….…..10
4.2
System Information…………………………….….…...12
4.3
Limitations………………………………………….….12
CHAPTER 5
Summary and Conclusion
5.1
Summary…………………………………….…………14
5.2
Conclusion……………………………………………..14
5.3
Scope and Future Work………………………..………14
References…………………………………………...……….16
v
LIST OF FIGURES
Fig no.
Title
Page no.
1.
Results of IIFT………………………………….8
2.
Result of Blurring……………………………….10
3.
Result of Deblurring…………………………….11
4.
Result of Noise Reduction………………………11
5.
Final Comparison………………………………..13
vi
CHAPETER 1
INTRODUCTION
1.1
PROJECT OBJECTIVE
The goal of this project was to create a program that can take an image
input that has been blurred and to employ image deblurring techniques to
restore the image and create a sharp, more recognizable output image with
as few blur artefacts and noise as much as possible.
1.2
GENERAL INTRODUCTION
Noise is the result of errors in the image acquisition process that results in
pixel values that do not reflect the true intensities of the real scene. Noise
reduction is the process of removing noise from a signal. Noise reduction
techniques are conceptually very similar regardless of the signal being
processed, however a priori knowledge of the characteristics of an expected
signal can mean the implementations of these techniques vary greatly
depending on the type of signal. The image captured by the sensor
undergoes filtering by different smoothing filters and the resultant images.
All recording devices, both analogue and digital, have traits which make
them susceptible to noise. The fundamental problem of image processing
is to reduce noise from a digital colour image.
1
CHAPTER 2
LITERATURE REVIEW
2.1
NOISE
The most commonly occurring types of noise are i) Impulse noise, ii)
Additive noise (e.g. Gaussian noise) and iii) Multiplicative noise(e.g.
Speckle noise).
Impulse noise is usually characterized by some portion of image pixels
that are corrupted, leaving the remaining pixels unchanged. Examples of
impulse noise are fixed-valued impulse noise and randomly valued impulse
noise.
We talk about Additive noise when value from a certain distribution is
added to each image pixel, for example, a Gaussian distribution.
Multiplicative noise is generally more difficult to remove from images
than additive noise because the intensity of the noise varies from the signal
intensity (e.g., speckle noise)
2.2
Image processing
Image Processing is a method to convert an image into digital form and
perform some operations on it, in order to get an enhanced image or to
extract some useful information from it. It is a type of signal dispensation
in which input is image, like video frame or photograph and output may be
image or characteristics associated with that image. Usually Image
Processing system includes treating images as two dimensional signals
while applying already set signal processing methods to them.
2
2.3
Deblurring
We found a paper regarding image deblurring and noise suppression called
"Image Deblurring with Blurred/Noisy Image Pairs" by Lu Yuan, et al. that
we utilized in helping us understand the techniques and algorithms that go
into reducing the noise of and deblurring an image. In their research they
used a blurry image with proper intensity and poor sharpness and paired it
with an identical picture with good sharpness but poor intensity and riddled
with noise to create a sharp, correct intensity output with few or no artefacts
left in the output image. Another paper we read discusses an algorithm that
the group of researchers discovered that allows for a mostly accurate
estimation of the blur kernel, or the function through which the pixel values
of the image are blurred. Their algorithm takes four inputs: the blurry
image, a section of the image that has a good sample of blurring (in case
the image is not uniformly blurred), if the blur is estimated to be more
horizontal or more vertical, and the estimated size of the blue kernel. Given
these inputs, their algorithm can sufficiently estimate the blur kernel such
that the image, which was captured using poor technique with a standard,
off-the-shelf camera, is satisfactorily deblurred with few artefacts after
deconvolution. Any artefacts that are left can generally be removed by an
experienced photo editor.
2.4
Noise reduction
Noise Reduction is the last step in our deblurring process. We've learned
that Gaussian smoothing (replacing every pixel with a weighted average of
its surrounding pixels) can reduce noise by averaging it with surrounding
color values. The problem exists, though, that it smooths the image and
therefore softens its edges, reducing the desired effect of the deblurring
process. To learn how to reduce noise and maintain edge sharpness, we
3
read a paper by German scientist Dr. Holger Adelmann entitled "An edgesensitive noise reduction algorithm for image processing." In his paper,
Adelmann discusses using a 5x5 neighbourhood to determine in which of
four directions the edge that pixel is located on is oriented. Then, using that
information, Gaussian smoothing is applied using a kernel that matches the
direction of the edge, instead of the normal 3x3 square kernel. By using
edge sensitive kernels, edge sharpness is maintained while still reducing
noise.
2.5
Other Research
Through our own work we have accrued a little understanding of basic and
intermediate image processing techniques and algorithms from various
online
worksheets
and
lessons
at
http://homepages.inf.ed.ac.uk/rbf/HIPR2/wksheets.htm . We plan to use
these techniques to help us code and understand the more complex concepts
behind image deblurring and the intermediate steps involved. For example,
I have extensively used the section referring to the Fourier Transform,
located here: http://homepages.inf.ed.ac.uk/rbf/HIPR2/fourier.htm .
4
CHAPTER 3
METHODOLOGY
3.1
Language Used: Python.
3.2
Project Design
We decided to use Python because of its simplicity and adaptability. As for
the images, we used the uncompressed, grayscale .pgm image format. This
allowed us to confirm the accuracy of the outputs because of the
uncompressed nature of the .pgm format, which means that the image
information doesn't need to be altered before being saved. Also, it is much
easier to code using the .pgm format since it can be read in as and saved as
a string without using any packages or software, also making it more
reliable.
Also, for the purposes of this project, we chose to focus on motion
blur convolution and deconvolution. This doesn't affect the image to be
deblurred, but rather the design of the kernel used to blur and deblur the
image. We found that our chosen method of image division deconvolution
doesn't work correctly on other forms of blur, such as Gaussian blur and
potentially out-of-focus camera blur.
The first step in this project was to artificially blur an input image
using a known and given blur kernel. This is accomplished by converting
both images to the frequency domain, using the Fast Fourier Transform
(FFT), point multiplying the two images, then converting them back to the
spatial domain using the Inverse Fast Fourier Transform (IFFT). This is
known as convolution.
The next step was the deconvolution algorithm that, when given an
image and its known blur kernel, could deblur the input image. This is fairly
straightforward and involves the reverse of the aforementioned convolution
5
algorithm. This is done by instead point dividing the blurred image by the
blur kernel in the frequency domain. The final step was to take the noisy
image that was acquired through the deconvolution process and apply a
noise reduction algorithm. This noise reduction algorithm attempted to
remove as much of the noise from the deblurred image as possible, while
still maintaining sharpness and clarity.
3.3
Testing
Our project's success was measured by its ability to take an artificially
blurred image and return it to its original, sharp quality. We tested our
project's adaptability and thoroughness by running a series of tests that
entailed attempting to deblur images of different contrast and content with
varying magnitudes and directions of blur. This tested the program's ability
to repair images regardless of image content, or magnitude or direction of
blur distortion, although there will obviously be an upper limit to the
amount of blur that can plausibly be removed.
3.4
Theory
Fourier Transform: The Fourier Transform is heavily involved with
image convolution and deconvolution because it allows for greater speed
and simpler code. The Fourier Transform converts values in an array from
the spatial domain to the frequency domain using a sum of complex
numbers, as given by the equation:
6
The 2-Dimensional Discrete Fourier Transform (DFT) does this using a
matrix or 2D array of values and uses a nested sum:
Since the 2-Dimensional Discrete Fourier Transform uses a nested sum, it
can be separated to create two 1-Dimensional Fourier Transforms in a row,
first in one direction (vertically or horizontally), then in the other direction.
This is known as the Fast Fourier Transform (FFT) and runs significantly
faster than the DFT, since the DFT has a runtime of O(n2) and the FFT has
a runtime of O(nlog2n). The following is an example of a picture being
converted from the spatial domain to the frequency domain via the Fourier
Transform.
is then transformed to
The reason the FFT is so important to image convolution and deconvolution
is that it takes long iterative algorithms and turns them into simple point
arithmetic. For example, image convolution becomes as simple as taking
the Fourier Transform of the image and the blur kernel (known as the Point
7
Spread Function (PSF) after transformation), transforming them to the
frequency domain and point multiplying the two images. Then the two
images can be converted back to the spatial domain by the Inverse Fourier
Transform, given by:
and the result will be a blurry (convoluted) image. To reverse this process
and deconvolution the image, assuming the blur kernel is known, is as
simple as point dividing the transformed image by the PSF, instead of
multiplying. The IDFT can also be separated and turned into the Inverse
Fast Fourier Transform (IFFT). When using the IDFT or IFFT, though, the
values need to be the full complex numbers from the Fourier Transform.
This means that the IFFT cannot be performed on an image that is
transformed to display the magnitude or the phase of the Fourier
Transform. This is demonstrated below.
FIG NO.1 This shows the original image, the result of the IFFT using only the
magnitude of the Fourier Transform output, and the result of the IFFT using only the
phase of the Fourier Transform output.
8
3.5
Noise Reduction Algorithm
We learned that Gaussian smoothing can be employed to reduce the noise
in an image. However, this softens the image and therefore is not a desirable
approach. To start, we took the basic concept of Gaussian smoothing
(weighted average of the 3x3 neighbourhood for each pixel) and adapted it
to better handle noise. For each pixel, we took the average of all the pixels
in the 3x3 neighbourhood. Then, we determined which of the 9 pixels in
the neighbourhood had values that fell within 15 of the average and took
the average of those to use as the new pixel value. This works by
establishing a rough range for what that pixel value should be by analysing
the surround pixels in conjunction with its own value. However, since noise
is usually an extreme value (high or low), any noise pixels within that 3x3
neighbourhood won't fall within the range, and therefore will be excluded
from the final average used to calculate the new pixel value. In the event
that none of the neighbouring pixels fall within the 30 point range, the value
of the pixel will remain unchanged. This reduces the noise much better than
a simple weighted average of the 3x3 neighbourhood.
9
CHAPTER 4
RESULTS AND DISCUSSION
4.1
Experimental Result
We completed our project as much as we had originally expected. The
convolution program works correctly and can convolute any image using
any blur kernel. The deblurring program can also deblur that image, as long
as the aforementioned blur kernel is an acceptable motion blur kernel, as
mentioned in my project development. After that using the Gaussian
smoothing program noise of the deblurred image is reduced. A successful
run is outlined below. The results shown below are the zoomed version of
the actual image.
The code for bluring the image is:
import cv2
import numpy as np
from matplotlib import pyplot as plt
img = cv2.imread('new1.pgm')
blur = cv2.blur(img,(5,5))
plt.subplot(121),plt.imshow(img)
plt.subplot(122),plt.imshow(blur)
cv2.imwrite('new2.pgm',blur)
plt.show()
Result :
FIG NO.2 First the image is blurred using convolution with a known kernel.
10
The code for deblurring the image is :
import cv2
import numpy as np
from matplotlib import pyplot as plt
img = cv2.imread('new2.pgm')
kernel = np.ones((5,5),np.float32)/25
dst = cv2.filter2D(img,-1,kernel)
plt.subplot(121),plt.imshow(img)
plt.subplot(122),plt.imshow(dst)
cv2.imwrite('new3.pgm',dst)
plt.show()
Result:
FIG NO.3 Then it is deblurred using deconvolution with that known kernel.
The code for noise reduction process is :
import numpy as np
import cv2
from matplotlib import pyplot as plt
img = cv2.imread('new3.pgm')
dst = cv2.fastNlMeansDenoisingColored(img,None,10,10,7,21)
plt.subplot(121),plt.imshow(img)
plt.subplot(122),plt.imshow(dst)
cv2.imwrite('new4.pgm',dst)
plt.show()
Result:
FIG NO.4 Then using Gaussian smoothing algorithm denoising is performed.
11
4.2
System Information
OS Name:
OS Manufacturer:
Processor:
Installed Memory (RAM):
System Type:
4.3
Windows 10 Home Single Language
Microsoft Corporation
Intel(R) Core™ i5-5200U CPU @ 2.20GHz
8.00 GB(7.89 GB usable)
64-bit Operating System, x64-based processor
Limitations
Even though we have deemed our project successful within the goals we
originally set, there are obviously limitations. First and foremost, our
deconvolution algorithm only works with some kernels that fall within a
certain type of blur kernel. Anything outside that category and our program
will just return unreadable noise. Another limitation is simply that our
program needs the known blur kernel to deblur the image; it doesn't have
the ability to estimate the kernel. This is a severe limitation as it can only
be used to deconvolute artificially blurred images where the kernel is
known, it has no real world applicability yet. Another limitation of our
project is the vast difference in colour/intensity between the original image
and the final, corrected image. In theory and ideally these two images will
be identical, however, it can be seen from the figure below that they are
very dissimilar in intensity.
FIG NO.5 The deblurred and noise reduced image
(right) and the original image (left).
12
This also affects the contrast of the image. We speculate that this happens
because of the multiple transforms the image undergoes throughout the
process (one logarithmic transform for every convolution and
deconvolution). This could potentially pose a problem in machine vision
applications, though we expect only because of the reduced contrast, not
the change in colour/intensity, which could possibly be adjusted using
another, linear contrast stretching transform.
13
Chapter 5
SUMMARY AND CONCLUSION
5.1
Summary
The project has offered an effective noise reducing procedure. To achieve
this first we blurred an image then deblurred the image and then reduced
the noise of the deblurred image to get the final result.
5.2
Conclusion
During the project we came to know that there are various ways to reduce
noise in images like image filtering and Gaussian smoothing method and
some other methods. We chose the Gaussian Smoothing Algorithm to do
the job and after using it we can conclude that though it has a few
limitations of its own but still it is a very effective method to reduce noise
in an image.
5.3
Scope and Future Work
The scope of this project is rather narrow, but important. It pertains only to
blurry images, and in this case only those blurred using a motion blur
kernel, but the concept is a rather large problem in the worlds of image
processing, photography, and machine vision. In photography, blurry
images are undesirable because they lack sharpness or clarity and in
machine vision, blurriness can make an image indecipherable by the
computer or render certain processes ineffective, such as edge detection.
These results can help explore the worlds of image deconvolution and noise
reduction, within the larger world of image correction. There is a lot of
room for future work on this project since and in this field in general. The
next area of research would be into blur kernel and point spread function
14
types so that the deconvolution process can be made more adaptive and
generalized, since my program is very restrictive and specific. The largest
area for future research for this project is the one that is most applicable to
the real world and also the subject of much study in the computer science
community. This is the topic of blind deconvolution, which estimates the
blur kernel from an image in which it is not known (e.g. a naturally blurred
image acquired through poor image capture) and then deconvolutes the
image based on this estimate. Research is ongoing on trying to find the
most efficient and adaptive method of estimating the blur kernel. Lastly, it
would be beneficial to look into colour or contrast correction algorithms in
an attempt to try and match the output images colour intensity and contrast
with the original image so that the output is as accurate as possible.
15
REFERENCES
[1] Adelmann, H. G. (1999, March). An edge-sensitive noise reduction
algorithm for image processing. Computers in Biology and Medicine,
29(2),137-145.
[2] Brayer, J. M. (n.d.). Introduction to the Fourier transform. In Topics in
human and computer vision. Retrieved from University of New Mexico
Department of Computer
Science website:http://www.cs.unm.edu/ brayer/vision/fourier.html
[3] Convolution and deconvolution using the FFT. (1992). In Numerical
recipes in C: The art of scienti_c computing (pp. 538-545). Cambridge,
Massachusetts:
Cambridge University Press.
[4] Fergus, R., Singh, B., Hertzmann, A., Roweis, S. T., & Freeman, W. T.
(2006,
July). Removing camera shake from a single photograph. ACM
Transactions on
Graphics, 25(3), 787-794. doi:10.1145/-
[5] Fisher, R., Perkins, S., Walker, A., & Wolfart, E.(2000, October).
Hypermedia Image Processing Resource (HIPR2) [Image processing
learning resource].
Retrieved from http://homepages.inf.ed.ac.uk/rbf/HIPR2/wksheets.htm
[6] Smith, S. W. (1997). A closer look at image convolution. In The
scientist and engineer's guide to digital signal processing (pp. 418-422).
Retrieved from http://www.dspguide.com/
[7] Young, I. T., Gerbrands, J. J., van Vliet, L. J. (n.d.). Properties of
Fourier transforms. In Image processing fundamentals. Retrieved from
http://www.ph.tn.tudelft.nl/Courses/FIP/noframes/_p-Properti2.html
16