Advances in Engineering Research, volume 207

Proceedings of the 2nd International Seminar of Science and Applied Technology (ISSAT 2021)

Image Processing Implementation on the Field

Programmable Gate Array
Dianthika Puteri Andini1,* Gunawan Sugiarta1 Feni Isdaryani1 Fadhel Muhammad
Hafidh1
1
Department of Electrical Engineering, Bandung State Polytechnic, Bandung, Indonesia
*
Corresponding author. Email: dianthika@polban.ac.id

ABSTRACT
Image processing is an essential principle that is increasingly being applied in innumerable applications to optimize
abundant design complexities to modified version by implementing the hardware platform to practice the algorithms.
This is especially for the practice of a real-time system that performing frames of the image processing directly.
Currently, the FPGA technology becomes a viable target for implementing suitable image processing applications. Real-
time image processing can be implemented into the FPGA platform by using VHDL. FPGA can be the requirement of
real-time imaging applications through maintaining data and computational. This paper focuses on software simulation
and hardware implementation of image processing algorithms for the use in an FPGA based on the real-time processing.
It aims to verify the real-time performance and functionality of the proposed image processing algorithm using the
FPGA board from Altera named Cyclone IV EP4CE10E22C8. It is compatible to implement the process in a real-time
system with the Histogram Equalization proposed method. This paper shows design exploration to the real-time
environment in the practice of implementing image processing algorithm in the specified FPGA board, especially in
customized image processing simulation of Histogram Equalization, application of proposed algorithm into image
processing FPGA board, and exploration of practice in FPGA board of compilation and downloading.

Keywords: Image Processing, Histogram Equalization, FPGA, Altera Cyclone.

1. INTRODUCTION Speed Integrated Circuit (VHSIC); VHSIC Hardware

Design Language (VHDL) [1][2].
The growth of developing applications using image
processing then becomes the image processing Many image processing applications are essential that
algorithms plays essential principle that is increasingly various operations be performed on each pixel in the
being applied in innumerable applications. The image resulting in an even large number of operations per
processing algorithms try to optimize abundant design second. One option is to apply an FPGA because frequent
complexities to modified version implementing in growth in the size and functionality of FPGAs over recent
hardware platform to practice the algorithms. This is years has resulted in an increasing interest in the
especially for the practice of a real-time system that application as implementation platforms for image
performing frames of the image processing directly. processing, particularly real-time video processing
Conversion and transformation of the algorithm for [3][4][5]. FPGA could be structured to separate the image
efficiency take part in the role of practice performance. and distribute the resulting sections to multiple pipelines
This performance can be applied on the hardware as which could process data concurrently. Practically,
processor called Field Programmable Gate Array parallelization is the main idea to the processing mode
(FPGA). Currently, FPGA technology becomes a viable and hardware constraints of the system. This forces the
target for implementing suitable image processing designer to solve hardware issues such as concurrency,
applications. Real-time image processing can be pipelining, and priming, which many image processing
implemented into the FPGA platform by using Verilog experts are unfamiliar with. The different image
Hardware Description Language (VHDL); Very High- processing algorithms for RGB to grayscale, an

Copyright © 2021 The Authors. Published by Atlantis Press International B.V.

This is an open access article distributed under the CC BY-NC 4.0 license -http://creativecommons.org/licenses/by-nc/4.0/. 505
 Advances in Engineering Research, volume 207

algorithm for image negatives, and image enhancement A real-time system response to a specific time
are available to be implemented in a system [6][7]. meaning in image processing is regularly capturing
images, analyzing images to obtain data, and controlling
To overcome this problem, the general approach of
several activities. This algorithm can be applied in vision
the language is to hide low-level details by compiling the
systems for path planning where the timing is a critical
parallelism optimization automatically. Compiling and
point of the parameter. Delay is a quiet criterion
downloading is the result of an FPGA configuration file.
depending on the result. Then, the execution time is
Software-to-hardware conversion schemes can be considered as a performance impact for the results. One
employed in MATLAB to VHDL which results can be way to support this approach is developed by image
implemented to hardware. MATLAB can manipulate processing algorithm in real-time enhancement image
matrices in general and is perfect for some image processing algorithm. A histogram is one of many
processing applications. In MATLAB, an image will be methods to support the image processing approach [8-
represented in the form of a matrix that allows a designer 12].
to develop optimal matrix operations for the application
A histogram is a distribution of the number of pixels
of algorithms. Whereas if the algorithm simulated with
according to their intensities. In this case, it is to
the board implementation is different, it will cause a
scrutinize the image to regulate this distribution. The
slightly slow process so synchronization between
various intensity images are the input that can be
software and hardware is required.
controlled and the perceived results of the normalized
This paper focuses on software simulation and histogram and corresponding level of output images. The
hardware implementation of image processing part of equalization shows the enhance of dark image
algorithms for the use in an FPGA based on real-time contrast that the histogram is already distributed on all
processing. Regarding the programming aspect, level. The histogram is the basis for various spatial
MATLAB can handle images as a matrix which allows a domain processing techniques. Histogram manipulation
better understanding of the processing. The can be used for image enhancement. In addition, to
implementation in using MATLAB is far from being the provide beneficial image statistics, the information
fastest to generate HDL programs. The HDL Coder tool attached to the histogram is also entirely beneficial in
of MATLAB can make designers be focused on system- other image processing applications, such as image
level design instead of struggling in specific compression and segmentation. Histograms are
programming. MATLAB/Simulink can interpret an straightforward to calculate in software as well as
environment of model-based design to shorten the time hardware implementations, making them an accepted
and meet the end performance approach. approach for real-time image processing [13][14][15].
To verify the real-time performance and functionality Histogram Equalization produces a transformation
of the proposed image processing algorithm, FPGA function that generates output images with uniform
board from Altera named Cyclone IV EP4CE10E22C8 histograms. When automatic upgrades are desired, this
that is compatible to implement the process in a real-time way of approach can be considered because the results of
system is used. The Cyclone board presents some specific this technique are predictable, and these methods are
signals to display images with the signals. effortless to implement. However, there are applications
where histogram equalization does not match. It is
2. LITERATURE REVIEW sometimes useful to be able to determine the histogram
shape that matches with the image processed. The
2.1. Image Processing Algorithm method used to create images that have a specific
histogram is called histogram matching or histogram
The project of image processing is to develop some specification [16][17][18].
image processing functions based on a proposed
Histogram equalization is a preliminary process for
algorithm called Histogram Equalization. The histogram
image processing and enhancement with an
of a digital image is a classification of its discrete
implementation on hardware as the focus [19][20]. The
intensity levels in the range. Histogram equalization is a
adjustment of the criteria of histogram equalization and
method to process images to adjust the contrast of an
intensities can be better applied in the image
image by modifying the intensity distribution of the
improvement than the manual adjustment. First, the input
histogram. The objective of this technique is to set out a
will be calculated to histogram from the image pixels.
linear tendency to the cumulative probability function
Then, the next process is to compute the Cumulative
associated with the image. The processing of histogram
Distribution Function (CDF) based on the created
equalization relies on the utilization of the cumulative
histogram. The histogram is computed as a set of bins by
probability function which is a cumulative sum of all the
each tile. Finally, the last step is scaling and mapping the
probabilities lying in its domain.
image pixels and creating equalized output.

506
 Advances in Engineering Research, volume 207

2.2. FPGA Board design philosophy for the rapid new design creation and
easy integration of custom value-added features.
Over the last 20 years, FPGA has transformed from
glue logic to computing platforms. It functionally comes 3. RESEARCH METHODOLOGY
up with a reconfigurable hardware platform for
performing logic and algorithms. Being fine-grained The algorithm method for image processing in this
hardware, FPGA is capable to draw on the parallelism research is in two intermediate levels. The histogram is
inherent within a hardware design while simultaneously utilized as a representation in another version of pixel
keep going the reconfigurability and programmability of conversion. This method can be shown in the higher level
software. FPGA is being practiced as a platform for that is high-level. It means that using an image as
accelerating computationally intensive tasks. This is representation in the final information. The operation of
particularly depicted in the field of image processing, intermediate and high levels is a corresponding to
where the FPGA-based acceleration of imaging decrease parallelism by changing the pixel that brings
algorithms has become mainstream. This is even more about a reduction of total data and processing time.
within an embedded environment where the power and
Histogram equalization is used to improve contrast in
computational resources of conventional processors are
images by computing histograms that correspond to the
not up to the task of managing the data throughout and
section of images and distribute the luminance values of
computational requirements of real-time imaging
the images. It is suitable for improving local contrast and
applications.
strengthening the definitions of edges in each region of
In contrast, it is compulsory in FPGA to design not an image.
only the algorithm but also the computational
MATLAB/Simulink can transform the model-based
architecture, which leads to an explosion in the design
design to a familiar hardware approach by using HDL
space complexity. This way, integrated with the
code generation automatically. It also verifies and
complexities of managing the concurrency of a highly
optimizes the model based on design requirements.
parallel design and the bandwidth issues associated with
Therefore, the model-based design effectively works as a
the high volume of data associated with images and
rapid tool for system validation and testing. HDL coder
video, has led to a wide range of approaches and
allows image processing algorithm to FPGA
architectures used for realizing FPGA-based image
implementation as illustrated in Figure 1.
processing systems [21][22][23].
Specifications [24]:
1. Size: 10cm × 6.5cm.
2. Main chip: ALTERA FPGA Cyclone IV
EP4CE10E22C8.
3. Extended Interface: 2*22 pin interface, 2*28 pin
interface, 2.54mm pitch pins.
4. Power supply: USB power supply interface, used for
power supply.
5. Program debug and download interface ： 2*5 pin
JTAG interface, used for download program.
6. Onboard a power LED, 2 user LEDs.
7. Onboard 1 reset key, 2 user keys.
8. Onboard large capacity serial configuration chip
EPCS16 or M25P16, save the program when power
is off.
9. NIOS or large capacity cache, using SDRAM module Figure 1 Model-based design
to expand.
MATLAB Vision HDL Toolbox™ Histogram library
The Video and Image Processing Suite is a collection implements an approach of histogram equalization for
of intellectual property (IP) cores that can be applied to image processing algorithm. A simulation can be run in
facilitate the development of custom video and image the model of Simulink part as software development. The
processing algorithms. The cores range from simple model provides a hardware-compatible algorithm to
building-block functions, such as color-space conversion enhance the contrast of images by applying this proposed
to sophisticated video-scaling functions that can algorithm. Video Source reads the multimedia file.
implement programmable polyphase scaling. Along with Figure 2 shows the block system that is the combination
an easily integrated connectivity portfolio, the Video and of software and hardware design.
Image Processing Suite presents a comprehensive system

507
 Advances in Engineering Research, volume 207

4. CONCLUSION

4.1 Result
The domain tasks are solved by software and
hardware implementation as image processing in the first
step and FPGA board is continuously in execution.
Histogram equalization is an enhancement process that
Figure 2 Algorithm design involves in improving the quality of the image.
The main part of histogram equalization is depicted
a. Pixel control is a video partition that divides large in Figure 3, containing Video Source, Video Partition,
images into four non-overlapping small images for HDLHistogram, Equalization, and Video Display.
parallel histogram computing.
b. The histogram is HDLHistogram that calculates the
accumulated histogram of an image.
c. Equalization implements the histogram equalization
technique in the original image and produces an
image that has been increased in contrast.
d. Pixel In and Out are input and output of the system
Video Partition, the histogram is enumerated for the
complete image and this operation grasps a considerable
total of time because each of the regions of interest (ROI)
of the image part is processed. The video partition
component in this example separates large images into
four small images that do not overlap. The histogram is
computed on four small images simultaneously. Each Figure 3 Block of Histogram Equalization [25]
small image is connected to the Frame To Pixels block to
The simulation depicted in Figure 4 shows the
produce a pixel stream and corresponding control signal.
difference between the original image (left) and the result
HDLHistogram is optimized for HDL code (right) performing enhancement of contrast image.
generation. Histograms of pixels are calculated by using
the Vision HDL Toolbox Histogram. Since the input
image is grayscale with the uint8 data type, the input
pixels are grouped into 256 bins. The model scrutinizes
the bin histogram figuring sequentially. The bin value is
sent for cumulative histogram calculation. After 256 bin
values are interpreted, the model generates a big reset to
reset all bins to zero. Histograms are collected from each
small image then next calculating the histogram
accumulation of large images.
Equalization can be appealed to frames where
accumulated histograms are being counted or frames
afterward. If it is applied to the current frame, the input
video needs to be stored. The results of the video that has
been done in equalization then be compared to the Figure 4 Result of Histogram Equalization
original video. Simulation is the first operation software procedure
Input can be taken from the source of the camera with with direct remaining hardware which is implemented on
a specific parameter for the simulation. Consequently, the FPGA board. The image processing realizes on the
the output would be shown in the VGA monitor as an platform by corresponding function leading to the
interface. resources. The model requires to be maintained for less
memory and fewer elements to be classified by reducing
the computation time. The FPGA gives modest
acceleration over software implementation on a high-end
computer. Before implementation to the board, HDL
code generation should be applied to create an easier
program than writing it from zero. HDL code generation

508
 Advances in Engineering Research, volume 207

can be applied with Simulink as in Figure 5 or using device, and board settings of the program setting. Family
MATLAB command. of Cyclone IV E dan EP4CE6E22C8 should be chosen as
shown in Figure 8. Next, in EDA Tool Setting, ‘Tool
makehdl(HistogramEqualizationHDL/HDLHistogram)
Name’ and ‘Format Simulation’ should be changed to
makehdltb(HistogramEqualizationHDL/HDLHistogram) ‘ModelSim-Altera’ and ‘VHDL’ completed with the
finish button figured in Figure 9. The top-level entity of
the project can be the same as the VHDL code displaying
in Figure 10 that is a flow summary of compilation and
downloaded VHDL file to FPGA board.

Figure 5 Simulink HDL code generator

All processes are generated to the folder that can be
synchronized into the path in computer system completed
HDL TestBench generation displays in Figure 6 proved
with “*.vhd” file extensions are successfully categorized
as VHDL file into a folder. Every single file and
parameter set is generated by MATLAB and HDL Coder
into VHDL programming to execute in the FPGA
objective. Figure 7 depicts the file that has been created
by converting the Simulink model. In that case, the
VHDL file has been successfully created.

Figure 8 Board setting wizard

Figure 6 MATLAB HDL generator

Figure 9 Tool settings

Figure 7 File VHDL on a computer system

System implementation on the FPGA board
establishes with compilation in the design tool which is Figure 10 Practice to FPGA board
Intel Quartus Prime. Lite Edition. To upload the program,
This second method is utilized by using the camera as
the system must run New Project Wizard that family,
input and output to VGA monitor implemented on FPGA

509
 Advances in Engineering Research, volume 207

board using VHDL language. First, the software program

for FPGA implementation should be synthesized to
complete validation as shown in Figure 11. The next
process is to upload the program to FPGA proven by
“100% Successful” on the software as shown in Figure
12.

Figure 13 Running FPGA program

Figure 11 Synthezise program

Figure 14 FPGA board implementation

Figure 12 Uploading program to FPGA 4.2 Conclusion

The result of the camera interface to the VGA monitor
This paper attempts to show design exploration to the
can be seen in Figure 13 with the specification of LCD 7
real-time environment in the practice of implementing
inch, 1024x600. The camera has an active array size of
image processing algorithms in the specified FPGA
640x480 with the output formats 8 bit, RGB colour, and
board. Adapting high-level language can be implemented
30 fps for VGA. Then, the FPGA board implementation
in both software and hardware design by the mapping
can be seen in Figure 14. The new entity for camera
process. For software operation, the requirements and
configuration by creating a .vhd file is in the same
specifications differ little by functionality based on
directory. This entity is also used to capture images from
software design and hardware application algorithm.
camera or real-time mode to video. Therefore, RAM
Reformulating algorithms to successfully implement the
configuration is also set up by specifying memory size as
design should be applied by the designer. Subsequently.
several bits and change the total of bits memory. The final
MATLAB algorithm design is implemented to VHDL
step is the VGA configuration aims to give the signal to
using HDL Coder and continuously applied to the FPGA
the VGA monitor. The RGB code is the useful
board. FPGA board code generator is simulated in
information because of the specification of the camera
Simulink and compiled portion calculated in actual
interface. The compilation is done to assign the node
FPGA hardware While verifying functional correctness
name and location of the pin in the FPGA board.
of the hardware. Simulation of design can be seen in the
Connecting the FPGA and VGA monitor is done by using
Simulink of MATLAB that equalized produced in the
the specific cable and seeing the result in the monitor.
output. Another method is to implement a real-time
camera as input and a VGA monitor as output.
The algorithm suits well in the implementation by
using Altera Cyclone IV EP4CE10E22C8. The process

510
 Advances in Engineering Research, volume 207

entirely depicts as the practice beginning from the block processing. Nonlinear Dynamics, 82(4), pp.1879-
of approached image processing algorithm remaining 1892.
into compilation report of VHDL implementation on the
[6] Guragain DP, Ghimire P, Budhathoki K.
FPGA board. Testing of the algorithm by using the FPGA
Implementation of FPGA Based Image Processing
board can be completed to the next action to verify and
validate through the finalized system. Moreover, Algorithm Using Xilinx System Generator.
generated VHDL code can be easily verified and mapped International Research Journal of Engineering and
into FPGA by allowing the rapid prototyping of design. Technology (IRJET). 2018 Jan;5(01):2395-0056.
[7] Siddiqui F, Amiri S, Minhas UI, Deng T, Woods R,
AUTHORS’ CONTRIBUTIONS Rafferty K, Crookes D. Fpga-based processor
acceleration for image processing applications.
This paper conceived the presented practice to
Journal of Imaging. 2019 Jan;5(1):16.
develop and perform the procedure in writing aspect to
the manuscript for the proposed discussion. Specifically [8] Yadav G, Maheshwari S, Agarwal A. Contrast
exchange of views, limited adaptive histogram equalization based
 Customized image processing simulation of enhancement for real time video system. In2014
Histogram Equalization. International Conference on Advances in
 Application of proposed algorithm into image Computing, Communications and Informatics
processing FPGA board. (ICACCI) 2014 Sep 24 (pp. 2392-2397). IEEE.
 Exploration of practice in FPGA board of [9] Ishii I, Tatebe T, Gu Q, Takaki T. Color-histogram-
compilation and downloading. based tracking at 2000 fps. Journal of Electronic
 Attempt method of real-time image processing Imaging. 2012 Mar;21(1):013010.
method into FPGA board implementation.
 Figure the camera and monitor as input and output [10] Padilla DA, Villaverde JF, Magdaraog JJ, Oconer
section of the system process. AJ, Ranjo JP. Vehicle and Weather Detection Using
Real Time Image Processing Using Optical Flow
ACKNOWLEDGMENTS and Color Histogram. In2019 5th International
Conference on Control, Automation and Robotics
First and foremost, this paper expresses gratitude for (ICCAR) 2019 Apr 19 (pp. 880-883). IEEE.
DIPA Politeknik Negeri Bandung, Bandung State
Polytechnic, Bandung, Indonesia for the financial [11] Mehta M, Goyal C, Srivastava MC, Jain RC. Real
support to complete the research that can be a part of time object detection and tracking: Histogram
laboratory development in practice learning. matching and kalman filter approach. In2010 The
2nd International Conference on Computer and
REFERENCES Automation Engineering (ICCAE) 2010 Feb 26
(Vol. 5, pp. 796-801). IEEE.
[1] Chou, C., Mohanakrishnan, S., Evans, J.: “FPGA
[12] Jasmine J, Annadurai S. Real time video image
Implementation of Digital Filters,” Proc. ICSPAT,
enhancement approach using particle swarm
1993.
optimisation technique with adaptive cumulative
[2] Benedetti, A., Perona, P.: “Real-time 2-D Feature distribution function based histogram equalization.
Detection on a Reconfigurable Computer,” Measurement. 2019 Oct 1;145:833-40.
Proceedings of the 1998 IEEE Conference on
[13] Mathworks, Inc.: “MATLAB 5.3 Fact Sheet,”
Computer Vision and Pattern Recognition, 1998.
Natick, MA, 1999.
[3] Johnston, C. T., K. T. Gribbon, and D. G. Bailey.
[14] Mehra R, Verma R. Area efficient FPGA
"Implementing image processing algorithms on
implementation of sobel edge detector for image
FPGAs." In Proceedings of the Eleventh
processing applications. International Journal of
Electronics New Zealand Conference, ENZCon’04,
Computer Applications. 2012 Jan 1;56(16).
pp. 118-123. 2004.
[15] Raut NP, Gokhale AV. FPGA implementation for
[4] Bailey D.G. Design for embedded image processing
image processing algorithms using xilinx system
on FPGAs. John Wiley & Sons; 2011 Jun 13.
generator. IOSR Journal of VLSI and Signal
[5] Tlelo-Cuautle, E., Carbajal-Gomez, V.H., Obeso- Processing (IOSR-JVSP). 2013 May;2(4):26-36.
Rodelo, P.J., Rangel-Magdaleno, J.J. and Nunez-
[16] Gonzalez, R. C., and R. E. Wood. "Digital image
Perez, J.C., 2015. FPGA realization of a chaotic
processing, 3rd edtn." (2007).
communication system applied to image

511
 Advances in Engineering Research, volume 207

[17] Bailey DG. The advantages and limitations of high

level synthesis for FPGA based image processing.
InProceedings of the 9th International Conference
on Distributed Smart Cameras 2015 Sep 8 (pp. 134-
139).
[18] Stewart R, Duncan K, Michaelson G, Garcia P,
Bhowmik D, Wallace A. RIPL: A Parallel Image
processing language for FPGAs. ACM
Transactions on Reconfigurable Technology and
Systems (TRETS). 2018 Mar 14;11(1):1-24.
[19] Siddiqui F, Amiri S, Minhas UI, Deng T, Woods R,
Rafferty K, Crookes D. Fpga-based processor
acceleration for image processing applications.
Journal of Imaging. 2019 Jan;5(1):16.
[20] Woods, R. E., Eddins, S. L., & Gonzalez, R. C.
"Digital image processing using MATLAB."
(2009).
[21] Memon, F., Jameel, F., Arif, M. and Memon, F.A.
Model Based FPGA Design of Histogram
Equalization. Sindh University Research Journal-
SURJ (Science Series), 2016 Jun 14;48(2).
[22] Bailey, Donald G. "Image processing using
FPGAs." (2019): 53.
[23] HajiRassouliha A, Taberner AJ, Nash MP, Nielsen
PM. Suitability of recent hardware accelerators
(DSPs, FPGAs, and GPUs) for computer vision and
image processing algorithms. Signal Processing:
Image Communication. 2018 Oct 1;68:101-19.
[24] Altera. Intel Literature Cyclone IV. Cyclone IV
Device Handbook, Volume 1. Altera Corporation.
Marcg 2016.
[25] Mathwork Vision HDL Histogram.
[Online]. Available:
https://www.mathworks.com/help/visionhdl/ug/his
togram-equalization.html

512