OpenCV and Visual C++ Programming in Image Processing
OpenCV and Visual C++ Programming in Image Processing
Image processing is the process of manipulating image data in order to make it suitable for
computer vision applications or to make it suitable to present it to humans. For example,
changing brightness or contrast is a image processing task which make the image visually
pleasing for humans or suitable for further processing for a certain computer vision
application.
Computer vision which go beyond image processing, helps to obtain relevant information
from images and make decisions based on that information. In other words, computer vision
is making the computer see as humans do. Basic steps for a typical computer vision
application as follows.
1.
Image acquisition
2.
Image manipulation
3.
Obtaining relevant information
4.
Decision making
If you are new to computer vision, you may be wondering where to start. First
you have to understand the basic principles of image processing and computer
vision. Then you have to choose a suitable language to develop your computer
vision application. Some of the most popular methods are using OpenCV with
C/C++, MATLAB and AForge. If you don't really know why you would choose
one over the other, here is my explanation.
MATLAB is the most easiest and the inefficient way to process images and
OpenCV is the most efficient and hardest way to process images. AForge has
qualities in between OpenCV and MATLAB.
OpenCV has become hardest only because there is no proper documentation
and error handling codes. But OpenCV has lots of basic inbuilt image processing
functions so that those who want to learn computer vision can develop their
applications through proper understanding about what they do.
So, I think that it is worthy to learn computer vision with OpenCV. Therefore in
this blog, it is presented basic image processing functions and computer vision
applications with line by line explanations.
Before you run this program, put any image file (MyPic.JPG) into the folder where
your c++ file is.Otherwise you have to change the first argument of imread()
function and give the absolute path to your image file.
You can download this OpenCV visual c++ project from here. (The downloaded
file is a compressed .rar folder. So, you have to extract it using Winrar or other
suitable software)
Mat
img
=
imread(const
string& filename,
int flags=CV_LOAD_IMAGE_COLOR)
Mat is a data structure to store images in a matrix. It is declared
in "opencv2/core/core.hpp" header file.
imread() is a function declared in "opencv2/highgui/highgui.hpp" header
file. It loads an image from a file and stores it in Mat data structure.
o
o
o
o
o
imageimage-
If you are not sure what to do, use CV_LOAD_IMAGE_COLOR as the 2nd parameter of
imread()
function.
To understand image-depth and concept of channels, you should be familiar with theory of
image processing. So, let's discuss little bit of theory of image processing.
Any digital image consists of pixels. Any pixel should have some value. The minimum value
for a pixel is 0 and it represents black.When the value of the pixel is increased, the intensity
of that pixel is also increased. In a computer memory, ). In decimal, it is 255. fixed number
of bits are allocated for every pixel. Say the number of allocated bits per pixel is 8. Then the
maximum number that a pixel can have is 255 (11111111 in binary)
Now what is image-depth? The image-depth means the number of bits allocated for each
pixel. If it is 8, each pixel can have a value between 0 and 255. If it is 4, each pixel can have
a value between 0 to 15 (1111 in binary).
Here is a simple model of a image with image-depth of 8 bits. Each small box represents a
pixel. So, each box may contain a value between 0 to 255.
o
o
o
o
o
Following image is a simple model of a color image. Color image should consist of at least 3
planes; Red, Green and Blue. Any color can be created using a particular combination of
these 3 colors. Any pixel is a combination of three 3 values. (255, 0, 0) represent pure red.
(0, 255, 0) represent pure green. (255, 0, 255) represents pure violate. In the same way,
you can create many color. Image-depth is 24 because each pixel is represented with 8 x 3
bits (8 bits from each channel).
o
o
o
o
o
In the above model, top left pixel is (23, 231, 46). It will be shown as a greenish color
because the green value(231) of that pixel is larger than the red(23) and blue(46) value.
if (img.empty())
If imread() function fails to load the image, 'img' will not be loaded any data.
Therefore 'img.empty()' should return true. It's a good practice to check whether the image
is loaded successfully and if not exit the program. Otherwise your program will crash when
executing imshow()function.
bool Mat::empty()
This function returns true, if Mat::data==NULL or Mat::total() == 0
//system("pause");
If you are using Visual Studio, it's better to uncomment this line because it will pause the
program until user press any key. If we don't uncomment it, the program will exit
immediately so that user will not see the error message.
void namedWindow(const string& winname, int flags = WINDOW_AUTOSIZE);
This function creates a window.
Parameters -
winname - Title of the window. That name will display in the title bar of the
newly created window
o
flags - determine the size of the window. There are two options
Parameters winname - Title of the window. This name is used to identify the window
created by namedWindow() function.
o
mat - hold the image data
o
This program is also very much similar to the previous application. The only
difference is that this program creates a blank image instead of loading an
existing image from a file.
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
///////////////////////
#include "opencv2/highgui/highgui.hpp"
#include <iostream>
using namespace cv;
using namespace std;
int main( int argc, const char** argv )
{
Mat img(500, 1000, CV_8UC3, Scalar(0,0, 100)); //create an image ( 3 channels, 8 bit
image depth, 500 high, 1000 wide, (0, 0, 100) assigned for Blue, Green and Red plane
respectively. )
if (img.empty()) //check whether the image is loaded or not
{
cout << "Error : Image cannot be loaded..!!" << endl;
//system("pause"); //wait for a key press
return -1;
}
namedWindow("MyWindow", CV_WINDOW_AUTOSIZE); //create a window with the
name "MyWindow"
imshow("MyWindow", img); //display the image which is stored in the 'img' in the
"MyWindow" window
waitKey(0); //wait infinite time for a keypress
destroyWindow("MyWindow"); //destroy the window with the name, "MyWindow"
return 0;
}
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
//////////////////////
Before you run this program, put any image file (MyPic.JPG) into the folder where
your c++ file is.Otherwise you have to change the first argument of imread()
function and give the absolute path to your image file.
You can download this OpenCV visual c++ project from here. (The downloaded
file is a compressed .rar folder. So, you have to extract it using Winrar or other
suitable software)
Parameters :
rows - Number of rows in the 2D array ( height of the image
pixels)
o
cols - Number of columns in the 2D array ( width of the image
pixels)
o
type - specify the bit depth, data type and number of channels
the image. I gave CV_8UC3 and it specify 8 bit unsigned integers with
channels. Here are some of possible inputs for this parameter
in
in
of
3
If you want more details about this, please refer to Data Types for Arrays in
the Basics of OpenCV API
o
Summary
In this program, I created a 3 channel image with 500 height and 1000 width. 8
bit unsigned integer is allocated for each pixel in each channel. (8x3 = 24 bits
per each pixel) And each pixel is assigned with (0,0,100) scalar value. That
means 1st channel is all zero, 2nd channel is also all zero and the 3rd channel is
all 100. Therefore we can see a red image as the output of the program.
break;
}
imshow("MyVideo", frame); //show the frame in "MyVideo" window
if(waitKey(30) == 27) //wait for 'esc' key press for 30 ms. If 'esc' key is pressed,
break loop
{
cout << "esc key is pressed by user" << endl;
break;
}
}
return 0;
}
////////////////////////////////////////////////////////////////////////////////////////////////////////////
You can download this OpenCV visual c++ project from here. (The downloaded
file is a compressed .rar folder. So, you have to extract it using Winrar or other
suitable software)
This is how I play a video using my OpenCV program
New OpenCV functions which are not found earlier are explained here
VideoCapture::VideoCapture(const string& filename)
This is one of few constructors available in VideoCapture class. This constructor
open the video file and initializes the VideoCapture object for reading the video
stream from the specified file.
The destructor of this class will deallocated any associated memory with this
object. Therefore you don't need to deallocate memory explicitly in your
program.
bool VideoCapture::IsOpened()
If the previous call to VideoCapture constructor is successful, this method will return true.
Otherwise it will return false.
It is essential to check that whether the VideoCapture initialize successfully. If it is
unsuccessful, program should be exited. Otherwise when you try to read a frame from the
VideoCapture object, your program will crash.
Parameters :
o int propID - This argument specify the property you are going to change. There are many
options for this argument. Some of them are listed here.
CV_CAP_PROP_POS_MSEC - current position of the video in milliseconds
o double value - This is the new value you are going to assign to the property, specified by
the propID
doubleVideoCapture::get(int propId)
This function returns the value of the property which is specified by propId. You should try
to obtain some properties of the video stream in your code. In my code, I have shown you
how to obtain the frame rate (frames per second) of the video, by using
the CV_CAP_PROP_FPS argument.
Parameters o int propID - This argument specify the property you are going to obtain. There are many
options for this argument. Some of them are listed here.
CV_CAP_PROP_POS_MSEC - current position of the video in milliseconds
waitKey(30)
The function waits for 30 milliseconds. If a key was pressed before the specified
time, it returns the ASCII value of the pressed key. If that value is 27 (ASCII value
of 'esc' key is 27), the program will execute inside the if block. If no key is
pressed during that 30ms, the function returns -1 program will continue the
while loop.
VideoCapture::~VideoCapture()
Destructor of VideoCapture object will destroy any associated memory of
that particular object. This destructor will be called implicitly on exit of the main
method of the above program.
Summary
At first, this program captures a video from a file. Then the program enters into
a infinite loop. In that loop, it grabs frames from the captured video sequentially,
decodes it, shows it in a window and waits for 30 milliseconds. If the video file
has no more frames or if the user presses the 'esc' key, the program will break
the infinite loop.
Note:
Using waitKey(int) function is very important because imshow(string&,
MAT) function need time to paint the image in the window and waitKey(int) will
give that necessary time.
///////////////////////////////////////////////////////////////////////////////////////////////////////////////
#include "opencv2/highgui/highgui.hpp"
#include <iostream>
using namespace cv;
using namespace std;
int main(int argc, char* argv[])
{
VideoCapture cap(0); // open the video camera no. 0
if (!cap.isOpened()) // if not success, exit program
{
cout << "Cannot open the video cam" << endl;
return -1;
}
double dWidth = cap.get(CV_CAP_PROP_FRAME_WIDTH); //get the
width of frames of the video
double dHeight = cap.get(CV_CAP_PROP_FRAME_HEIGHT); //get the
height of frames of the video
cout << "Frame size : " << dWidth << " x " << dHeight << endl;
namedWindow("MyVideo",CV_WINDOW_AUTOSIZE); //create a
window called "MyVideo"
while (1)
{
Mat frame;
bool bSuccess = cap.read(frame); // read a new frame from video
if (!bSuccess) //if not success, break loop
{
cout << "Cannot read a frame from video stream" << endl;
break;
}
imshow("MyVideo", frame); //show the frame in "MyVideo" window
if (waitKey(30) == 27) //wait for 'esc' key press for 30ms. If 'esc'
key is pressed, break loop
{
cout << "esc key is pressed by user" << endl;
break;
}
}
return 0;
}
///////////////////////////////////////////////////////////////////////////////////////////////////////////////
You can download this OpenCV visual c++ project from here. (The downloaded
file is a compressed .rar folder. So, you have to extract it using Winrar or other
suitable
software)
This is how I capture myself with my webcam using the OpenCV program
New OpenCV functions which are not found earlier are explained
here
VideoCapture::VideoCapture(int device)
This is one of constructors available in VideoCapture class. This constructor open the
camera indexed by the argument of this constructor and initializes the VideoCapture object
for
reading
the
video
stream
from
the
specified
camera.
Here the '0' means the index of the camera to be used. You can use 1,2,3..
instead of 0, if your computer is attached to more than 1 camera.
The destructor of this class will deallocated any associated memory with this
object. Therefore you don't need to deallocate memory explicitly in your
program.
if (!cap.isOpened())
If the VideoCapture object initialization unsuccessful, the expression inside the 'if
'parentheses will evaluate to true and the statements inside the 'if' block will be executed.
It is a good practice to check this scenario and handle it accordingly because otherwise it
may cause the program to crash.
namespace cv;
namespace std;
image
is
loaded
or
not
{
cout << "ERROR : Image cannot be loaded..!!" << endl;
//system("pause"); //wait
for
a
key
press
return -1;
}
vector<int> compression_params; //vector
compression parameters of the image
that
stores
the
compression_params.push_back(CV_IMWRITE_JPEG_QUALITY); //specify
the compression technique
compression_params.push_back(98); //specify the compression quality
bool bSuccess
=
imwrite("D:/TestImage.jpg",
compression_params); //write the image to file
img,
if ( !bSuccess )
{
cout << "ERROR : Failed to save the image" << endl;
//system("pause"); //wait for a key press
}
namedWindow("MyWindow", CV_WINDOW_AUTOSIZE); //create a window with the
name
"MyWindow"
imshow("MyWindow", img); //display the image which is stored in the 'img' in the
"MyWindow"
window
waitKey(0); //wait
for
a keypress
The above program is very much similar to the program under 'Create a Blank
Image & Display' section in the lesson of Read & Display Image. If you need
further clarifications of any OpenCV functions which are not explained here,
please refer to Read & Display Image lesson.
o
o
params - This is a int vector to which you have to insert some int parameters
specifying the format of the image
JPEG format - You have to
puch_back CV_IMWRITE_JPEG_QUALITY first and then a number between 0 and
100 (higher is the better). If you want the best quality output, use 100. I have
used 98 in the above sample program. But higher the value, it will take longer
time to write the image
PNG
format
- You
have
to
puch_back CV_IMWRITE_PNG_COMPRESSION first and then a number between 0
and 9 (higher is the better compression, but slower).
The image format is chosen depending on the file name extension. Only images
with 8 bit or 16 bit unsigned single channel or 3 channel ( CV_8UC1,
CV_8UC3, CV_8SC1, CV_8SC3, CV_16UC1, CV_16UC3) with 'BGR' channel order,
can be saved. If the depth or channel order of the image is different,
use 'Mat::convertTo()' or 'cvtColor' functions to convert the image to
supporting
format
before
using imwritefunction.
The above program is very much similar to the program under 'Create a Blank
Image & Display' section in the lesson of Read & Display Image. If you need
further clarifications of any OpenCV functions which are not explained here,
please
refer
to Read
&
Display
Image lesson.
Summary
This program creates an yellowish image ( 3 channels, 16 bit image depth, 650
high, 600 wide, (0, 50,000, 50,000) assigned for BGR channels). Because the
image has 16 bit depth, you can use values between 0 and 65535 for each
element in each channel. I have used 50,000 for each element in the green and
red planes which gives the yellowish color. You can try different values.
Then it specifies the compressing technique. I have used JPEG as the
compression technique in the above example. Then it saves the image in
the "D:/TestImage.jpg" location. Then it displays the newly created image in a window and
wait indefinitely until an user presses a key.
In the following example, a video is captured from the webcam and written into
a file. Let's see how to do it with OpenCV.
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
///////
#include "opencv2/highgui/highgui.hpp"
#include <iostream>
using namespace cv;
New OpenCV functions which are not found earlier are explained here
Size
frameSize(static_cast<int>(dWidth), static_cast<int>(dHeight))
Create a Size object with a given width and height. Here I have cast the width
and height to integers because they are originally double values and
the Size constructor does not accept double values as its parameters.
VideoWriter::VideoWriter(const string&
filename, int fourcc, double fps, Size frameSize, bool isColor=true)
This is the constructor of the VideoWriter class. It initializes the object with
following parameters
o
const string& filename - Specify the name and the location of the
output file. The video stream is written into this file
o
int fourcc - specify the 4 character code for the codec which is
used to compress the video. Your computer may not be supported some codecs.
So, if you fail to save the video, please try other codecs. Here are some popular
codecs.
double fps - frames per seconds of the video stream. I have used
20. You can try different values. But the codec should support the fps value. So,
use an appropriate value.
o
Size frameSize - Size object which specify the width and the
height of each frame of the video stream.
o
bool isColor - If you want to save a color video, pass the value as
true. Otherwise false. Remember codec should support whatever value, you
pass. In the above example, you have to give true as the 5th argument.
Otherwise it will not work.
o
if ( !oVideoWriter.isOpened() )
Check whether the VideoWriter object initialize successfully. If not exit the
program immediately.
void write(const Mat& image)
Write a frame to the video stream. The size of the frame should be same as the
size you specified when initializing the VideoWriter object.
All other OpenCV functions have been discussed in earlier lessons. So, if you are
not familiar with those OpenCV functions yet, please go through Capture Video
from File or Camera
Filtering Images
Image filtering is an important part of computer vision. For most of computer
vision applications, filtering should be done before anything else. OpenCV
supports lots of in-build filtering methods for images. Here is the list of
filtering methods that I am going to discuss with you in the following
posts (with OpenCV 2.4.5 and C++ )
Here is the list of image filtering methods which are explained using examples
with OpenCV 2.1 in C style (not C++)
Eroding
Dilating
Inverting
Here is the original image which I am going to filter using above methods.
Original Image
If you have not install and configure OpenCV yet, please refer to Installing &
Configuring with Visual Studio.
Eroding
Eroding is a simple way of filtering images. Here is how it can be done with
OpenCV.
///////////////////////////////////////////////////////////////////////////////////////
#include "stdafx.h"
#include <cv.h>
#include <highgui.h>
int main()
{
//display the original image
cvDestroyWindow("MyWindow");
cvDestroyWindow("Eroded");
cvReleaseImage(&img);
return 0;
}
///////////////////////////////////////////////////////////////////////////////////////
You can download this OpenCV visual c++ project from here.
Eroded Image
New OpenCV functions which are not found earlier are explained
here
cvErode(img, img, 0, 2)
The 1st parameter is the source image.
The 2nd parameter is the destination image which is to be the eroded image.
Here the 3rd parameter is the structuring element used for erosion. If it is 0,
a 33 rectangular structuring element is used.
The 4th parameter is the number of times, erosion is applied.
This function can process images in place. That means same variable can be
used for the 1st and 2nd parameters.
If you want more explanation about various methods in the above computer
application , please refer to Capturing Images & Videos.
Dilating
#include <cv.h>
#include <highgui.h>
int main()
{
//display the original image
IplImage* img = cvLoadImage("C:/MyPic.jpg");
cvNamedWindow("MyWindow");
cvShowImage("MyWindow", img);
//dilate and display the dilated image
cvDilate(img, img, 0, 2);
cvNamedWindow("Dilated");
cvShowImage("Dilated", img);
cvWaitKey(0);
//cleaning up
cvDestroyWindow("MyWindow");
cvDestroyWindow("Dilated");
cvReleaseImage(&img);
return 0;
}
///////////////////////////////////////////////////////////////////////////////////////
You can download this OpenCV visual c++ project from here.
Dilated Image
New OpenCV functions which are not found earlier are explained
here
cvDilate(img, img, 0, 2)
The 1st parameter is the source image.
The 2nd parameter is the destination image which is to be the dilated image.
Here the 3rd parameter is the structuring element used for dilation. If it is 0,
a 33 rectangular structuring element is used.
The 4th parameter is the number of times, dilation is applied.
This function can process images in place. That means same variable can be
used for the 1st and 2nd parameters.
Inverting
{
//display the original image
IplImage* img = cvLoadImage("C:/MyPic.jpg");
cvNamedWindow("MyWindow");
cvShowImage("MyWindow", img);
//invert and display the inverted image
cvNot(img, img);
cvNamedWindow("Inverted");
cvShowImage("Inverted", img);
cvWaitKey(0);
//cleaning up
cvDestroyWindow("MyWindow");
cvDestroyWindow("Inverted");
cvReleaseImage(&img);
return 0;
}
///////////////////////////////////////////////////////////////////////////////////////
You can download this OpenCV visual c++ project from here.
Inverted Image
New OpenCV functions which are not found earlier are explained
here
cvNot(img, img)
This function inverts every bit in every element of the image in the 1st
parameter and places the result in the image in the 2nd parameter.
This function can process images in place. That means same variable can be
used for the 1st and 2nd parameters.
e.g - For a 8 bit image, the value 0 will be mapped to (255-0)=255
the value 46 will be mapped to (255-46)=209
For a 16 bit image, the value 0 will be mapped to (65535-0)=65535
the value 46 will be mapped to (65535-46)=65489