Computer Vision Using Fuzzy Logic

for Robot Manipulator

Lab. Implementation

Adriano Breunig1 , Haroldo R. de Azevedo2 , Edilberto P. Teixeira2 , and

Elmo B. de Faria3
1
Escola Técnica Federal de Mato Grosso
Depto. de Informática, Cuiabá, 78.005-390/MT, Brasil
abreunig@uol.com.br
2
Universidade Federal de Uberlândia, Depto. de Engenharia Elétrica
Uberlândia, 38.400-902/MG, Brasil
{haroldo,edilbert}@ufu.br
3
Instituto Luterano de Ensino Superior de Itumbiara, Depto. de Informática
Itumbiara, 75.523-200/GO, Brasil
ebfaria@ufu.br

Abstract. This paper presents a Laboratory implementation of a com-

puter vision using fuzzy logic techniques for the detection of boundaries
in images obtained through a camera installed in a robotic arm. Here,
these images were captured, digitized and analysed using fuzzy algo-
rithms, in order to used for the control of the robotic arm carrying the
camera. The image treatment process uses the method proposed by Pal
and Majumder (1987), which employs image enhancement techniques
through the use of fuzzy set concepts. The image recognition process
generates the control signal necessary to move the robotic arm in a given
specific pathway, as well as to determine the next action to be taken at
the end of the task.

1 Introduction
The artificial Vision System tecnology is increasingly becoming more important.
Its aplications may be found in several areas, such as in industries, in medicine, in
robots, etc. In the case of robots the vision is indispensable. The vision systems
for use with robots should be designed following to basic criterion (Groover,
Weiss, Nagel and Odrey, 1989):
1st - Relatively Low Cost;
2nd - Relatively Fast Response Time.
Usually there are two methods for the implementation of vision systems. One
takes the image of the object and tries to reduce it to contour lines that form the
profile of that object. This method utilises filters to obtain the image information
and uses contrast enhancement to turn all parts of the image into either black
or white (thresholding).The formed image is usually called a binary image.

W. Ziarko and Y. Yao (Eds.): RSCTC 2000, LNAI 2005, pp. 229–237, 2001.
c Springer-Verlag Berlin Heidelberg 2001


230 A. Breunig et al.

The advantage of binary images is that they supply well defined limits, easily
recognized by simple algorithms. This type of implementation is used by systems
for the processing of two dimensional images.
The other method for the implementation of vision systems tries to give
to the computer an image closer to what humans perceive. This method gives
information about the brightness of the image to the computer. This allows the
computer to obtain two important image caracteristics, which are not possible
to obtain with the technique of high image contrast: surfaces and shadows. This
can be used to obtain three dimensional image information and to solve conflicts
when one object partially blocks an other. This method is usually used for three
dimensional vision systems.
In this work we shall use the first method, as we will have a controlled working
environment, utilizing images with suficient clear shapes.

2 Vision System

The images is obtained with a camera connected straight to the parallel port
of a computer. The image is digitized with a resolution of 64 × 64 pixels and
256 levels of grey. The image will be analyzed through the techniques of Image
Enhancement and Histogram, using for this purpose, Fuzzy Algorithms. The
image analysis will generate a control signal for the movement of the six joints
robotic arm Fig. 1.

Fig. 1. General scheme of the robot system

3 Pre-processing

The analogical video signal obtained with the camera is sampled and quantized
in order to be suitable for computer processing.
 Computer Vision Using Fuzzy Logic for Robot Manipulator 231

3.1 Image Representation - Matrix Form

A monocromatic image may be described as a mathematical function f(x,y) of

the light intensity, where its value at a point (x, y) is proportional to the level
of grey at that point.
This function f(x,y), Eq. 1, is represented by a M × N matrix with “l” levels
of grey, each one representing a pixel.

3.2 Image Representation - Fuzzy Set Theory

A image F, where F = f(x, y) (Eq. 2), with dimensions M × N and “l” levels of
grey may be considered as a matrix of singletons fuzzy, each one with an f(x,y)
value that indicates the relative brightness value of grey level “l”, where l =
0, 1, 2, 3...L − 1. Applying the fuzzy set theory we may write (Pal and Majumder,
1987):
 
f (0, 0) f (0, 1) · · · f (0, N − 1)
 f (1, 0) f (1, 1) · · · f (1, N − 1) 
 
f (x, y) =  .. .. .. ..  (1)
 . . . . 
f (M − 1, 0) f (M − 1, 1) · · · f (M − 1, N − 1)
 
m11 /x11 m12 /x12 · · · m1n /x1n · · · m1N /x1N
 m21 /x21 m22 /x22 · · · m2n /x2n · · · m2N /x1N 
 
 .. .. .. .. .. .. 
 . . . . . . 
 
F =  mm1 /xm1 mm2 /xm2 · · · mmn /xmn · · · mmM /xmN  (2)
 
 .. .. .. .. .. .. 
 . . . . . . 
 
.. ..
mM 1 /xM 1 mM 2 /xM 2 . mM n /xM n . mM N /xM N

or, in union form (Eq. 2):


x= µmn /xmn , m = 1, 2, ...M and n = 1, 2, ...N
m n

where µmn /xmn , (0 ≤ µmn ≤ 1) represents the grade of possessing some property
µmn by the (m,n)th pixel xmn . This fuzzy property µmn may be defined in a
number of ways with respect to any brightness level depending on the problem
at hand.

3.3 Digitizing

The signal acquired through the camera must be sampled and quantized in order
to be suitable for computer processing. An increase in the valuess of M and N in
(2) means a better image resolution. Quantization rounds up the values of each
pixel and places, them in the range of 0 a 2n − 1, where the larger the value of
n the greater the number of grey levels present in the digitized image.
232 A. Breunig et al.

The digitized signal is a converted analogic to digital signal, in which the

number of samples per unit of time gives the sampling rate, and the number of
bits of the analogical/digital converter determines the number of levels of grey.
It should be decided which values of M, N and l are suitable, talking into
account image quality and the number of bits necessary for storage. We may
calcule the quantity of bytes utilized by means of the expression: N × M × l/8
(Marques and Vieira, 1999). It should be remember that image quality is a
subjective concept and it is strougally dependent on the aplication itself.

4 Image Treatment

The captured image is made up of two parts: the object to be identified and the
background. In order to separate these two parts it is necessary to have a good
contrast, that is, a significant difference in light intensity between points of the
object and point of the background must exist.
Image treatment is based on the principal of thresholding (Gonzalez. e
Woods, 1992), that is, to assign the value 0 for pixels above a given value X
(determined by the Histogram analises), and a value 1 for pixels below or at
most equal to X. The image enhancement algorithm utilized in this work was
suggested by Pal e Majumder (1987), and show in Fig. 2. This technique mod-
ifics the pixels using the properties of the fuzzy set. This procedure involves a
preliminary image enhancement in block “E”, followed by a smoothing in block
“S” and then a further enhancement. The fuzzy operator INT (contrast inten-
sification) is used in both enhancements. The pupose of the image smoothing is
to blur the image before the second enhancement.

Fig. 2. Block diagram of the enhancement model

The algorithm used in the image enhancement (Fig. 2) is explained in the

pages that follow.

4.1 Histogram

The histogram gives na idea of the image quality in terms of contrast and relative
proportions of white and black in the image.
The histogram of an image is usually a grafic representation of bars, which
indicates the l levels of greys and the quantity of pixels in the image. The vertical
axis represents the amountly of pixels for a given l levels of grey. The horizontal
axis represents the l levels of grey.
 Computer Vision Using Fuzzy Logic for Robot Manipulator 233

Here, the histogram is used to give na inicial idea of the image quality for
future comparison with the image obtained after blocks “E” and “S”. These will
be explained in sub-section 4.2 and 4.3.

4.2 Image Enhancement through the Concepts of Fuzzy Sets

Image enhancement is acomplished through the INT operator. The operation

of contrast intensification (INT) upon a fuzzy set A criates another fuzzy set
A = INT(A), where the membership function, is given by:

2[µA (x)]2 , 0 ≤ µA (x) ≤ 0.5 (3a)
µA
(x) = µINT(A) (x) =
[1 − 2(1 − µA (x))]2 , 0.5 ≤ µA (x) ≤ 1 (3b)

This operation reduces the fuzziness of a set A by increasing the values of

µA (x) which are above 0.5 and decreasing those which are below it. Let us now
define operation (3) by transformation T1 of the membership function µ(x) (Pal
e King, 1981). In general, each µmn in F (Eq. 2) may be modified to µmn to
enhance the image F in the property domain by a transformation function Tr ,
where:
 
Tr (µmn ), 0 ≤ µmn ≤ 0.5 (4a)
µmn = Tr (µmn ) =
Tr (µmn ), 0.5 ≤ µmn ≤ 1 (4b)

and, r = 1, 2, ....
The transformation function Tr is defined as successive applications of T1 by
the recursive relationship:

Ts (µmn ) = T1 {Ts−1 (µmn )}, s = 1, 2, · · · (5)

and T1 (µmn ) represents the operator INT defined in Eq.3. This is shown that, as
r increases, the curve tends to be steeper because of the successive application
of INT. In the limiting case, as r → ∞, Tr produces a two-level (binary) image.

4.3 Smoothing Algorithm - Averaging

This method is based on averaging the intensities within neighbors and is usually
used to remove “pepper and salt” noise. The smoothed (Block “S” - Fig. 2)
(m,n)th pixel intensity is:

xmn = 1/N1 xij , (i, j) = (m, n), (i, j) ∈ Q1 (6)
Q1

The higher the values of Q1 the greater is the degree of blurring.

The smoothing algorithm described above blur the image by attenuating
the high spatial frequency components associated with edges and other abrupt
changes in grey levels.
234 A. Breunig et al.

5 Hardware System

In this section, the hardware system, developed to implement the control strategy
presented in this paper, is described. There are six DC motors to drive the joints
and six potentiometers to measure the joint angles.
The driving circuit includes high frequency transistors, switched by pulse
width (PWM). The end effector is actuated using a pneumatic transmitter. The
three base joints are free to turn up to 270◦ , and the upper three joints can only
rotate 180◦ .
To test the strategies presented in this paper, a specific circuit had to be
developed, since the manufacturer’s driving circuit does not allow for most of
the requirements of the approach developed in this research project. Therefore,
only the original mechanism, the motors, and the potentiometers are used in this
work. All the electronic circuits and software programs have been developed and
implemented. Three MicrochipTM PIC micro controllers are used (Microchip,
1997).
They have been chosen because they are cheap and relatively ease to imple-
ment. They use RISC technology of 14 bits. Its possible to deal with a large
number of interruption schemes and to generate PWM signals independently of
the CPU synchronization command.
Figure 1 presents a general scheme of the robot system used in this work.
Basically, the system consists of two hierarchical levels. In the highest level there
is a Pentium computer working as a host. In the second level, there are three
micro controllers of the type PIC16C73A.

6 The Fuzzy Controller

The use of a fuzzy controller for trajectory tracking of such a robot arm is very
appropriate, considering the nonlinear behavior of the system. There are some
propositions for the application of fuzzy logic to robot control (Luh, 1983). The
main reason for the application of fuzzy control for this kind of application is, ba-
sically, the nonlinear characteristic of robots. There are many other approaches
for controlling robot arms (Bonitz, 1996, Moudgal, Passino, & Yurkovich, 1995,
Rocco, 1996). Some of them demand a lot of computer effort. In some cases,
where very fast response is required, the computational burden could be pro-
hibiting. The use of fuzzy logic in such a case is very attractive since it requires
a minimum of computer time. On the other hand, the powerful of fuzzy logic
is such that, the coupling effects of the joints can be completely compensated
using the appropriate fuzzy rules. The approach presented in this paper differs
in some extend to the methods found in the literature (Young, & Shiah, 1997,
Kawasaki, Bito, & Kanzaki, 1996, Cox, E., 1994) since there is a combination
of individual fuzzy controllers for each joint and a master fuzzy controller used
to generate the set points. Therefore, the couplings found in the general robot
dynamic equation are completely compensated. There are also two fuzzy con-
trollers for each joint: one for speed control and another for position control.
 Computer Vision Using Fuzzy Logic for Robot Manipulator 235

They work in such a way that, speed controller actuates in the first part of the
trajectory. At the end of it, the position controller takes place leading to a very
smooth positioning. The decision for the switching of all those controllers is also
accomplished by the master fuzzy controller. A simplified scheme is presented
in Fig.3. In Fig. 3, Q and •Q are the desired position and speed of the joint,
respectively. The position controller uses the error and error variation as input.
On the other hand, the inputs for the speed controller are the speed error and
error variation. Each fuzzy controller has the following characteristics:
• Gaussian membership functions
• Defuzzification using the center of area
• The use of simple product as the minimum operation
• Concentration operator for the output function.
The universe of discourse of position for each one of the base joints is [−270◦ to
270◦ ] and for the upper joints is [−180◦ to 180◦ ]. The universe of discourse of
the output, for each joint is [−192◦ to 192◦ ], corresponding to the PWM input
of the motor drives. The rule base was defined using cardinality equal to 7, for
every controller, although smaller rule basis could also be tested with excellent
results. The rules were implemented based on the experience acquired from the
observation of the robot behavior.

Fig. 3. Control system for join 1

7 Practical Results
The result of the enhancement method (Fig. 2) is shown in Figures 5 a 10.
The results shown in figures 5 to 10, were obtained directly from the camera
acquisition system. The Fig. 5 and 8 are the original image and histogram of
236 A. Breunig et al.

Fig. 5. Original Image Fig. 6. Image after first enhance

Fig. 7. Image after smoothing Fig. 8. Image after second enhance

the original image. In the next step the original image is Enhance, smoothed
Fig. 5(b) and them a second enhancement Fig. 5(c). Finally the binary image
Fig. 5(c) and 5(e) is obtained and applied in the control system of the robot
manipulator.
The results shown in figures 5(a) to 5(f), were obtained directly from the
camera acquisition system. The figures 6 and 9 are the original image and his-
togram of the original image. In the next step the original image is Enhance
figure 6, smoothed figure 7 and them a second enhancement figure 8. Finally the
binary image figures 8 and 10 is obtained and applied in the control system of
the robot manipulator.
The results shown in figures 11 to 16, were obtained directly from the robot
acquisition system. In the test, each joint started from the 50◦ . The desired final
angle, for each joint is showed as a continuous line, in figure. Some very small
overshoots could be seen.
But they could be very well compensated with a more careful adjustment
of the rule base. The switching from the speed to the position controller is ac-
complished by the master fuzzy controller. Figure 17 explains how the switching
idone.

8 Conclusions

The results obtained shown that the method applied is effective for controlled
environment. The model Fig.3, shown good possibilities for improvement since
 Computer Vision Using Fuzzy Logic for Robot Manipulator 237

the method used in block “S”, may be altered, by means of, for instance a max-
mim filter, and in this way improve the results sent to the second block “E”.
The study also shows that although the averaging filter had been used, with
only two interactions in each block “E”, we have obtained a binary image.
The results obtained for image enhancement using fuzzy logic, show us effec-
tive, very speed and of the simple implementation.

References
1. Bonitz, R. G., and Hsia, T. C., February (1996). Internal Force-Based Impedance
Control for Cooperating Manipulators, IEEE Transactions on Robotics and Au-
tomation, vol. 12, no. 1.
2. Cox, E., (1994).The Fuzzy Systems Handbook: A practitioner’s Guide to Building,
Using, and Maintaining Fuzzy Systems, Academic Press.
3. Gonzalez, R. C. and Woods, R. E. (1992). Digital Image Processing, Addison-
Wesley Publishing Company.
4. Kawasaki, H., Bito, T. and Kanzaki, K., June (1996). An Efficient Algorithm for
the Model-Based Adaptive Control of Robotic Manipulators, IEEE Transactions
on Robotics and Automation, vol. 12, no. 3.
5. Luh, J. Y. S. (1983). Conventional controller design for industrial robots - A tuto-
rial. IEEE Trans. Syst. Man. Cyber.,vol. SMC-13, n◦ 3,Mai.
6. Microchip Technology Incorporated (1997). PIC16C7X Data Sheet, DS30390E,
USA.
7. Moudgal, V. G., Kwong, W. A., Passino, K. M. and Yurkovich, S., May (1995).
Fuzzy Learning Control for a Flexible-Link Robot, IEEE Transactions Fuzzy Sys-
tems, vol. 3, no. 2.
8. Pal, Sankar K. and Majumder, Dwijesh K. Dutta. (1987). Fuzzy Mathematical
Approach to Pattern Recognition, John Wiley & Sons.
9. Rocco, P., August (1996). Stability of PID Control for Industrial Robot Arms,
IEEE Transactions on Robotics and Automation, vol. 12, no. 4.
10. Young, K. Y., & Shiah, S. J., November (1997). An Approach to Enlarge Learning
Space Coverage for Robot Learning Control, IEEE Transactions on Fuzzy Systems,
vol. 5, no. 4.