See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/304409626

Grading system of tomato grafting machine based on machine vision

Conference Paper · October 2015

DOI: 10.1109/CISP.2015.7407950

CITATIONS READS
4 81

5 authors, including:

Wang Ruili
Shenyang Agricultural University
10 PUBLICATIONS   22 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

straw returning View project

All content following this page was uploaded by Wang Ruili on 24 June 2021.

The user has requested enhancement of the downloaded file.

 2015 8th International Congress on Image and Signal Processing (CISP 2015)

Grading system of tomato grafting machine

based on machine vision

Xiurong Zhao Zifan Wang, Siyao Liu, Ruili Wang*, Subo Tian*
College of Mechanical and Vehicle Engineering College of Engineering
Shenyang Institue of Technology Shenyang Agricultural University
Fushun, China Shenyang, China
* Corresponding Author

Abstract—Grafting technique helps to reduce soil-borne diseases, to reduce soil-borne diseases, overcome obstacles cropping
overcome the obstacles of cropping cultivation, increase yield and cultivation, improve fertilizer use efficiency, enhance
fruit quality. But grafting requires precise technique; the resistance, increase yield and fruit quality, and is currently
proficiency of operation will also directly affect the productivity wide to use in agricultural production[1]. However, grafting
and the survival rate of grafted seedlings. With the wide requires precise technique, the proficiency of operation will
application of grafting technology, grafting robots were also directly affect the survival rate of grafted seedlings. The
developed continually. However, the current grafting robots majority of grafting machines is semi-automatic, so they
could not realize the production automation completely, and it is usually finish grafting operation under artificial grading
necessary for them to grade seedlings by hand and match
seedling, and it is difficult to realize full-automatic grafting.
different diameters of the rootstock and scion, and it is urgent to
realize grading automatically before grafting. So, the machine Grafting Robot have been developed during the past twenty
vision system of the grafting robot will be designed to grade years,and the machine vision system of grafting robot have
grafting seedling automatically to improve grading accuracy, been studied recently.Yang Haijun studied the automated
reduce labor and improve the quality and speed of grafting quality inspection and grading methods of the grafted
effectively.Firstly, the hardware structure of the grading system vegetables[2]. Gu song et al. gave examples of grafting
was designed based on machine vision, which includes selection machine developed in Japan and analyzed its theory and its
of Japanese CIS industrial cameras, whose type is UXGA, with a current situation[3]. Qiu Yizhi et al. developed several grafting
resolution of 1600 × 1200, a frame rate of 20fps, 1/1.8CCD, and
robot system in Taiwan[4]. Ashraf et al. developed a tomato
8mm lens and so on. Then using proper lighting resource, the
algorithm of the image acquisition and segmentation about
seedling classification algorithm based on machine vision.
tomato grafting classification was analysed using the VC++ Zhang Tiezhong et al. presented the method of testing the
software and OpenCV tools. Taking the tomato seedlings for the growing point of melon flesh seedling and southern melon
research object, the statistical methods that gray value seedling based on machine vision recognition and classification
accumulation of pixels horizontally extracting stem diameter of research on robots grafted seedlings[5-6]. Liu chengliang
seedlings features were used in this paper. Thus the proper joint proposed a design idea to replace manual labor with a robot
point was detected and determined according to the middle point vision system[7]. Liu kai designed a visual software program
of straight and the smooth part of pixels’ curve. Finally, the based on the research of machine vision recognition of the
grading system software was designed and debugged, and 100 grafted seedlings[8]. Zhang Tiezhong and Gong Zhihua
tomato seedlings were acquired and processed using the visual developed a grafting machine automatic positioning system
grading system for tomato seedling and classified by A, B, C, D individually with the help of machine vision and image
levels, the results show that the system classification success rate processing technology,and then the automatic detection of
of 96%, because of irregular blade barrier stalks or stalks mechanical device was designed and then was developed a set
suddenly bend, the four seedlings grading failure. The results of automatic detection algorithm, the single-chip
also show the grading system was designed reasonably and microcomputer control system is developed at the same time[9-
performed reliably. 10]. Li Zhongqiu conducted the research on vegetable grafting
rootstocks automatic positioning and cutting machine systems
Keywords-Machine vision; Grafting Machine; Grading,
Tomato; Seedling
based on Machine Vision. He developed an automatic
detection algorithm and program, and designed the
corresponding device[11]. Qiu shuo carried out a study of
I. INTRODUCTION healthy seedlings recognition based on image processing
Grafting is a technique whereby branches or buds from one technology[12].
plant are inserted into appropriate parts of another plant which Tomato grafting adopts different types of tomato rootstock
have strong affinity. In the process of development, the branch and scions seedling. The average size of rootstock and scion is
or bud become a plant of independent by absorbing nutrients. about 2~5mm. Enlarging the joint area of scion and stock is the
The branch or bud that grafted is called the scion and the key of grafting survive. Therefore before grafting, choosing the
grafted plants is called the rootstock. Grafting technique helps

978-1-4673-9098-9/15/$31.00 ©2015 IEEE 604

basic equal plant size stock and scion is an efficient method. capture the images of objects, obtain standard parts side-length
We usually adopted artificial classification method in the past. a'(unit: pixel), while its actual side-length is a (unit: mm),then
But because of the fatigue resistance and limitations of people, the system calibration coefficient k is
long time tedious work could not assure the grafted seedlings a
to be divided into the right level accurately. It not only affects k= （1）
the quality of grafting but also grafting survival rate. So before a＇
grafting, the grafting robot need to grade grafting seedling For accurate calibration, multiple calibration should be
automatically to improve classification accuracy, reduce labor done in different orientations of the camera field of the view,
and improve the quality and speed of grafting effectively. and then the average value should be the final calibration
Therefore, we use the machine vision to replace the "eye" to coefficient k, which can reduce a distortion and the random
identify grafting seedling, and then use the image processing to errors.
grade the grafting seedling automatically in this paper. Taking 2.1.2 Selection of lens
tomato seedlings as the research object, the seedlings will be The role of the lens is similar to the human eye lens, the
graded by the stem diameter of the cutting point before grafting quality of lens will directly affect the performance of the
with the help of image processing and machine vision whole system. For the whole machine vision system, the lens
technology. The machine vision system has great significance
is an important image acquisition part, so it should be a
for improving the level of automation and mechanization of
vegetable grafting robot. reasonable choice to install the lens. Seedlings classification
and detection parameters are the features of stem-diameter.
Thus, each pixel of the image for precise measurement of the
II. PRINCIPLES AND METHODS diameter of the seedlings is important. In the study, according
to the position of object and camera, 8mm lens was selected
2.1 Hardware design of machine-vision grading system
and the proper distance between the lens and the object is
A typical machine vision system generally includes camera, 120mm. Depending on the size and the principle of the
lens, light source, the image acquisition card, image processing imaging, the distance relationships between the lens and the
platform, and control execution parts, etc. Based on the object were shown in Fig. 2.
working conditions, inspection object, performance, and cost
requirements, the appropriate grading machine vision system
was designed in this paper, which is shown in Fig. 1.

6.9 mm
L

Fig. 2 The relationship of the focal length, the actual distance and the size of
the object

As shown in the Fig.2, we can calculate the actual length

L of the object, and the total pixel in this direction is 1600, so
Fig. 1 The composition of the machine visual hardware system each pixel point represent the actual length is 0.065mm.
Calculations are as follows:
2.1.1 Selection of the camera ⎛ 6.9 ⎞
Camera in the machine vision system plays a crucial role: θ = arctan ⎜⎜ 2 ⎟ = 23.33° （2）
8 ⎟
it is a kind of photoelectric converter that converts optical ⎝ ⎠
signals into electrical signals and outputs in a certain format. L = 2 × 120 × tan θ = 103.5 mm （3）
The current imaging devices are mainly the Charge Coupled 103.5
Device (CCD) and Complementary Metal Oxide l pixel = = 0.065 mm （4）
Semiconductor (CMOS). This study selected CCD camera, 1600
which has better image quality [13]. Because the actual length of each pixel has been
In this study, we chose the Japanese CIS industrial calculated, the actual size of the object can be gotten . If we
camera, the screen is UXGA, with a resolution of 1600 × can obtain the pixels of stem-diameter, we will easily get
1200(pixels × pixels), a frame rate of 20fps, 1/1.8CCD and the tomato actual stem diameter of grafting seedling.
image area of 6.9 × 5.2 (mm × mm). 2.1.3 Selection of light source
When selecting a monocular camera to inspect and grade In the visual system, we usually use the light source and
the tomato seedlings, a simple and fast calibration method the controller matching the lighting to stand out the
should be used. The system takes a standard square-shape characteristic of object, remove the influence of environment
object as a calibration reference and adopts an experiment and acquire a steady image. The way of lighting commonly
calibration method. After determining the working distance include back light and front lighting. According to the
between the measured object and the detection system, we purpose—the detection of stem diameter, we select back light,

605
front lighting and both back light and front lighting, then carry normally recognize the seedling stem, then these seedlings are
out contrast experiments. With the help of Win Roof software listed as "D" and that is the "unqualified".
and other tools, we adjust the different light source and 2.2.3 Image binaryzation processing
acquire different images of the same object, and get the gray
Image binaryzation is one of the most common way
value curve to contrast.According to the principle of
to gray-scale transformation, which can make the visual effect
maximum contrast difference of gray image, the appropriate
of the whole image display only black and white. In addition,
light source will be selected in the paper. When we use the
it can increase the speed of image processing, and takes
back light to take images, compare with other two cases, the
resources as less as possible. At present, common binaryzation
gray value curve is very steep. We can conclude that the back
algorithm contain threshold limit binaryzation algorithm and
light is the best chosen, so we make use of back light to take
adaptive threshold binaryzation algorithms. In this study, the
pictures and conduct the further research and experiment [14].
seedlings of graphic information, relative to the background, is
2.1.4 Computer and Image capture card a clear outline and uniform gray value area, so we choose the
The computer system is the guarantee of normal threshold limit binaryzation algorithm. The algorithm is
operation of image processing software, which can be used to relatively simple, according to the image grey value range,
storage grafting seedlings images, process and analyze the choosing a grey value as a threshold T. The image is divided
stem characteristics and grade seedling [15-16]. The computer into two parts, then the grey value g (i, j) of any point (i, j) in
system should configure with the memory of 256M, the unit each image will be compared with T, if it is greater than or
frequency of 2.93GHz, hard software of 40G and the graphics equal to T, this grey value will be set to 1; otherwise, if it is
card of 64M. less than T, it is set to 0, (cf. Equation (5)):
The function of imagine capture card is to take digital ⎧1, g (i, j ) ≥ T （5）
video image information, storage and running, which is the f (i, j ) = ⎨
⎩0, g (i, j ) < T
interface of the image acquisition and processing parts. Many
where g (i, j) denotes the grey value of original image, f (i, j)
camera with image capture card don’t need another one, such
denotes the grey value of binary image
as the FireWire Camera and Gigabit Ethernet Camera.
In this study, the image is processed in binaryzation, the
2.2 Software design of machine-vision grading system grey value of stem characteristic is 1, the rest is 0, as shown in
Fig.3.
2.2.1 Selection of software
The key technology of this system is computer image
processing technology [14]. The ultimate goal of the study is to
achieve sorted seedlings by automatic grafting machine, so
Microsoft Visual c++ is used as the development software,
which is more flexible and wide function, and match with
OpenCV that is a professional computer vision library and
other tools such as Matlab and Winroof, so it is easier to
realize the development of grading algorithm and program. Fig. 3 Binary image
2.2.2 Grading criterion of grafting seedling
2.2.4 Image characteristics extracting algorithm
Seedling shape is characterized by its phyllotaxis,but
This experiment adopts the method of grey accumulation
these phyllotaxis do not provide any useful quantitative
by all horizontal pixels to extract the diameter characteristics
information enough to pattern recognition.Because the shape
of seedlings stem. Image is a two-dimensional signal, and all
of seedling have even changed a lot in the same species of
pixels form a matrix. The value of each pixel in the horizontal
seedlings. The parameters that can identify dimensions of the
direction adds, that is to say pixel value in the same horizontal
plant body are inter-nodal length, stem diameter and leaf
lines will plus together. Since the value of each pixel in the
arrangement, which always differ from one seedling to
image is 1 or 0, the accumulation is to calculate the sum of the
another. Usually, each of seedlings has to be measured by the
number of pixel value 1, then we capture the stem-diameter
stem-diameter separately to determine its size and shape,
distribution histogram characteristic of g (y), as shown in
thereby to recognize the required characteristics. So, the main
formula (6):
feature of the seedlings to be graded is the stem-diameter. W −1
Because the diameter is very thin, each pixel of an image is g ( y ) = ∑ f ( x, y ) y = 0,1,2, …, W - 1 （6）
important to measure the exact seedling diameter. x=0

Generally, the stem diameter of grafting seedling is where f (x, y) denotes image pixel values (x, y), W denotes
between 2.7mm and 5.1mm, so we determine that "A" width of image.
represents the stem-diameter range of more than 4.3mm , "B" 2.2.5 Design of software program
represents that of 3.5 mm to 4.3 mm, "C" represents that of This study selects the Windows XP operating
less than 3.5 mm. According to the above grading criterion, if system,development tool with powerful Microsoft Visual
some reasons such as irregular blade shelter stem, we can’t Studio 2008,as well as open source computer design library

606
OpenCV2.0 which is embed and integrated into development light and front lighting, experiment results show that blue back
system formed in the Microsoft Visual Studio 2008. The lighting system was the best lighting resource and was chosen
process of the system running roughly was divided into three for acquiring images of seedlings.
parts:image preprocessing,image segmentation and
3.2 Running result of software program of machine-vision
detection.Image preprocessing include image acquisition and a
grading system
series of processing before binaryzation.Image segmentation is
an important part of the process,and the processed image This program is a collection of images to be processed to
quality directly decides whether the feature extraction and extract characteristic gray values, and the data of seedling
analysis can be successfully done.The purpose of detecting characteristics was obtained and stored as an Excel file in
process is the extraction and analysis of the features. order to provide the necessary data basis for the following
analysis. After completed, program calls the images and
2.3 Experiment design of machine-vision grading system
analyze the data generated Excel form, then use the mapping
The tomato seedlings called "momotaro" were selected as function to draw the curve of accumulated gray value of image
the experiment material. The age of seedlings is about two pixel along the horizontal axis,as shown in Fig.4. Finally we
months. After selecting the device and lighting of machine can conclude that the proper joint point was detected and
vision system of grafting robot, some images of the tomato determined, that is the middle point of straight and smooth
seedlings will be captured and processed for grading. part of pixels’ curve.
In this study, grafting seedling is between camera and
3.3 Experiment results and discussion
back light, and the distance is 120mm. In this experiment, we
will rotate the 90 ° direction for every seedling and capture According to the experimental design, we acquire two
two images in two direction, therefore, we get two different images of each seedling in 90°direction and choose the better
images for each seedling and select the better one to process.
image to be graded, then total 100 images of 100 tomato
III. RESULTS AND DISCUSSIONS grafted seedlings were processed. Finally, classification
results of the 100 seedlings are as shown in Tab.1. From the
3.1 Hardware structure of machine-vision grading system
table, it can be seen that the classification success rate is
The hardware structure of grading system was designed 96%,and 4% of the image classification fails, that is, 4
based on machine vision, which includes selection of Japanese seedlings’ diameter can not be detected. We can find that
CIS industrial cameras, whose type is UXGA, with a failure of classification is mainly due to irregular seedling
resolution of 1600 × 1200, a frame rate of 20fps, 1/1.8CCD, leaf stalk(Fig.5) or stem suddenly bending.
and 8mm lens and so on. After compared with different
lighting resource such as front lighting, back light, both back

200

400

600

800

1000
0 50 100 150 200 250 300 350 400

Fig. 4 The curve of accumulated gray value of the image pixel along the horizontal axis

200

400

600

800

1000
0 50 100 150 200 250 300 350 400

Fig. 5 The seedling image of grading failure

607
 Pixel along the vertical axis

Accumulated gray value of image pixel along the horizontal axis

Fig.6 (a) Original image; (b) Binary image; (c) Accumulated gray value of image pixel along the horizontal axis
Pixel along the vertical axis

Accumulated gray value of image pixel along the horizontal axis

Fig.7 (a) Original image; (b) Binary image; (c) Accumulated gray value of image pixel along the horizontal axis

608
 There are five classification success graft examples in REFERENCES
Fig.6,and two classification failure graft examples in Fig.7. [1] Lee J, Kubota C, Tsao S, et al. Current status of vegetable grafting:
From the tab.1, we can also find that the number of "A" Diffusion, grafting techniques, automation. Science Horticulture, 2010,
grade seedlings is 20, "B" is 62, and “C" is 14, which is the 127(2):93-105.
middle diameter of the seedlings accounted for 62% of the [2] Yang H. Studies on the automatic quality estimation and grading of
grafting seedling. Changsha: Hunan Agricultural University, 2003.
total seedlings, so it is very suitable for mechanical grafting
[3] Gu S, Jiang L. Development of domestic and foreign vegetable grafting
seedlings. robot. Journal of Northeast Agricultural University, 2007, 38(6):847-851.
Tab. 1 The grading result
[4] Chiu Y, Chen S, Chang Y. Development of a circular grafting robotic
The success rate The failure rate system for watermelon seedlings. Applied Engineering in Agriculture,
Grade A B C D of grading of grading 2010, 24(1): 1077-1084.
(100%) (4%) [5] Zhang T, Wei J. Study on vision system of vegetable grafting robot (I):
Searching for growth point of cucurbit seedling with morphological
method. Journal of China Agricultural University, 1999, 4(4):45-47.
Number 20 62 14 4 96 4
[6] Zhang T, Wei J. Study on vision system of vegetable grafting robot (Ⅱ):
Searching for growth point of pumpkin seedling with geometrical
IV. CONCLUSIONS method. Journal of China Agricultural University, 1999, 4(4):48-50.
[7] Liu C, Wang H, Cao Q, et al. Study on robot vision and judgment
Based on a review of domestic and foreign application system for seedling grafting robot. Transactions of the Chinese Society
status of grafting robot and machine vision in agriculture,the for Agricultural Machinery, 2001, 32(4):38-41.
hardware of the machine vision grading system were selected [8] Liu K, Gu S. The research on identification of seedling based on
and developed, including the selection of camera, lens, blue machine vision. Journal of Agricultural Mechanization Research,
2009(11):46-48.
back light, and the image information acquisition processing
[9] Zhang T, Sun G, Gong Z. Study on automatic orientation system of
card,etc. Thus a hardware platform of the tomato grafting stock in vegetable automation drafting based on image processing
machine vision grading system was built. Then by using technology. Journal of Weifang University, 2003, 3(6):34-36.
VC++ programming language and computer vision library [10] Gong Z. Study on automatic orientation system of stock in vegetable
Open CV the eigenvalue algorithm of stem diameter of grafted automation grafting based on image processing technology. Beijing:
tomato seedlings was determined. On this basis, the machine China Agricultural University, 2003.
vision software of tomato seedling grading was designed, and [11] Li Z. Studies on automatic orientation cutting system of stock in
vegetable automation grafting based on machine vision. Changsha:
experiment study were carried out. By the hardware platform Hunan Agricultural University, 2007.
of machine vision grading system and grading software, 100 [12] Qiu S. Research on methodology to identify seeding healthy condition
tomato seedling images were acquired, processed and graded, based on image processing technology. Changchun: Jilin University,
then the success rate of the grading system for tomato 2009.
mechanized seedling reaches 96%, which shows the grading [13] Anna Fabija ´nska, Dominik Sankowski. Computer vision system for
high temperature measurements of surface properties. Machine Vision
system is designed reasonably and performed reliably. and Applications, 2009, 20:411-421.
ACKNOWLEDGMENT [14] Tian S, Ashraf M A, Kondo N, et al. Optimization of Machine Vision
for Tomato Grafting Robot. Sensor Letters, 2013, 11(6/7):1190–1194.
This work was financially supported by the National [15] Chandra Sekhar Nandi, Bipan Tudu, and Chiranjib Koley,et al.A
Natural Science Foundation of China (31101076), Program for Machine Vision-Based Maturity Prediction System for Sorting of
Liaoning Excellent Talents in University(LJQ 2012061) and Harvested Mangoes.IEEE Transactions on instrumentation and
Measurement,2014,63(7):1724-1727.
Scholars of Mount Tianzhu in Shenyang Agricultural
University(2014). [16] Brian L. DeCost, Elizabeth A. Holm. A computer vision approach for
automated analysis and classification of microstructural image data.
Computational Materials Science, 2015, 110:126-133.

609
View publication stats