［ Solution・Supporting Service ］

Detection of Falling Objects in Taiki Yamamoto

Tracks by Using Deep Learning※1

Keywords Railroad, Detection of falling object, Deep learning

If tools, bolts, and and other objects remain on railroad tracks after the com-
Abstract pletion of maintenance and improvement work, there is a serious threat that a
dangerous accident can occur. In order to eliminate accidents caused by such
fall objects, checking work for objects is carried out immediately after the work
ends. For automating this checking work, we developed a detection technology
for fallen objects on railroad tracks by using an image processing function of a
camera.
For the detection of fallen objects, we adopted the method of deep learning
that is noted for its high performance in resolving prediction problems and its
attention to detail. By the deep learning, the presence of fallen objects displayed
on the screen can be detected. By virtue of this expertise, the identification
accuracy of fallen objects and debris is raised to 99.3%. If this technology is
adopted for the dedicated inspection vehicle that is used for the checkout work
for finding fallen objects and debris on the tracks after the maintenance and/or
improvement work, it will contribute considerably to the automation of fallen
object detection work for railroad facilities.

nology for automatically detecting fallen objects on

1 Preface
railway tracks. This paper introduces an automatic
Maintenance and improvement work for rail- detection technology for fallen objects on the rail-
road facilities is generally carried out at night after way tracks by using deep learning.
the last train and before the first train of the following
day. If any tools, bolts, or fallen objects used for the
2 Fallen Object Detection Approach
work are carelessly left on the railroad tracks, a
danger for a serious accident like derailment pres- For the detection of fallen objects, an area
ents. For this reason, checking work is carried out where a camera tilted downwards is installed on a
by workers after the completion of night work in vehicle roof for view. Fig. 1 shows an example of an
order to find any objects left behind.
Such checking work may be done by using a
dedicated inspection vehicle. Such an inspection
vehicle is equipped with an obstacle detection bar
under the vehicle floor. If anything comes in contact
with this bar, it is sensed as an obstacle. In many
cases, however, most fallen objects on the track do
not enter the minimum rolling stock gauge and the
obstacle detection bar may fail to detect them. For
the current checkout work, multiple individuals
stand on the track, side-by-side in line and search
Fig. 1 Example of Input Image
for any fallen objects by illuminating their steps with
A normal image is shown on the left and an abnormal image on
a flashlight. Such a work is taxing on workers. the right. A normal image is devised to show ballast, sleepers,
Against this background, the market calls for a tech- and rails only, and the abnormal image shows a fallen object.

MEIDEN REVIEW Series No.181 2021 No.1

55
 128
11
Normal
11 image
29
14 3 14 14
59 5
5 3 3 3
3 3 Abnormal
image
14 14 14
384 384 256 2
29 1024
59 256 4096
128
96
Input image 3
Convolution layer Fully connected layer

Fig. 2 Network Configuration Used for Auto-Detection Techniques

A network configuration is shown where a fully connected layer of AlexNet is made to adapt this approach by fine tuning.

input image. In this approach, an accurate image is lems in the image recognition field. As such, fine
taken, only of the ballast (gravels laid on roads and tuning is believed to be effective. Still more, fine tun-
tracks), sleepers, and rails. An image where any ing offers an advantage featuring efficient learning
fallen object is taken is regarded as an abnormal based on a small amount of learning data. In this
image. In this approach, a technology of deep learn- approach, the fully connected layer consisting of
ing is used only for two classes of recognition: a three layers can be made to learn newly with a vec-
normal image and an abnormal image. Deep learn- tor input of 4096 dimensions obtained from the
ing is one machine learning approach where a AlexNet convolution layer. Fig. 2 shows a network
large-scale neural network having deep structures configuration used for auto-detection techniques.
is employed. Recently, this approach has been rec- Using a network obtained from the aforementioned
ognized in the world for its outstanding performance learning, 2-class recognition of normal image and
in the field of identification and prediction of prob- abnormal image is carried out.
lems.
This approach employs a network structure
3 Accuracy Verification
proposed by Alex et al. ( AlexNet hereafter). Among
many deep learning network structures, the AlexNet We verified an accuracy of this technology
has proven track records in the field of image object through the photo-taking of normal and abnormal
recognition. Since the AlexNet has a network struc- images in the daytime inside the railroad track. The
ture where images are identified into 1000 classes, number of data was 26,562 in total of both normal
the output of the fully connected layer also contains and abnormal images. 87.5% of total data volume
1000 classes. Apart from this, the output of our for both normal and abnormal images was used as
approach involves only two classes: normal image the learning data respectively. Each remainder of
or an abnormal image. For this reason, it is neces- 12.5% was used as evaluation data. Fig. 3 shows a
sary to make fine-tuning beforehand in order to relationship between learning frequencies and iden-
adapt the fully connected layer of the AlexNet to our tification accuracy. Around 300 times of learning fre-
approach. Fine-tuning is a process when a network quencies, the identification accuracy came to con-
finishing learning a certain problem, it is then made vergence. The obtained identification accuracy was
to learn the last part of the weight again so that it 99.7% for the learning data and 99.3% for the eval-
can then adapt it to another problem. The convolu- uation data. Fig. 4 shows an example of successful
tion layer of deep learning can be used as an fallen object detection. Input images are shown on
extractor feature. In addition, it has a characteristic the left side while the right side shows images where
capable of extraction common to a variety of prob- the influence rate on the recognition is visualized.

56 MEIDEN REVIEW Series No.181 2021 No.1

 1.0
4 Postscript
0.9
We introduced a method for detecting fallen
0.8 Learning data Test data objects on railway tracks by using deep learning.
Accuracy

With this technology, we were able to show the pos-

0.7
sibility of automating the work of checking for fall
0.6 objects. In the future, we aim to put it into practical
use by verifying it on an actual vehicle assuming
0.5
actual operation.
0 50 100 150 200 250 300
Learning frequencies (times)
・All product and company names mentioned in this paper are
Relationship between Learning Frequencies and the trademarks and/or service marks of their respective owners.
Fig. 3
Identification Accuracy
The axis of ordinates shows identification accuracy and that of (Note)
abscissas shows learning frequencies. The blue line shows the
result of identification of an image included in learning and the
※1. An approach to let a computer learn so that the computer can
black line indicates the result of identification of an image not
included in learning. execute a human being s tasks such as voice recognition, image
definition, and prediction.

Fig. 4 Example of Successful Fallen Object Detection

An example of images of the detected fallen object is shown.
Input images are shown on the left side while the right side
shows images where the influence rate on recognition is visual-
ized. The influence rate on recognition by deep learning is shown
with grayscale values. The closer to white, the greater the influ-
ence rate.

MEIDEN REVIEW Series No.181 2021 No.1

57