Volume 6, Issue 4, April – 2021 International Journal of Innovative Science and Research Technology

ISSN No:-2456-2165

Navigation in Crowded Spaces Using Trajectory

Prediction
Raghav Utpat Sameer Joshi
Pune Institute of Computer Technology Pune Institute of Computer Technology

Seema Chandak Shaunak Joshi

Prof., Pune Institute of Computer Technology Pune Institute of Computer Technology

Shubham Loya
Pune Institute of Computer Technology

Abstract:- Navigation in cluttered and crowded unique approaches for each sub-statement to achieve overall
environments has been an important and difficult optimality. Predicted trajectories of the identified
problem in technology. This involves accurately pedestrians are used to calculate their next positions, which
predicting pedestrians’ movements, dynamically are then used to model the surroundings and plan an
analysing developments in the surroundings, and optimum path while avoiding all obstacles.
adjusting the path accordingly. This paper focuses on
solving the navigation problem by predicting the II. SCOPE
trajectories of pedestrians. Humans are identified and
tracked using state-of-the-art object detection ● Identifying pedestrians as obstacles and tracking them
techniques. R-CNN and YOLO are proven to have the across the environment using visual input.
best accuracy and speed to perform the task. We used ● Implementing state of the art trajectory prediction
both social and non-social algorithms to predict algorithm to calculate future trajectories of pedestrians
trajectories of the detected pedestrians. These with respect to their neighbours.
trajectories are used to estimate future positions of the ● Developing a client-side software to generate a map of
pedestrians. Finally these positions are used to calculate the environment using predicted positions of all
the path through the environment. pedestrians.
● Implementing an algorithm to path-find through a
Keywords:- Navigation, Realtime, Trajectory Prediction, cluttered map based on prior environment information,
Deep Learning, Object Detection, Path planning, Long obstacle position and goal by selecting an optimal
Short Term Memory Network (LSTM), Region based navigation strategy.
Convolutional Network (R-CNN) ● This system can be extended to various housekeeping
and hospitality applications, small scale payload delivery
I. INTRODUCTION bots.

Automation of menial human tasks has been a constant III. RELATED WORK
driving force in the history of technology. Humans’ ability
to navigate their surroundings is a key element of a lot of Akansha Bathija[6] discusses and proposes a state of
these tasks. Navigation in a crowded environment involves the art approach to solve the object detection model which is
detecting obstacles, modelling the environment, localizing more suitable for real time tracking and detection using
positions and avoiding obstacles. Doing all these tasks for a YOLO.
dynamic system with changing surroundings has been
computationally very expensive and thus difficult. YOLO, which performs object detection by means of a
Improvements in object detection, high bandwidth and low fixed grid regression. All bring different degrees of
latency cloud technology and increased capability of improvements in detection efficiency over the primary R-
commodity hardware in the past years, now enables us to CNN and make object recognition more feasible in real-time
perform each of these tasks in conjunction and solve the and accuracy. YOLO is one of the fastest algorithms out
navigation problem for pedestrian environments of medium there to detect objects. Although it is no longer the most
to low density crowds. Navigation in congested accurate algorithm for object detection, when you need real-
environments can be set out as a three part problem - Object time detection without losing too much precision, it is a very
Detection and tracking, Pedestrian trajectory prediction and good choice.
future position estimation and Path planning in obstacle-rid
surroundings. This paper solves the problem by using

IJISRT21APR178 www.ijisrt.com 76
Volume 6, Issue 4, April – 2021 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
Huynh Manh, Gita Alaghband[2] proposed two scene- Model for human trajectory prediction
LSTM models which are used for predicting human Until 2016,the major way to calculate trajectory
trajectories. The Model learns the information regarding the prediction was using a physics and mathematics based
movement of people in the scene, this is a more detailed approach. These methods were later overtaken by data
look in trajectory prediction and performs better in driven approaches [1],[2],[7],[8] as they can train a machine
comparison. The model consisted of three parts mainly learning model which can learn the complex patterns of
pedestrian movement, scene model and scene data filter. human movement and avoid collisions.
scene model trains the grid-cells memories in order to
characterize the common trajectories in the cells which are The first model that we are planning on using is an
the passed through filters to select the target trajectory. It advancement to the basic LSTM model. It takes LSTM
was aimed for more social interactions not only involving models for all the individuals present in the scene and
humans but also other social objects. combines the “hidden states”(outputs) of all these LSTM
models. This way it helps to take into account the
Alexandre Alahi, Kratarth Goel, Vignesh interactions that are present between the humans in the
Ramanathan, Alexandre Robicquet and Li Fei-Fei[1] put scene. It works on the same principle mentioned in [1].
forward a novel approach for building a model which can
account for the behavior of social objects within a large The second model that we are going to use is a Transformers
neighborhood, while predicting the target’s path. The use of Model.
one LSTM model per person did not encapture the social We use a transformer network to model each
environment in a scene. To address the limitation they individual and leverage the sequential nature of the task to
proposed connecting neighboring LSTMs through using a predict the future motion of the pedestrians. The attention
pooling strategy. mechanism enables the transformer model to focus on
multiple points in a pedestrian trajectory and identify
Junwei Liang, Lu Jiang,Juan Carlos Niebles, sequence non-linearities. We use the same parameters as the
Alexander Hauptmann and Li Fei-Fei [7] worked upon the original transformer paper for our implementation.
modeling of paths and activities to benefit future path
prediction and its various possible application. Accordingly The metrics that are going to be used to evaluate these
the method was divided into different parts addressing the models mentioned above are:
path prediction and correlating activities. The person ADE(Average Displacement Error) and FDE(Final
behaviour is typically tested out using the CNNs trained Displacement Error).
which are then focused onto the LSTM encoders to predict
the path the person takes. This approach paves the way for Navigation module
creating a applicable usage of the path planning problem by A weighted graph is constructed using the predicted
integrating it with the activity prediction. future positions of the targets and the shortest path through
the graph is calculated using A* search algorithm.
Francesco Giuliari, Irtiza Hasan, Marco Cristani and
Fabio Galasso [9] put forward a transformer model for The second method we use is a Voronoi graph. This
trajectory prediction that does not make use of “social” method is able to minimize collision risks and works by
elements of the environment. Rather by assigning individual dividing the environment into regions.
transformer units to each pedestrian/targets to forecast the
resulting trajectory frame by frame. The model is a step
ahead of previous attempts since it does not combine or pool
different LSTMs thereby giving a better chance at real-time
usage and efficiently generalises in certain scenarios.

IV. METHODOLOGY

Object detection and tracking with YOLO and DeepSort

You only look once or YOLO is used for real time
object detection. It is one of the fastest algorithms which
works on 45 frames per second. Input video is passed to the
system and it extracts frames and sends it to the object
detector which is YOLO here. YOLO then generates
bounding boxes with class ID for each object. Here we need
to track objects for predicting their trajectory. So Deep
SORT extension of Simple Real time Tracker is used.
DeepSort tracks the objects detected using YOLO and the
coordinates of objects for a particular time frame are stored
in the system which will next be used for trajectory
prediction. DeepSort is efficient in real time and cost for Fig. 1. System Workflow
running is also low which is favourable for our system.

IJISRT21APR178 www.ijisrt.com 77
Volume 6, Issue 4, April – 2021 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
V. CONCLUSION [9]. Transformer Networks for Trajectory Forecasting
Francesco Giuliari, Irtiza Hasan, Marco Cristani, Fabio
The proposed set of methods are able to successfully Galasso
tackle the three-part problem of real-time online navigation
in crowded pedestrian surroundings. While there is still
scope for future improvement of efficiency and latency of
the technique, the current collection of processes can be
effectively applied to a number of navigation problems
ranging from a wide variety of industrial payload delivery
uses to small scale hospitality and housekeeping situations.

VI. LIMITATIONS

As our system includes two Deep Learning Models,

this makes it computationally heavy and expensive, thus
raising issues for deployment. As a result, constant
updations and efforts are in progress to try and lighten the
model, thus making it easy for deployment and ready to use
for any individual. Another drawback would be that our
model cannot be implemented or executed on any normal
machine. A specialized GPU is a must which is required due
to the heaviness of the model.

REFERENCES

[1]. Alahi, A.; Goel, K.; Ramanathan, V.; Robicquet, A.;

Li, F.-F.; Savarese, S. Social lstm: Human trajectory
prediction in crowded spaces. In Proceedings of the
IEEE Conference on Computer Vision and Pattern
Recognition, Las Vegas, NV, USA, 26 June–1 July
2016; pp. 961–971.
[2]. Huynh Manh, Gita Alaghband; Scene-LSTM: A
Model for Human Trajectory Prediction; University of
Colorado Denver; August 2018; arXiv:1808.04018
[cs.CV]
[3]. Hong Qu, Ke Xing, T. Alexander; An improved
genetic algorithm with co-evolutionary strategy for
global path planning of multiple mobile robots;
Neurocomputing, Volume 120, Pages 509-517, 23
November 2013
[4]. Chen, Z.; Song, C.; Yang, Y.; Zhao, B.; Hu, Y.; Liu,
S.; Zhang, J. Robot Navigation Based on Human
Trajectory Prediction and Multiple Travel Modes.
Appl. Sci. 2018, 8, 2205
[5]. Choset, Burdick Howei, Joel.;Sensor-based
exploration: the hierarchical generalized Voronoi
graph.; The International Journal of Robotics Research
19 (2) (2000), pp. 96-125; February 1, 2000
[6]. Visual Object Detection and Tracking using YOLO
and SORT, International Journal Of Engineering
Research &Technology (IJERT) Volume 08, Issue 11
(November 2019)
[7]. Peeking into the Future: Predicting Future Person
Activities and Locations in Videos,Junwei Liang1, Lu
Jiang,Juan Carlos Niebles, Alexander Hauptmann, Li
Fei-Fei.
[8]. Agrim Gupta et al. “Social GAN: Socially Acceptable
Trajectories with Generative Adversarial Networks”.
In: 2018 IEEE/CVF Conference on Computer Vision
and Pattern Recognition (2018), pp. 2255–2264.

IJISRT21APR178 www.ijisrt.com 78