Rina Project
Rina Project
Rina Project
ON
TUPAKULA RINA
(22091F0042)
YEAR: 2023-2024
Rajeev Gandhi Memorial College of Engineering &Technology
(AUTONOMOUS)
AFFILIATED TO J.N.T UNIVERSITY ANANTAPUR. ACCREDITED BY NBA (TIER-1) &
NAAC OF UGC. NEW DELHI, WITH A+ GRADE
RECOGNIZED UGC-DDU KAUSHAL KENDRA
NANDYAL-518501, (Estd-1995)
(ESTD – 1995)
CERTIFICATE
This is to certify that TUPAKULA RINA (22091F0042), of MCA III- semester, has carried out
the mini-project work entitled “ENHANCING NIGHTTIME PEDESTRIAN DETECTION
WITH VISUAL AND MILLIMETER-WAVE RADAR FUSION” under the supervision and
guidance of G.RAJASEKHAR REDDY M.tech.(Ph.D), Professor, CSE Department, in partial
fulfillment of the requirements for the award of Degree of Master of Computer Applications
from Rajeev Gandhi Memorial College of Engineering & Technology (Autonomous),
Nandyal is a bonafied record of the work done by him during 2023-2024.
Place: Nandyal
Date: External Examiner
CONFIRMATION LETTER
05-03-24
To
The HOD,
Andhra Pradesh.
Respected Sir/Madam,
This is to certify Ms.TUPAKULA RINA, bearing Reg. No. 22091F0042 student of Rajeev
Gandhi Memorial College of Engineering and Technology, Nandyal has been admitted to do a project
titled “ENHANCING NIGHTTIME PEDESTRIAN DEFECTION WITH VISUAL AND
MILLIMETER WAVE-RADAR FUSION” on Python Technology at “Tru Research Support, Hyderabad”
The project duration is from 1th Feb, 2024 to 1st June, 2024.
The project will be done under the guidance of our technical Team. The project Team is guided by P CARMEL
VANDHANA, Project Developer
From
P CARMEL VANDHANA
AUTHORISED SIGNATURE
ACKNOWLEDGEMENT
The satisfaction that accomplishes completion of any task would be incomplete without
the mention of people who made it possible and whose encouragement and guidance has been
a source of inspiration throughout the course of the project I want to express my sincere
gratitude to my project guide G.RAJASEKHAR REDDY, M.tech,Ph.D. Professor in CSE
Department Under his guidance, I have learned a lot about different aspects in research,
including finding a good research problem.
TUPAKULA RINA
(22091F0042)
DECLARATION
This is a record of bonafide work carried out by us and the results embodied in this
project have not been reproduced or copied from any source. The results embodied in this
project have not been submitted to any other university or institute for the award of any other
Degree.
TUPAKULA RINA
(22091F0030)
CONTENTS
1. INTRODUCTION 2
2. LITERATURE REVIEW 3
2.2 Disadvantages 4
2.4 Advantages 5
3.1. Algorithms 6
3.6 Modules 21
4. Implementation 23
4.1 Python 23
5 Testing 27
6 Test Cases 30
7 Future Enhancement 31
8 Conclusion 32
Abstract
A target detection algorithm based on the fusion of vision sensors and millimeter wave radar data
can effectively improve the safety of self-driving vehicles. However, a single sensor cannot obtain
comprehensive category status information on the target in the nighttime environment. To improve
pedestrian detection in nighttime traffic scenarios, this paper proposes a nighttime pedestrian
detection method based on the fusion of infrared vision information and millimeter wave (MMW)
radar data. The lateral localization and category features of the target are obtained using the
improved YOLOv5 deep learning algorithm, the distance and velocity information of the target is
obtained by preprocessing the MMW radar acquisition data, the pedestrian target is tracked by
using the extended Kalman filter for MMW radar detection, the projection of the valid radar target
point on the Infrared Radiatio (IR) image is completed according to the spatiotemporal fusion, and
then the correlation gate method is used to correlate the data, and the visual information is inherited
to the radar points to get the target multimodal information, and the success associated valid target
sequences are weighted to obtain the accurate target position. Finally, a decision-level fusion
algorithm framework is proposed to complete the output of pedestrian multimodal information in
nighttime traffic scenes. Theoretical analysis and experimental results show that the accuracy and
robustness of this method for nighttime pedestrian recognition are better than those of a single
sensor. In propose work author has modified YoloV5 but not experimented with advance YOLO
family other version as like Yolo6, 7 or 8. So as extension we have trained same dataset with
Yolov6 and it’s giving better prediction accuracy compare to Optimized Yolov5.
CHAPTER-1
1. INTRODUCTION
At present, the research related to automotive self-driving technology is getting more and more in-
depth, and the perception of the surrounding environment in complex traffic scenes is an important
part of self-driving cars. The environment perception system is equipped with sensors such as
LIDAR, MMW radar, and cameras to obtain information on the category, location, and speed of
surrounding traffic targets, which can effectively reduce the risk of collision and improve the safety
performance of self-driving cars. Target detection is the process of extracting object classes and
locations based on the observation information from different sensors. As a key step in an
autonomous driving environment sensing system, target detection techniques based on deep
learning have received extensive attention and research. Among them, single-stage target detection
algorithms include YOLO [1], [2], [3], [4], [5], SSD [6], and RetinaNet [7]. The two-stage target
detection algorithms are mainly based on R-CNN [8] and Faster R-CNN [9] as the main research.
In recent years, the YOLO series algorithms have good detection accuracy and detection speed in
target detection. However, the current target detection task mainly focuses on visible images, but
visible images are greatly affected by the environment, especially in extremely bad weather and
nighttime conditions, and visible images lose some details, leading to degradation of target
detection performance.
CHAPTER-2
2. LITERATURE REVIEW
In complex scenes, it is a huge challenge to accurately detect motion-blurred, tiny, and dense
objects in the thermal infrared images. To solve this problem, robust thermal infrared vehicle and
pedestrian detection method is proposed in this paper. An important weight parameter β is first
proposed to reconstruct the loss function of the feature selective anchor-free (FSAF) module in its
online feature selection process, and the FSAF module is optimized to enhance the detection
performance of motion-blurred objects. The proposal of parameter β provides an effective solution
to the challenge of motion-blurred object detection. Then, the optimized anchor-free branches of
the FSAF module are plugged into the YOLOv3 single-shot detector and work jointly with the
anchor-based branches of the YOLOv3 detector in both training and inference, which efficiently
improves the detection precision of the detector for tiny and dense objects. Experimental results
show that the method proposed is superior to other typical thermal infrared vehicle and pedestrian
detection algorithms due to 72.2% mean average precision (mAP).
2.2 Disadvantages
1. However, a single sensor cannot obtain comprehensive category status information on the
target in the nighttime environment.
2. Sensors were dependent on similarity distance to identify objects but this technique prove
to be insufficient.
The author of the proposed work has altered YoloV5, but hasn't played around with other advanced
Yolo family versions like Yolo6, 7, or 8. As a result, we trained Yolov6 on the same dataset, and
its prediction accuracy is higher than that of Optimized Yolov5.
2.4 Advantages
1. improving the network structure of the YOLOv5 algorithm model, and verifying the good
performance of this improved algorithm in infrared image target detection through target
detection algorithm comparison experiments.
3. A decision-level fusion strategy is designed for target detection in the nighttime environment,
and it is experimentally verified that the strategy reduces the missed false detection rate of a
single sensor and improves the accuracy of target detection.
CHAPTER-3
3. SYSTEM DESIGN
Faster R-CNN (FRCNN): Faster R-CNN is a two-stage object detection algorithm. It employs a
region proposal network to generate candidate regions and a fast RCNN for object detection.
Region proposals and features are simultaneously learned, enhancing accuracy and speed in
detecting objects within an image.
Improved YOLOv5: Improved YOLOv5, an evolution of the YOLO (You Only Look Once)
architecture, introduces advanced backbone networks, efficient anchor generation, and optimized
training strategies. This enhances real-time object detection performance, achieving higher
accuracy and speed for a wide range of applications.
Extension YOLOv6: YOLOv6 extends the YOLO (You Only Look Once) series by incorporating
innovative features like dynamic model scaling, advanced augmentation, and improved network
architectures. This extension aims to enhance object detection performance, providing a flexible
and efficient solution for various computer vision tasks with improved accuracy.
A SPIRAL MODEL is used in this project. Here are some details regarding the project: The
Spiral Model is an iterative approach to software development that blends aspects of waterfall and
prototyping approaches The Spiral Model could be used in this project in the following ways:
• Identification of Goals and Requirements (Planning): Specify the goals, limitations, and
specifications of the project for the AI-powered e-vaccination system.
• Risk Analysis and Evaluation (Risk Assessment): Determine possible dangers connected to
user acceptability, data security, and AI integration.
Analyze the consequences and probability of these hazards.
• Engineering Prototype Development: Create a simple e-vaccination system prototype to
highlight important aspects.This might be a condensed version of the system with an emphasis on
key features like AI Chabot interaction and appointment scheduling.
• Build and Test (Engineering and Testing): Create the essential e-vaccination system
components, such as the database configuration and AI integration.
Test everything thoroughly to find and address any problems or bugs.
• Risk Analysis and Evaluation (Risk Assessment - Iteration 1): Reassess the risks that have
been identified, taking into account comments and insights from the testing and prototyping stages.
• Risk Analysis and Evaluation (Risk Assessment - Iteration 2): Reevaluate risks in light of
developments and new information.
• Complete System Development (Engineering - Iteration 2): Create the user interfaces, data
security protocols, and all anticipated features of the whole e-vaccination system.
• Comprehensive Testing (Testing - Iteration 2): Test the completely developed system
thoroughly to make sure it satisfies all requirements and is error-free.
The fully completed e-vaccination system should be implemented in a controlled setting, like a
pilot program for a particular community.
Economic feasibility, technical feasibility, and social feasibility are the three main factors taken
into account in the feasibility analysis.
The purpose of this study is to evaluate the system's potential financial impact on the company.
The amount of money the corporation can devote to system research and development is limited.
The costs have to make sense. Given the financial limits, the intended system could also be
implemented because the majority of the technologies were publicly available. Just the items that
can be customized need to be bought.
The purpose of this study is to evaluate the system's technological requirements, or technical
feasibility. Any system's development shouldn't put an undue burden on the available technical
resources. The availability of technological resources will consequently be in high demand. As a
result, the client will need to follow strict guidelines. Because implementing the intended system
will only require minimal or nonexistent adjustments, it must have the fewest requirements
possible.
One of the goals of the study is to assess the system's level of user acceptability. This entails
instructing the user on the proper usage of the technology. The user needs to accept the system as
a necessity rather than a threat. The methods employed to acquaint and instruct users about the
system will dictate the degree of acceptance among them. Being the final user of the system, he
needs more confidence to provide some insightful criticism, which is much welcomed.
Finding that the system request is doable is a significant result of the preliminary investigation.
Only if it is doable with the time and resources available will this be viable. The many
feasibilities that require analysis include
Operational Feasibility
Economic Feasibility
Technical Feasibility
Economic Feasibility
An evaluation of the financial case for a computer-based project is known as economic feasibility
or cost-benefit analysis. The hardware project had a cheap cost because hardware was installed
from the start and served many tasks. Any number of employees linked to the organization's local
area network (LAN) can utilize this tool at any time because it is network-based. The organization's
current resources will be used to construct the virtual private network. Therefore, the proposal is
financially viable.
Technical Feasibility
Technical feasibility, according to Roger S. Pressman, is the evaluation of the organization's
technical resources. The company requires IBM compatible computers that are linked to the
Intranet and Internet using a graphical web browser. The system is designed to operate in a
platform-independent setting. The system is developed using Java Server Pages, JavaScript,
HTML, SQL Server, and WebLogic Server. The technical viability analysis has been completed.
The system can be created using the current facility and is technically possible.
Not every project that is requested is worthwhile or practical. Only a small percentage of the
project requests that client users submit to an organization are actually taken up. On the other
hand, undertakings that are both desirable and doable ought to be scheduled. Following approval,
a project request's projected cost, priority, completion time, and staffing needs are utilized to
decide where on the project list to place it. Indeed, once the aforementioned conditions are met,
development activity can begin.
Input screens are present in nearly every module of this system. Error messages are designed to
warn users of mistakes they make and point them in the correct direction to prevent the entry of
erroneous data. Let's examine this in more detail under module design.
Input design is the process of converting user-created data into a format that computers can
understand. The goal of the input design is to provide logical and error-free data entry. Errors in
the input are managed by the input design. User-friendliness has been the primary focus of the
application's design. Because of the way the forms are designed, while processing the pointer is
automatically relocated to the required entry area. In certain cases, the user is further provided with
the option to select a suitable input from a list of options related to the field.
Each entry of data needs to be verified. The user can go on to the following pages if they have
completed all of the entries on the current page. Every time a user enters inaccurate data.
An application launches when it is used for the first time. Once the server is started, the browser
used is Internet Explorer. The other connected computers can operate as clients and the server
computer as the administrator because the project will be carried out over a local area network.
Import libraries
Yes NO NO PROCESS
VERIFY
Data processing
Feature Selection
User input
Final outcome
End process
User System
3.6 Modules
Data loading: using this module we are going to import the dataset.
Splitting data into train & test: using this module data will be divided into train & test
User signup & login: Using this module will get registration and login
User input: Using this module will give input for prediction
RAM : 8 GB
Hard Disk : 25 GB
Monitor : SVGA
Front-End : Flask
Back-End : Jupiter-Notebook
CHAPTER-4
4. Implementation
4.1 Python
Python is Interpreted: The interpreter handles Python at runtime. Your software doesn't need to
be assembled in order to execute. This reminds me of PHP and PERL.
Interactive: Python enables you to sit down at a prompt and work directly with the interpreter to
create programs.
Object-oriented programming, which encapsulates code into objects, is supported by Python.
Python is an Excellent Language for Novice Programmers: If you're just starting out with
programming, Python is an excellent language. It makes a wide range of program creation simple,
including basic text processing, Web browsers, and games.
4.1.1 History of Python
At the Netherlands' National Research Institute for Mathematics and Computer Science in the late
eighties and early nineties, Guido van Rossum created the programming language Python.
Numerous additional languages, such as ABC, Modula-3, C, C++, Algol-68, Small Talk, Unix
shell, and other scripting languages, are developed from Python. Copyright protects Python. The
GNU General Public License (GPL) is now used to license Python source code, just like it does
Perl. Although a core development team at the institute currently manages Python, Guido van
Rossum continues to play a crucial role in steering its direction.
4.1.2 Features of Python
Python's features include:
Easy-to-learn: Python has few keywords, simple structure, and a clearly defined syntax.
Easy-to-read: Python code is more clearly defined and visible to the eyes.
A broad standard library: Python's bulk of the library is very portable and cross-platform
compatible on UNIX, Windows, and Macintosh.
Interactive Mode: Python has support for an interactive mode which allows interactive
testing and debugging of snippets of code.
Portable: Python can run on a wide variety of hardware platforms and has the same
interface on all platforms.
Extendable: You can add low-level modules to the Python interpreter. These modules
enable programmers to add to or customize their tools to be more efficient.
GUI Programming: Python supports GUI applications that can be created and ported to
many system calls, libraries and windows systems, such as Windows MFC, Macintosh,
and the X Window system of Unix.
Scalable: Python provides a better structure and support for large programs than shell
scripting.
• It can be used as a scripting language or to compile large programs into byte-code; it supports
both functional and structured programming approaches in addition to object-oriented
programming.
• It provides very high-level dynamic data types and enables dynamic type checking. It integrates
well with C, C++, COM, Java, ActiveX, CORBA, and C. It makes garbage collection automatic.
4.2 TECHNOLOGY DESCRIPTION
Python is an object-oriented, high-level, general-purpose, interpreted programming language.
Python is an interpreted language with a syntax that lets programmers express ideas in fewer lines
of code than they might in languages like C++ or Java. Its design philosophy emphasizes code
readability, most notably by using whitespace indentation to delimit code blocks rather than curly
brackets or keywords. It offers building blocks that facilitate understandable programming at both
small and large sizes. There are interpreters for Python on a wide range of operating systems. The
non-profit Python Software Foundation oversees Python, the reference implementation of Python.
Python has automated memory management and a dynamic typing system. It features a sizable
and extensive standard library and supports a variety of programming paradigms, such as
imperative, functional, procedural, and object-oriented programming.
What is Python?
Python has automated memory management and a dynamic typing system. It features a sizable
and extensive standard library and supports a variety of programming paradigms, such as
imperative, functional, procedural, and object-oriented programming.
One well-liked programming language is Python. Guido van Rossum was the creator, and it was
published in 1991.
Database systems can be connected to using Python. It has the ability to read and edit files.
Complex mathematics and handling of large data can be accomplished with Python.
Python's syntax is straightforward and resembles that of the English language. Compared to some
other programming languages, Python's syntax enables programmers to construct programs with
less lines.
CHAPTER-5
5. Testing
System testing, also referred to as system-level tests or system-integration testing, is the process
in which a quality assurance (QA) team evaluates how the various components of an application
interact together in the full, integrated system or application. System testing verifies that an
application performs tasks as designed. This step, a kind of black box testing, focuses on the
functionality of an application. System testing, for example, might check that every kind of user
input produces the intended output across the application.
Phases of system testing:
A video tutorial about this test level. System testing examines every component of an application
to make sure that they work as a complete and unified whole. A QA team typically conducts system
testing after it checks individual modules with functional or user-story testing and then each
component through integration testing.
If a software build achieves the desired results in system testing, it gets a final check via acceptance
testing before it goes to production, where users consume the software. An app-dev team logs all
defects, and establishes what kinds and amount of defects are tolerable.
Software Testing Strategies:
Optimization of the approach to testing in software engineering is the best way to make it effective. A
software testing strategy defines what, when, and how to do whatever is necessary to make an end-product
of high quality. Usually, the following software testing strategies and their combinations are used to achieve
this major objective:
5.1Static Testing:
The early-stage testing strategy is static testing: it is performed without actually running the developing
product. Basically, such desk-checking is required to detect bugs and issues that are present in the code itself.
Such a check-up is important at the pre-deployment stage as it helps avoid problems caused byerrors in the
code and software structure deficits.
comparing the tests’ outcomes with the results of previous iterations (control flow testing).
CHAPTER-6
6.TEST CASES:
7. Future Enhancement
Despite solving the issue with the previous work, it has several drawbacks. The likelihood of the
estimate will be zero in the event that there are no class labels. Adding more machine learning
classification models to the proposed study would improve overall performance and boost the
accuracy of crime prediction in the future. By considering the income data for areas, it helps to
provide a better analysis for future improvements by predicting whether there is a relationship, if
any, between a neighborhood's crime rate and its residents' income levels.
CHAPTER-8
8. CONCLUSION
To solve the problem of low accuracy of pedestrian target detection by a single sensor at night
when the road environment is complex and the contrast of the surrounding environment is low,
this paper proposes a nighttime pedestrian detection algorithm based on the fusion of visual sensors
and MMW radar. The YOLOv5-ours algorithm is used to detect infrared images of pedestrians at
night, which improves the detection accuracy by 3.7% compared to the original YOLOv5
algorithm while increasing the detection speed from 29 FPS to 42 FPS. MMW radar can detect
target location and state information, and the EKF tracking algorithm is used to continuously
estimate and correct filter noise online for MMW radar observation data. statistical features to
detect and track the radar detection targets to improve the reliability and stability of radar data.
Then a decision-level fusion detection algorithm is proposed to compensate for the false detection
misses of infrared cameras and millimeter-wave radar. This paper designs target detection
experiments under different nighttime traffic road scenarios, and the experimental results show
that the accuracy of the fusion algorithm for nighttime pedestrian detection can reach 95.57%
under nighttime traffic scenarios such as small targets at a long distance, the presence of pedestrian
occlusion and foggy weather, which effectively reduces the false detection rate of single sensor
target detection and to a certain extent solves the fusion algorithm target detection performance
The problem of weak target detection performance of the fusion algorithm is solved to a certain
extent, and the accuracy and robustness of nighttime pedestrian detection are improved. The
current study cannot be directly applied to self-driving cars. In the next step, we will integrate the
sensors into the controller of self-driving cars through embedded devices to achieve dynamic real-
time target detection of pedestrians in front of self-driving cars in night scenarios, enriching the
application scenarios of this study.
APPENDIX-A:SCREEN SHOTS
1. User Login
2. Login Details
2. User Admin
3. Upload File
4. View Details
5. Upload Details
6. Coding
APENDIX-B: REFERENCES
[1] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, ‘‘You only look once: Unified, real-time
object detection,’’ in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2016, pp.
779–788.
[2] J. Redmon and A. Farhadi, ‘‘YOLO9000: Better, faster, stronger,’’ in Proc. IEEE Conf.
Comput. Vis. Pattern Recognit. (CVPR), Jul. 2017, pp. 6517–6525.
[4] A. Bochkovskiy, C.-Y. Wang, and H.-Y. Mark Liao, ‘‘YOLOv4: Optimal speed and accuracy
of object detection,’’ 2020, arXiv:2004.10934.
[5] P. Jiang, D. Ergu, F. Liu, Y. Cai, and B. Ma, ‘‘A review of YOLO algorithm developments,’’
Proc. Comput. Sci., vol. 199, pp. 1066–1073, 2022.
[6] W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, Alexander, and C. Berg,
‘‘SSD: Single shot MultiBox detector,’’ in Proc. 14th Eur. Conf. Comput. Vis. (ECCV), 2016,
pp. 21–37.
[7] T. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár, ‘‘Focal loss for dense object detection,’’
in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Oct. 2017, pp. 2999–3007.
[8] R. Girshick, J. Donahue, T. Darrell, and J. Malik, ‘‘Rich feature hierarchies for accurate object
detection and semantic segmentation,’’ in Proc. IEEE Conf. Comput. Vis. Pattern Recognit.
(CVPR), Jun. 2014, pp. 580–587.
[9] R. Girshick, ‘‘Fast R-CNN,’’ in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Dec. 2015, pp.
1440–1448.
[10] S. M. Patole, M. Torlak, D. Wang, and M. Ali, ‘‘Automotive radars: A review of signal
Dept. Of MCA, RGMCET, Nandyal 36
OPTIMIZED YOLOV5 FOR SOLAR CELL SURFACE
DEFECT DETECTION
processing techniques,’’ IEEE Signal Process. Mag., vol. 34, no. 2, pp. 22–35, Mar. 2017.
[11] X. Liu, T. Yang, and J. Li, ‘‘Real-time ground vehicle detection in aerial infrared imagery
based on convolutional neural network,’’ Electronics, vol. 7, no. 6, p. 78, May 2018.
[13] Y. Liu, H. Su, C. Zeng, and X. Li, ‘‘A robust thermal infrared vehicle and pedestrian
detection method in complex scenes,’’ Sensors, vol. 21, no. 4, p. 1240, Feb. 2021.
[14] Z. J. Zhu et al., ‘‘A parallel fusion network-based detection method for aerial infrared
vehicles with small targets,’’ J. Photon., vol. 51, no. 2, pp. 182–194, 2022.
[15] T. Wu, T. Wang, and Y. Liu, ‘‘Real-time vehicle and distance detection based on improved
YOLO v5 network,’’ in Proc. 3rd World Symp. Artif. Intell. (WSAI), Jun. 2021, pp. 24–28.
[16] H.-K. Jung and G.-S. Choi, ‘‘Improved YOLOv5: Efficient object detection using drone
images under various conditions,’’ Appl. Sci., vol. 12, no. 14, p. 7255, Jul. 2022.
[17] X. Zhu, S. Lyu, X. Wang, and Q. Zhao, ‘‘TPH-YOLOv5: Improved YOLOv5 based on
transformer prediction head for object detection on drone-captured scenarios,’’ in Proc.
IEEE/CVF Int. Conf. Comput. Vis. Workshops (ICCVW), Oct. 2021, pp. 2778–2788.
[18] X. Jin, Z. Li, and H. Yang, ‘‘Pedestrian detection with YOLOv5 in autonomous driving
scenario,’’ in Proc. 5th CAA Int. Conf. Veh. Control Intell. (CVCI), Oct. 2021, pp. 1–5.
[19] M. Kasper-Eulaers, N. Hahn, S. Berger, T. Sebulonsen, Ø. Myrland, and P. E. Kummervold,
‘‘Short communication: Detecting heavy goods vehicles in rest areas.