Solar energy is getting a lot of traction due to the reduced cost and friendlier to the environme... more Solar energy is getting a lot of traction due to the reduced cost and friendlier to the environment compared to fossil fuel. It is essential to inspect the PV farms to ensure that the correct capacity produced through early PV fault detection. We proposed a full autonomous solution, where the drone mission is programmed to follow a specific Global Positioning System (GPS) waypoints. The collected videos will undergo various image processing techniques to detect and track the PV panels. In this paper, we tried two different PV panel detection approaches. Both detections gave acceptable results. The first detection relies on various image processing techniques. The second detection relies on deep learning architecture called mask Region-based Convolution Neural Network (R-CNN). After that, we track the PV panels in every frame using camera data alone. The advantage of tracking the PV panels is to ensure unrepeated PV panel through tagging even if the drone flies over the panel again s...
Additive Manufacturing (AM) technologies offer numerous solutions to today’s challenges and a var... more Additive Manufacturing (AM) technologies offer numerous solutions to today’s challenges and a variety of applications. Since AM is a promising manufacturing route and is frequently involved with product development, the designing process for such products or parts involves multi-disciplinary teams, such as subject-matter experts and AM processing experts. Often, there is a gap in communication between the two sides occurs in such projects, leading to multiple avoidable iterations that can affect the project resources and outcomes. This study proposes an innovative approach to teach AM for engineers from multiple disciplines to be involved in future projects that integrate AM. The course was designed using three teaching strategies and approaches, which are Design with Additive Manufacturing, Project-Based Learning, and engineering facilitator. The proposed AM accelerated course has proven to be a practical approach that can facilitate the communication process during such projects, ...
Autonomous navigation in wide outdoor areas faces challenges not present in structured environmen... more Autonomous navigation in wide outdoor areas faces challenges not present in structured environments. Simultaneous localization and mapping (SLAM) is a popular method of creating the map using the robot, and navigating in it. SLAM is difficult to achieve where there is no paved roads, buildings, and trees like in the sandy deserts. Autonomous robots cannot predict what is waiting behind the sand dune. Even if the terrain was mapped earlier, sand dunes can shift unpredictably. Therefore, mapping and GPS-based path planning should be done before every autonomous mission in such a dynamically shifting environment. Navigation can rely on GPS signals given the lack of obstructions. We develop a method for an Unmanned Aerial Vehicle (UAV) to autonomously take hundreds of images, convert terrain images into an elevation map, analyze the elevation map and generate traversable GPS waypoints for the Unmanned Ground Vehicle (UGV).
Localization of Unmanned Ground Vehicle (UGV) in multi-terrain environments, specifically soft te... more Localization of Unmanned Ground Vehicle (UGV) in multi-terrain environments, specifically soft terrain, is challenging. High slip scenarios make localization a difficult task. The internal sensors of the UGV are highly dependent on the terrain structure and are prone to errors. Thus, an appropriate measure should be in place to address the mentioned challenges to achieve precise navigation of the UGV. To address this issue, we propose using the GNSS-RTK/IMU/odometer/optical flow-based EKF system to improve state estimation of the UGV and compare it with some legacy approaches. The ground-facing camera will be integrated with image processing techniques such as optical flow to get linear and angular velocities. Additionally, a convolutional neural network (CNN) is used to classify the terrain conditions and update the covariance matrices for the odometer in EKF accordingly.
Title of dissertation: CONCURRENT LOCALIZATION AND MAPPING WITH SONAR SENSORS AND CONSIDERATION O... more Title of dissertation: CONCURRENT LOCALIZATION AND MAPPING WITH SONAR SENSORS AND CONSIDERATION OF VEHICLE MOTIONS Hesham Ismail, 2016 Dissertation directed by: Professor Balakumar Balachandran Department of Mechanical Engineering Simultaneous Localization and Mapping (SLAM) is a procedure used to determine the location of a mobile vehicle in an unknown environment, while constructing a map of the unknown environment at the same time. Mobile platforms, which make use of SLAM algorithms, have industrial applications in autonomous maintenance, such as the inspection of flaws and defects in oil pipelines and storage tanks. A typical SLAM consists of four main components, namely, experimental setup (data gathering), vehicle pose estimation, feature extraction, and filtering. Feature extraction is the process of realizing significant features from the unknown environment such as corners, edges, walls, and interior features. In this work, an original feature extraction algorithm specific ...
Title of dissertation: CONCURRENT LOCALIZATION AND MAPPING WITH SONAR SENSORS AND CONSIDERATION O... more Title of dissertation: CONCURRENT LOCALIZATION AND MAPPING WITH SONAR SENSORS AND CONSIDERATION OF VEHICLE MOTIONS Hesham Ismail, 2016 Dissertation directed by: Professor Balakumar Balachandran Department of Mechanical Engineering Simultaneous Localization and Mapping (SLAM) is a procedure used to determine the location of a mobile vehicle in an unknown environment, while constructing a map of the unknown environment at the same time. Mobile platforms, which make use of SLAM algorithms, have industrial applications in autonomous maintenance, such as the inspection of flaws and defects in oil pipelines and storage tanks. A typical SLAM consists of four main components, namely, experimental setup (data gathering), vehicle pose estimation, feature extraction, and filtering. Feature extraction is the process of realizing significant features from the unknown environment such as corners, edges, walls, and interior features. In this work, an original feature extraction algorithm specific ...
In recent years, there has been a lot of interest in clean energy especially in solar energy lead... more In recent years, there has been a lot of interest in clean energy especially in solar energy leading to the construction of numerous photovoltaic (PV) panels around the world. These panels need to be inspected to make sure they produce the desired amount of electricity. There are many methods to inspect which are manual, semi-autonomous and fully autonomous. The manual and semi-autonomous already exist but the process is tedious and requires a lot of time and resources. In this paper, we propose a fully autonomous solution where the drone will be preprogrammed to follow certain waypoints inputted via GPS data. The drone is equipped with a thermal camera where the video is recorded for postprocessing. The video is processed offline in order to detect the panels using image processing techniques such as thresholding, binary, canny edge, hough transform, and others. The novelty of this paper is on proposing a fully integrated solution for PV panel detection using a drone; it paves a wa...
Site assessments for bifacial Photovoltaic (PV) installation are quite challenging to conduct man... more Site assessments for bifacial Photovoltaic (PV) installation are quite challenging to conduct manually due to the area size and the extreme temperature conditions at desert sites. We designed and built an autonomous Unmanned Ground Vehicle (UGV) fitted with a Global Navigation Satellite Network-System Real-Time Kinematic (GNSS-RTK) positioning device, an Inertial Measurement Unit (IMU), encoder to improve and aid site assessments in desert condition. Sandy terrains deserts are challenging for UGV’s because they increase the likelihood of wheel slippage due to reduced traction. Sensor details such as IMU, GNSS-RTK, and encoder should be taken into consideration to account for the errors that the desert terrains pose. This study compared the Extended Kalman Filter (EKF) for standard GPS & GNSS-RTK to verify which performs better for the UGV’s position estimation. The estimated UGV’s position from the kinematics model and EKF are validated using a drone camera system that uses an image processing technique to verify the UGV’s position with the help of the visible reference cones. Throughout the experiments, the GNSS-RTK performed better than GPS. Also, the EKF performed as well as the GNSS-RTK by trusting it more than the encoder/gyroscope reading.
Mobile platforms that make use of concurrent localization and mapping algorithms have industrial ... more Mobile platforms that make use of concurrent localization and mapping algorithms have industrial applications for autonomous inspection and maintenance, such as the inspection of flaws and defects in oil pipelines and storage tanks. An important component of these algorithms is feature extraction, which involves detection of significant features that represent the environment. For example, points and lines can be used to represent features such as corners, edges, and walls. Feature extraction algorithms make use of relative position and angle data from sensor measurements gathered as the mobile vehicle traverses the environment. In this paper, sound navigation and ranging (SONAR) sensor data obtained from a mobile vehicle platform are considered for feature extraction and related algorithms are developed and studied. In particular, different combinations of commonly used feature extraction algorithms are examined to enhance the representation of the environment. The authors fuse the...
Simultaneous localization and mapping (SLAM) is a technique used to determine the location of a m... more Simultaneous localization and mapping (SLAM) is a technique used to determine the location of a mobile vehicle in an unknown environment, while constructing a map of the unknown environment at the same time. Mobile platforms, which make use of SLAM algorithms, have industrial applications in autonomous maintenance, such as the inspection of flaws and defects in oil pipelines and storage tanks. An important component of SLAM is feature extraction, which is the process of detecting and extracting significant features such as corners, edges, and walls in an environment. Here, the use of sonars as sensors mounted on a mobile platform is examined, and a comparison of different algorithms currently in use is made and presented. This comparison is performed through a combination of experimental and numerical studies. The triangulation-based fusion algorithm is examined for point feature detection, and the standard Hough Transform and the triangulation Hough fusion (THF) are used for line d...
According to the International Renewable Energy Agency (IRENA), photovoltaic (PV) production anti... more According to the International Renewable Energy Agency (IRENA), photovoltaic (PV) production anticipates an increase of 10% in 2021. The rapid increase in the PV panels installation requires a robust inspection method of existing solar parks to maintain efficiency and productivity levels. With advancements in drone technology, Unmanned Aerial Vehicles (UAV) are being used to inspect the solar parks, either flown manually or autonomously. Furthermore, there are various image processing approaches to analyze the data gathered. However, current practical application techniques do not effectively localize the defective panels present within the solar farm. This paper proposes a method to inspect large-scale solar parks using an autonomous drone equipped with Real-Time Kinematic (RTK) and camera. The proposed method is a fully autonomous solution for inspecting PV panels, with effective detection and localization of faults. It can ease the procedure of inspection by automating it and giv...
Solar energy is getting a lot of traction due to the reduced cost and friendlier to the environme... more Solar energy is getting a lot of traction due to the reduced cost and friendlier to the environment compared to fossil fuel. It is essential to inspect the PV farms to ensure that the correct capacity produced through early PV fault detection. We proposed a full autonomous solution, where the drone mission is programmed to follow a specific Global Positioning System (GPS) waypoints. The collected videos will undergo various image processing techniques to detect and track the PV panels. In this paper, we tried two different PV panel detection approaches. Both detections gave acceptable results. The first detection relies on various image processing techniques. The second detection relies on deep learning architecture called mask Region-based Convolution Neural Network (R-CNN). After that, we track the PV panels in every frame using camera data alone. The advantage of tracking the PV panels is to ensure unrepeated PV panel through tagging even if the drone flies over the panel again s...
Additive Manufacturing (AM) technologies offer numerous solutions to today’s challenges and a var... more Additive Manufacturing (AM) technologies offer numerous solutions to today’s challenges and a variety of applications. Since AM is a promising manufacturing route and is frequently involved with product development, the designing process for such products or parts involves multi-disciplinary teams, such as subject-matter experts and AM processing experts. Often, there is a gap in communication between the two sides occurs in such projects, leading to multiple avoidable iterations that can affect the project resources and outcomes. This study proposes an innovative approach to teach AM for engineers from multiple disciplines to be involved in future projects that integrate AM. The course was designed using three teaching strategies and approaches, which are Design with Additive Manufacturing, Project-Based Learning, and engineering facilitator. The proposed AM accelerated course has proven to be a practical approach that can facilitate the communication process during such projects, ...
Autonomous navigation in wide outdoor areas faces challenges not present in structured environmen... more Autonomous navigation in wide outdoor areas faces challenges not present in structured environments. Simultaneous localization and mapping (SLAM) is a popular method of creating the map using the robot, and navigating in it. SLAM is difficult to achieve where there is no paved roads, buildings, and trees like in the sandy deserts. Autonomous robots cannot predict what is waiting behind the sand dune. Even if the terrain was mapped earlier, sand dunes can shift unpredictably. Therefore, mapping and GPS-based path planning should be done before every autonomous mission in such a dynamically shifting environment. Navigation can rely on GPS signals given the lack of obstructions. We develop a method for an Unmanned Aerial Vehicle (UAV) to autonomously take hundreds of images, convert terrain images into an elevation map, analyze the elevation map and generate traversable GPS waypoints for the Unmanned Ground Vehicle (UGV).
Localization of Unmanned Ground Vehicle (UGV) in multi-terrain environments, specifically soft te... more Localization of Unmanned Ground Vehicle (UGV) in multi-terrain environments, specifically soft terrain, is challenging. High slip scenarios make localization a difficult task. The internal sensors of the UGV are highly dependent on the terrain structure and are prone to errors. Thus, an appropriate measure should be in place to address the mentioned challenges to achieve precise navigation of the UGV. To address this issue, we propose using the GNSS-RTK/IMU/odometer/optical flow-based EKF system to improve state estimation of the UGV and compare it with some legacy approaches. The ground-facing camera will be integrated with image processing techniques such as optical flow to get linear and angular velocities. Additionally, a convolutional neural network (CNN) is used to classify the terrain conditions and update the covariance matrices for the odometer in EKF accordingly.
Title of dissertation: CONCURRENT LOCALIZATION AND MAPPING WITH SONAR SENSORS AND CONSIDERATION O... more Title of dissertation: CONCURRENT LOCALIZATION AND MAPPING WITH SONAR SENSORS AND CONSIDERATION OF VEHICLE MOTIONS Hesham Ismail, 2016 Dissertation directed by: Professor Balakumar Balachandran Department of Mechanical Engineering Simultaneous Localization and Mapping (SLAM) is a procedure used to determine the location of a mobile vehicle in an unknown environment, while constructing a map of the unknown environment at the same time. Mobile platforms, which make use of SLAM algorithms, have industrial applications in autonomous maintenance, such as the inspection of flaws and defects in oil pipelines and storage tanks. A typical SLAM consists of four main components, namely, experimental setup (data gathering), vehicle pose estimation, feature extraction, and filtering. Feature extraction is the process of realizing significant features from the unknown environment such as corners, edges, walls, and interior features. In this work, an original feature extraction algorithm specific ...
Title of dissertation: CONCURRENT LOCALIZATION AND MAPPING WITH SONAR SENSORS AND CONSIDERATION O... more Title of dissertation: CONCURRENT LOCALIZATION AND MAPPING WITH SONAR SENSORS AND CONSIDERATION OF VEHICLE MOTIONS Hesham Ismail, 2016 Dissertation directed by: Professor Balakumar Balachandran Department of Mechanical Engineering Simultaneous Localization and Mapping (SLAM) is a procedure used to determine the location of a mobile vehicle in an unknown environment, while constructing a map of the unknown environment at the same time. Mobile platforms, which make use of SLAM algorithms, have industrial applications in autonomous maintenance, such as the inspection of flaws and defects in oil pipelines and storage tanks. A typical SLAM consists of four main components, namely, experimental setup (data gathering), vehicle pose estimation, feature extraction, and filtering. Feature extraction is the process of realizing significant features from the unknown environment such as corners, edges, walls, and interior features. In this work, an original feature extraction algorithm specific ...
In recent years, there has been a lot of interest in clean energy especially in solar energy lead... more In recent years, there has been a lot of interest in clean energy especially in solar energy leading to the construction of numerous photovoltaic (PV) panels around the world. These panels need to be inspected to make sure they produce the desired amount of electricity. There are many methods to inspect which are manual, semi-autonomous and fully autonomous. The manual and semi-autonomous already exist but the process is tedious and requires a lot of time and resources. In this paper, we propose a fully autonomous solution where the drone will be preprogrammed to follow certain waypoints inputted via GPS data. The drone is equipped with a thermal camera where the video is recorded for postprocessing. The video is processed offline in order to detect the panels using image processing techniques such as thresholding, binary, canny edge, hough transform, and others. The novelty of this paper is on proposing a fully integrated solution for PV panel detection using a drone; it paves a wa...
Site assessments for bifacial Photovoltaic (PV) installation are quite challenging to conduct man... more Site assessments for bifacial Photovoltaic (PV) installation are quite challenging to conduct manually due to the area size and the extreme temperature conditions at desert sites. We designed and built an autonomous Unmanned Ground Vehicle (UGV) fitted with a Global Navigation Satellite Network-System Real-Time Kinematic (GNSS-RTK) positioning device, an Inertial Measurement Unit (IMU), encoder to improve and aid site assessments in desert condition. Sandy terrains deserts are challenging for UGV’s because they increase the likelihood of wheel slippage due to reduced traction. Sensor details such as IMU, GNSS-RTK, and encoder should be taken into consideration to account for the errors that the desert terrains pose. This study compared the Extended Kalman Filter (EKF) for standard GPS & GNSS-RTK to verify which performs better for the UGV’s position estimation. The estimated UGV’s position from the kinematics model and EKF are validated using a drone camera system that uses an image processing technique to verify the UGV’s position with the help of the visible reference cones. Throughout the experiments, the GNSS-RTK performed better than GPS. Also, the EKF performed as well as the GNSS-RTK by trusting it more than the encoder/gyroscope reading.
Mobile platforms that make use of concurrent localization and mapping algorithms have industrial ... more Mobile platforms that make use of concurrent localization and mapping algorithms have industrial applications for autonomous inspection and maintenance, such as the inspection of flaws and defects in oil pipelines and storage tanks. An important component of these algorithms is feature extraction, which involves detection of significant features that represent the environment. For example, points and lines can be used to represent features such as corners, edges, and walls. Feature extraction algorithms make use of relative position and angle data from sensor measurements gathered as the mobile vehicle traverses the environment. In this paper, sound navigation and ranging (SONAR) sensor data obtained from a mobile vehicle platform are considered for feature extraction and related algorithms are developed and studied. In particular, different combinations of commonly used feature extraction algorithms are examined to enhance the representation of the environment. The authors fuse the...
Simultaneous localization and mapping (SLAM) is a technique used to determine the location of a m... more Simultaneous localization and mapping (SLAM) is a technique used to determine the location of a mobile vehicle in an unknown environment, while constructing a map of the unknown environment at the same time. Mobile platforms, which make use of SLAM algorithms, have industrial applications in autonomous maintenance, such as the inspection of flaws and defects in oil pipelines and storage tanks. An important component of SLAM is feature extraction, which is the process of detecting and extracting significant features such as corners, edges, and walls in an environment. Here, the use of sonars as sensors mounted on a mobile platform is examined, and a comparison of different algorithms currently in use is made and presented. This comparison is performed through a combination of experimental and numerical studies. The triangulation-based fusion algorithm is examined for point feature detection, and the standard Hough Transform and the triangulation Hough fusion (THF) are used for line d...
According to the International Renewable Energy Agency (IRENA), photovoltaic (PV) production anti... more According to the International Renewable Energy Agency (IRENA), photovoltaic (PV) production anticipates an increase of 10% in 2021. The rapid increase in the PV panels installation requires a robust inspection method of existing solar parks to maintain efficiency and productivity levels. With advancements in drone technology, Unmanned Aerial Vehicles (UAV) are being used to inspect the solar parks, either flown manually or autonomously. Furthermore, there are various image processing approaches to analyze the data gathered. However, current practical application techniques do not effectively localize the defective panels present within the solar farm. This paper proposes a method to inspect large-scale solar parks using an autonomous drone equipped with Real-Time Kinematic (RTK) and camera. The proposed method is a fully autonomous solution for inspecting PV panels, with effective detection and localization of faults. It can ease the procedure of inspection by automating it and giv...
Uploads