-
RoadRunner - Learning Traversability Estimation for Autonomous Off-road Driving
Authors:
Jonas Frey,
Shehryar Khattak,
Manthan Patel,
Deegan Atha,
Julian Nubert,
Curtis Padgett,
Marco Hutter,
Patrick Spieler
Abstract:
Autonomous navigation at high speeds in off-road environments necessitates robots to comprehensively understand their surroundings using onboard sensing only. The extreme conditions posed by the off-road setting can cause degraded camera image quality due to poor lighting and motion blur, as well as limited sparse geometric information available from LiDAR sensing when driving at high speeds. In t…
▽ More
Autonomous navigation at high speeds in off-road environments necessitates robots to comprehensively understand their surroundings using onboard sensing only. The extreme conditions posed by the off-road setting can cause degraded camera image quality due to poor lighting and motion blur, as well as limited sparse geometric information available from LiDAR sensing when driving at high speeds. In this work, we present RoadRunner, a novel framework capable of predicting terrain traversability and an elevation map directly from camera and LiDAR sensor inputs. RoadRunner enables reliable autonomous navigation, by fusing sensory information, handling of uncertainty, and generation of contextually informed predictions about the geometry and traversability of the terrain while operating at low latency. In contrast to existing methods relying on classifying handcrafted semantic classes and using heuristics to predict traversability costs, our method is trained end-to-end in a self-supervised fashion. The RoadRunner network architecture builds upon popular sensor fusion network architectures from the autonomous driving domain, which embed LiDAR and camera information into a common Bird's Eye View perspective. Training is enabled by utilizing an existing traversability estimation stack to generate training data in hindsight in a scalable manner from real-world off-road driving datasets. Furthermore, RoadRunner improves the system latency by a factor of roughly 4, from 500 ms to 140 ms, while improving the accuracy for traversability costs and elevation map predictions. We demonstrate the effectiveness of RoadRunner in enabling safe and reliable off-road navigation at high speeds in multiple real-world driving scenarios through unstructured desert environments.
△ Less
Submitted 3 March, 2024; v1 submitted 29 February, 2024;
originally announced February 2024.
-
Pixel to Elevation: Learning to Predict Elevation Maps at Long Range using Images for Autonomous Offroad Navigation
Authors:
Chanyoung Chung,
Georgios Georgakis,
Patrick Spieler,
Curtis Padgett,
Ali Agha,
Shehryar Khattak
Abstract:
Understanding terrain topology at long-range is crucial for the success of off-road robotic missions, especially when navigating at high-speeds. LiDAR sensors, which are currently heavily relied upon for geometric mapping, provide sparse measurements when mapping at greater distances. To address this challenge, we present a novel learning-based approach capable of predicting terrain elevation maps…
▽ More
Understanding terrain topology at long-range is crucial for the success of off-road robotic missions, especially when navigating at high-speeds. LiDAR sensors, which are currently heavily relied upon for geometric mapping, provide sparse measurements when mapping at greater distances. To address this challenge, we present a novel learning-based approach capable of predicting terrain elevation maps at long-range using only onboard egocentric images in real-time. Our proposed method is comprised of three main elements. First, a transformer-based encoder is introduced that learns cross-view associations between the egocentric views and prior bird-eye-view elevation map predictions. Second, an orientation-aware positional encoding is proposed to incorporate the 3D vehicle pose information over complex unstructured terrain with multi-view visual image features. Lastly, a history-augmented learn-able map embedding is proposed to achieve better temporal consistency between elevation map predictions to facilitate the downstream navigational tasks. We experimentally validate the applicability of our proposed approach for autonomous offroad robotic navigation in complex and unstructured terrain using real-world offroad driving data. Furthermore, the method is qualitatively and quantitatively compared against the current state-of-the-art methods. Extensive field experiments demonstrate that our method surpasses baseline models in accurately predicting terrain elevation while effectively capturing the overall terrain topology at long-ranges. Finally, ablation studies are conducted to highlight and understand the effect of key components of the proposed approach and validate their suitability to improve offroad robotic navigation capabilities.
△ Less
Submitted 20 April, 2024; v1 submitted 30 January, 2024;
originally announced January 2024.
-
ROAMER: Robust Offroad Autonomy using Multimodal State Estimation with Radar Velocity Integration
Authors:
Morten Nissov,
Shehryar Khattak,
Jeffrey A. Edlund,
Curtis Padgett,
Kostas Alexis,
Patrick Spieler
Abstract:
Reliable offroad autonomy requires low-latency, high-accuracy state estimates of pose as well as velocity, which remain viable throughout environments with sub-optimal operating conditions for the utilized perception modalities. As state estimation remains a single point of failure system in the majority of aspiring autonomous systems, failing to address the environmental degradation the perceptio…
▽ More
Reliable offroad autonomy requires low-latency, high-accuracy state estimates of pose as well as velocity, which remain viable throughout environments with sub-optimal operating conditions for the utilized perception modalities. As state estimation remains a single point of failure system in the majority of aspiring autonomous systems, failing to address the environmental degradation the perception sensors could potentially experience given the operating conditions, can be a mission-critical shortcoming. In this work, a method for integration of radar velocity information in a LiDAR-inertial odometry solution is proposed, enabling consistent estimation performance even with degraded LiDAR-inertial odometry. The proposed method utilizes the direct velocity-measuring capabilities of an Frequency Modulated Continuous Wave (FMCW) radar sensor to enhance the LiDAR-inertial smoother solution onboard the vehicle through integration of the forward velocity measurement into the graph-based smoother. This leads to increased robustness in the overall estimation solution, even in the absence of LiDAR data. This method was validated by hardware experiments conducted onboard an all-terrain vehicle traveling at high speed, ~12 m/s, in demanding offroad environments.
△ Less
Submitted 30 January, 2024;
originally announced January 2024.
-
Classifying Text-Based Conspiracy Tweets related to COVID-19 using Contextualized Word Embeddings
Authors:
Abdul Rehman,
Rabeeh Ayaz Abbasi,
Irfan ul Haq Qureshi,
Akmal Saeed Khattak
Abstract:
The FakeNews task in MediaEval 2022 investigates the challenge of finding accurate and high-performance models for the classification of conspiracy tweets related to COVID-19. In this paper, we used BERT, ELMO, and their combination for feature extraction and RandomForest as classifier. The results show that ELMO performs slightly better than BERT, however their combination at feature level reduce…
▽ More
The FakeNews task in MediaEval 2022 investigates the challenge of finding accurate and high-performance models for the classification of conspiracy tweets related to COVID-19. In this paper, we used BERT, ELMO, and their combination for feature extraction and RandomForest as classifier. The results show that ELMO performs slightly better than BERT, however their combination at feature level reduces the performance.
△ Less
Submitted 7 March, 2023;
originally announced March 2023.
-
Identifying Misinformation Spreaders: A Graph-Based Semi-Supervised Learning Approach
Authors:
Atta Ullah,
Rabeeh Ayaz Abbasi,
Akmal Saeed Khattak,
Anwar Said
Abstract:
In this paper we proposed a Graph-Based conspiracy source detection method for the MediaEval task 2022 FakeNews: Corona Virus and Conspiracies Multimedia Analysis Task. The goal of this study was to apply SOTA graph neural network methods to the problem of misinformation spreading in online social networks. We explore three different Graph Neural Network models: GCN, GraphSAGE and DGCNN. Experimen…
▽ More
In this paper we proposed a Graph-Based conspiracy source detection method for the MediaEval task 2022 FakeNews: Corona Virus and Conspiracies Multimedia Analysis Task. The goal of this study was to apply SOTA graph neural network methods to the problem of misinformation spreading in online social networks. We explore three different Graph Neural Network models: GCN, GraphSAGE and DGCNN. Experimental results demonstrate that DGCNN outperforms in terms of accuracy.
△ Less
Submitted 7 March, 2023;
originally announced March 2023.
-
LunarNav: Crater-based Localization for Long-range Autonomous Lunar Rover Navigation
Authors:
Shreyansh Daftry,
Zhanlin Chen,
Yang Cheng,
Scott Tepsuporn,
Brian Coltin,
Ussama Naam,
Lanssie Mingyue Ma,
Shehryar Khattak,
Matthew Deans,
Larry Matthies
Abstract:
The Artemis program requires robotic and crewed lunar rovers for resource prospecting and exploitation, construction and maintenance of facilities, and human exploration. These rovers must support navigation for 10s of kilometers (km) from base camps. A lunar science rover mission concept - Endurance-A, has been recommended by the new Decadal Survey as the highest priority medium-class mission of…
▽ More
The Artemis program requires robotic and crewed lunar rovers for resource prospecting and exploitation, construction and maintenance of facilities, and human exploration. These rovers must support navigation for 10s of kilometers (km) from base camps. A lunar science rover mission concept - Endurance-A, has been recommended by the new Decadal Survey as the highest priority medium-class mission of the Lunar Discovery and Exploration Program, and would be required to traverse approximately 2000 km in the South Pole-Aitkin (SPA) Basin, with individual drives of several kilometers between stops for downlink. These rover mission scenarios require functionality that provides onboard, autonomous, global position knowledge ( aka absolute localization). However, planetary rovers have no onboard global localization capability to date; they have only used relative localization, by integrating combinations of wheel odometry, visual odometry, and inertial measurements during each drive to track position relative to the start of each drive. In this work, we summarize recent developments from the LunarNav project, where we have developed algorithms and software to enable lunar rovers to estimate their global position and heading on the Moon with a goal performance of position error less than 5 meters (m) and heading error less than 3-degree, 3-sigma, in sunlit areas. This will be achieved autonomously onboard by detecting craters in the vicinity of the rover and matching them to a database of known craters mapped from orbit. The overall technical framework consists of three main elements: 1) crater detection, 2) crater matching, and 3) state estimation. In previous work, we developed crater detection algorithms for three different sensing modalities. Our results suggest that rover localization with an error less than 5 m is highly probable during daytime operations.
△ Less
Submitted 3 January, 2023;
originally announced January 2023.
-
X-ICP: Localizability-Aware LiDAR Registration for Robust Localization in Extreme Environments
Authors:
Turcan Tuna,
Julian Nubert,
Yoshua Nava,
Shehryar Khattak,
Marco Hutter
Abstract:
Modern robotic systems are required to operate in challenging environments, which demand reliable localization under challenging conditions. LiDAR-based localization methods, such as the Iterative Closest Point (ICP) algorithm, can suffer in geometrically uninformative environments that are known to deteriorate point cloud registration performance and push optimization toward divergence along weak…
▽ More
Modern robotic systems are required to operate in challenging environments, which demand reliable localization under challenging conditions. LiDAR-based localization methods, such as the Iterative Closest Point (ICP) algorithm, can suffer in geometrically uninformative environments that are known to deteriorate point cloud registration performance and push optimization toward divergence along weakly constrained directions. To overcome this issue, this work proposes i) a robust fine-grained localizability detection module, and ii) a localizability-aware constrained ICP optimization module, which couples with the localizability detection module in a unified manner. The proposed localizability detection is achieved by utilizing the correspondences between the scan and the map to analyze the alignment strength against the principal directions of the optimization as part of its fine-grained LiDAR localizability analysis. In the second part, this localizability analysis is then integrated into the scan-to-map point cloud registration to generate drift-free pose updates by enforcing controlled updates or leaving the degenerate directions of the optimization unchanged. The proposed method is thoroughly evaluated and compared to state-of-the-art methods in simulated and real-world experiments, demonstrating the performance and reliability improvement in LiDAR-challenging environments. In all experiments, the proposed framework demonstrates accurate and generalizable localizability detection and robust pose estimation without environment-specific parameter tuning.
△ Less
Submitted 18 February, 2024; v1 submitted 29 November, 2022;
originally announced November 2022.
-
LiDAR-guided object search and detection in Subterranean Environments
Authors:
Manthan Patel,
Gabriel Waibel,
Shehryar Khattak,
Marco Hutter
Abstract:
Detecting objects of interest, such as human survivors, safety equipment, and structure access points, is critical to any search-and-rescue operation. Robots deployed for such time-sensitive efforts rely on their onboard sensors to perform their designated tasks. However, as disaster response operations are predominantly conducted under perceptually degraded conditions, commonly utilized sensors s…
▽ More
Detecting objects of interest, such as human survivors, safety equipment, and structure access points, is critical to any search-and-rescue operation. Robots deployed for such time-sensitive efforts rely on their onboard sensors to perform their designated tasks. However, as disaster response operations are predominantly conducted under perceptually degraded conditions, commonly utilized sensors such as visual cameras and LiDARs suffer in terms of performance degradation. In response, this work presents a method that utilizes the complementary nature of vision and depth sensors to leverage multi-modal information to aid object detection at longer distances. In particular, depth and intensity values from sparse LiDAR returns are used to generate proposals for objects present in the environment. These proposals are then utilized by a Pan-Tilt-Zoom (PTZ) camera system to perform a directed search by adjusting its pose and zoom level for performing object detection and classification in difficult environments. The proposed work has been thoroughly verified using an ANYmal quadruped robot in underground settings and on datasets collected during the DARPA Subterranean Challenge finals.
△ Less
Submitted 26 October, 2022;
originally announced October 2022.
-
A Framework for Collaborative Multi-Robot Mapping using Spectral Graph Wavelets
Authors:
Lukas Bernreiter,
Shehryar Khattak,
Lionel Ott,
Roland Siegwart,
Marco Hutter,
Cesar Cadena
Abstract:
The exploration of large-scale unknown environments can benefit from the deployment of multiple robots for collaborative mapping. Each robot explores a section of the environment and communicates onboard pose estimates and maps to a central server to build an optimized global multi-robot map. Naturally, inconsistencies can arise between onboard and server estimates due to onboard odometry drift, f…
▽ More
The exploration of large-scale unknown environments can benefit from the deployment of multiple robots for collaborative mapping. Each robot explores a section of the environment and communicates onboard pose estimates and maps to a central server to build an optimized global multi-robot map. Naturally, inconsistencies can arise between onboard and server estimates due to onboard odometry drift, failures, or degeneracies. The mapping server can correct and overcome such failure cases using computationally expensive operations such as inter-robot loop closure detection and multi-modal mapping. However, the individual robots do not benefit from the collaborative map if the mapping server provides no feedback. Although server updates from the multi-robot map can greatly alleviate the robotic mission strategically, most existing work lacks them, due to their associated computational and bandwidth-related costs. Motivated by this challenge, this paper proposes a novel collaborative mapping framework that enables global mapping consistency among robots and the mapping server. In particular, we propose graph spectral analysis, at different spatial scales, to detect structural differences between robot and server graphs, and to generate necessary constraints for the individual robot pose graphs. Our approach specifically finds the nodes that correspond to the drift's origin rather than the nodes where the error becomes too large. We thoroughly analyze and validate our proposed framework using several real-world multi-robot field deployments where we show improvements of the onboard system up to 90\% and can recover the onboard estimation from localization failures and even from the degeneracies within its estimation.
△ Less
Submitted 2 November, 2022; v1 submitted 25 October, 2022;
originally announced October 2022.
-
Present and Future of SLAM in Extreme Underground Environments
Authors:
Kamak Ebadi,
Lukas Bernreiter,
Harel Biggie,
Gavin Catt,
Yun Chang,
Arghya Chatterjee,
Christopher E. Denniston,
Simon-Pierre Deschênes,
Kyle Harlow,
Shehryar Khattak,
Lucas Nogueira,
Matteo Palieri,
Pavel Petráček,
Matěj Petrlík,
Andrzej Reinke,
Vít Krátký,
Shibo Zhao,
Ali-akbar Agha-mohammadi,
Kostas Alexis,
Christoffer Heckman,
Kasra Khosoussi,
Navinda Kottege,
Benjamin Morrell,
Marco Hutter,
Fred Pauling
, et al. (6 additional authors not shown)
Abstract:
This paper reports on the state of the art in underground SLAM by discussing different SLAM strategies and results across six teams that participated in the three-year-long SubT competition. In particular, the paper has four main goals. First, we review the algorithms, architectures, and systems adopted by the teams; particular emphasis is put on lidar-centric SLAM solutions (the go-to approach fo…
▽ More
This paper reports on the state of the art in underground SLAM by discussing different SLAM strategies and results across six teams that participated in the three-year-long SubT competition. In particular, the paper has four main goals. First, we review the algorithms, architectures, and systems adopted by the teams; particular emphasis is put on lidar-centric SLAM solutions (the go-to approach for virtually all teams in the competition), heterogeneous multi-robot operation (including both aerial and ground robots), and real-world underground operation (from the presence of obscurants to the need to handle tight computational constraints). We do not shy away from discussing the dirty details behind the different SubT SLAM systems, which are often omitted from technical papers. Second, we discuss the maturity of the field by highlighting what is possible with the current SLAM systems and what we believe is within reach with some good systems engineering. Third, we outline what we believe are fundamental open problems, that are likely to require further research to break through. Finally, we provide a list of open-source SLAM implementations and datasets that have been produced during the SubT challenge and related efforts, and constitute a useful resource for researchers and practitioners.
△ Less
Submitted 2 August, 2022;
originally announced August 2022.
-
Team CERBERUS Wins the DARPA Subterranean Challenge: Technical Overview and Lessons Learned
Authors:
Marco Tranzatto,
Mihir Dharmadhikari,
Lukas Bernreiter,
Marco Camurri,
Shehryar Khattak,
Frank Mascarich,
Patrick Pfreundschuh,
David Wisth,
Samuel Zimmermann,
Mihir Kulkarni,
Victor Reijgwart,
Benoit Casseau,
Timon Homberger,
Paolo De Petris,
Lionel Ott,
Wayne Tubby,
Gabriel Waibel,
Huan Nguyen,
Cesar Cadena,
Russell Buchanan,
Lorenz Wellhausen,
Nikhil Khedekar,
Olov Andersson,
Lintong Zhang,
Takahiro Miki
, et al. (11 additional authors not shown)
Abstract:
This article presents the CERBERUS robotic system-of-systems, which won the DARPA Subterranean Challenge Final Event in 2021. The Subterranean Challenge was organized by DARPA with the vision to facilitate the novel technologies necessary to reliably explore diverse underground environments despite the grueling challenges they present for robotic autonomy. Due to their geometric complexity, degrad…
▽ More
This article presents the CERBERUS robotic system-of-systems, which won the DARPA Subterranean Challenge Final Event in 2021. The Subterranean Challenge was organized by DARPA with the vision to facilitate the novel technologies necessary to reliably explore diverse underground environments despite the grueling challenges they present for robotic autonomy. Due to their geometric complexity, degraded perceptual conditions combined with lack of GPS support, austere navigation conditions, and denied communications, subterranean settings render autonomous operations particularly demanding. In response to this challenge, we developed the CERBERUS system which exploits the synergy of legged and flying robots, coupled with robust control especially for overcoming perilous terrain, multi-modal and multi-robot perception for localization and mapping in conditions of sensor degradation, and resilient autonomy through unified exploration path planning and local motion planning that reflects robot-specific limitations. Based on its ability to explore diverse underground environments and its high-level command and control by a single human supervisor, CERBERUS demonstrated efficient exploration, reliable detection of objects of interest, and accurate mapping. In this article, we report results from both the preliminary runs and the final Prize Round of the DARPA Subterranean Challenge, and discuss highlights and challenges faced, alongside lessons learned for the benefit of the community.
△ Less
Submitted 11 July, 2022;
originally announced July 2022.
-
Marsupial Walking-and-Flying Robotic Deployment for Collaborative Exploration of Unknown Environments
Authors:
Paolo De Petris,
Shehryar Khattak,
Mihir Dharmadhikari,
Gabriel Waibel,
Huan Nguyen,
Markus Montenegro,
Nikhil Khedekar,
Kostas Alexis,
Marco Hutter
Abstract:
This work contributes a marsupial robotic system-of-systems involving a legged and an aerial robot capable of collaborative mapping and exploration path planning that exploits the heterogeneous properties of the two systems and the ability to selectively deploy the aerial system from the ground robot. Exploiting the dexterous locomotion capabilities and long endurance of quadruped robots, the mars…
▽ More
This work contributes a marsupial robotic system-of-systems involving a legged and an aerial robot capable of collaborative mapping and exploration path planning that exploits the heterogeneous properties of the two systems and the ability to selectively deploy the aerial system from the ground robot. Exploiting the dexterous locomotion capabilities and long endurance of quadruped robots, the marsupial combination can explore within large-scale and confined environments involving rough terrain. However, as certain types of terrain or vertical geometries can render any ground system unable to continue its exploration, the marsupial system can - when needed - deploy the flying robot which, by exploiting its 3D navigation capabilities, can undertake a focused exploration task within its endurance limitations. Focusing on autonomy, the two systems can co-localize and map together by sharing LiDAR-based maps and plan exploration paths individually, while a tailored graph search onboard the legged robot allows it to identify where and when the ferried aerial platform should be deployed. The system is verified within multiple experimental studies demonstrating the expanded exploration capabilities of the marsupial system-of-systems and facilitating the exploration of otherwise individually unreachable areas.
△ Less
Submitted 11 May, 2022;
originally announced May 2022.
-
Locomotion Policy Guided Traversability Learning using Volumetric Representations of Complex Environments
Authors:
Jonas Frey,
David Hoeller,
Shehryar Khattak,
Marco Hutter
Abstract:
Despite the progress in legged robotic locomotion, autonomous navigation in unknown environments remains an open problem. Ideally, the navigation system utilizes the full potential of the robots' locomotion capabilities while operating within safety limits under uncertainty. The robot must sense and analyze the traversability of the surrounding terrain, which depends on the hardware, locomotion co…
▽ More
Despite the progress in legged robotic locomotion, autonomous navigation in unknown environments remains an open problem. Ideally, the navigation system utilizes the full potential of the robots' locomotion capabilities while operating within safety limits under uncertainty. The robot must sense and analyze the traversability of the surrounding terrain, which depends on the hardware, locomotion control, and terrain properties. It may contain information about the risk, energy, or time consumption needed to traverse the terrain. To avoid hand-crafted traversability cost functions we propose to collect traversability information about the robot and locomotion policy by simulating the traversal over randomly generated terrains using a physics simulator. Thousand of robots are simulated in parallel controlled by the same locomotion policy used in reality to acquire 57 years of real-world locomotion experience equivalent. For deployment on the real robot, a sparse convolutional network is trained to predict the simulated traversability cost, which is tailored to the deployed locomotion policy, from an entirely geometric representation of the environment in the form of a 3D voxel-occupancy map. This representation avoids the need for commonly used elevation maps, which are error-prone in the presence of overhanging obstacles and multi-floor or low-ceiling scenarios. The effectiveness of the proposed traversability prediction network is demonstrated for path planning for the legged robot ANYmal in various indoor and natural environments.
△ Less
Submitted 21 August, 2022; v1 submitted 29 March, 2022;
originally announced March 2022.
-
Learning-based Localizability Estimation for Robust LiDAR Localization
Authors:
Julian Nubert,
Etienne Walther,
Shehryar Khattak,
Marco Hutter
Abstract:
LiDAR-based localization and mapping is one of the core components in many modern robotic systems due to the direct integration of range and geometry, allowing for precise motion estimation and generation of high quality maps in real-time. Yet, as a consequence of insufficient environmental constraints present in the scene, this dependence on geometry can result in localization failure, happening…
▽ More
LiDAR-based localization and mapping is one of the core components in many modern robotic systems due to the direct integration of range and geometry, allowing for precise motion estimation and generation of high quality maps in real-time. Yet, as a consequence of insufficient environmental constraints present in the scene, this dependence on geometry can result in localization failure, happening in self-symmetric surroundings such as tunnels. This work addresses precisely this issue by proposing a neural network-based estimation approach for detecting (non-)localizability during robot operation. Special attention is given to the localizability of scan-to-scan registration, as it is a crucial component in many LiDAR odometry estimation pipelines. In contrast to previous, mostly traditional detection approaches, the proposed method enables early detection of failure by estimating the localizability on raw sensor measurements without evaluating the underlying registration optimization. Moreover, previous approaches remain limited in their ability to generalize across environments and sensor types, as heuristic-tuning of degeneracy detection thresholds is required. The proposed approach avoids this problem by learning from a collection of different environments, allowing the network to function over various scenarios. Furthermore, the network is trained exclusively on simulated data, avoiding arduous data collection in challenging and degenerate, often hard-to-access, environments. The presented method is tested during field experiments conducted across challenging environments and on two different sensor types without any modifications. The observed detection performance is on par with state-of-the-art methods after environment-specific threshold tuning.
△ Less
Submitted 1 August, 2022; v1 submitted 10 March, 2022;
originally announced March 2022.
-
Graph-based Multi-sensor Fusion for Consistent Localization of Autonomous Construction Robots
Authors:
Julian Nubert,
Shehryar Khattak,
Marco Hutter
Abstract:
Enabling autonomous operation of large-scale construction machines, such as excavators, can bring key benefits for human safety and operational opportunities for applications in dangerous and hazardous environments. To facilitate robot autonomy, robust and accurate state-estimation remains a core component to enable these machines for operation in a diverse set of complex environments. In this wor…
▽ More
Enabling autonomous operation of large-scale construction machines, such as excavators, can bring key benefits for human safety and operational opportunities for applications in dangerous and hazardous environments. To facilitate robot autonomy, robust and accurate state-estimation remains a core component to enable these machines for operation in a diverse set of complex environments. In this work, a method for multi-modal sensor fusion for robot state-estimation and localization is presented, enabling operation of construction robots in real-world scenarios. The proposed approach presents a graph-based prediction-update loop that combines the benefits of filtering and smoothing in order to provide consistent state estimates at high update rate, while maintaining accurate global localization for large-scale earth-moving excavators. Furthermore, the proposed approach enables a flexible integration of asynchronous sensor measurements and provides consistent pose estimates even during phases of sensor dropout. For this purpose, a dual-graph design for switching between two distinct optimization problems is proposed, directly addressing temporary failure and the subsequent return of global position estimates. The proposed approach is implemented on-board two Menzi Muck walking excavators and validated during real-world tests conducted in representative operational environments.
△ Less
Submitted 2 March, 2022;
originally announced March 2022.
-
Collaborative Robot Mapping using Spectral Graph Analysis
Authors:
Lukas Bernreiter,
Shehryar Khattak,
Lionel Ott,
Roland Siegwart,
Marco Hutter,
Cesar Cadena
Abstract:
In this paper, we deal with the problem of creating globally consistent pose graphs in a centralized multi-robot SLAM framework. For each robot to act autonomously, individual onboard pose estimates and maps are maintained, which are then communicated to a central server to build an optimized global map. However, inconsistencies between onboard and server estimates can occur due to onboard odometr…
▽ More
In this paper, we deal with the problem of creating globally consistent pose graphs in a centralized multi-robot SLAM framework. For each robot to act autonomously, individual onboard pose estimates and maps are maintained, which are then communicated to a central server to build an optimized global map. However, inconsistencies between onboard and server estimates can occur due to onboard odometry drift or failure. Furthermore, robots do not benefit from the collaborative map if the server provides no feedback in a computationally tractable and bandwidth-efficient manner. Motivated by this challenge, this paper proposes a novel collaborative mapping framework to enable accurate global mapping among robots and server. In particular, structural differences between robot and server graphs are exploited at different spatial scales using graph spectral analysis to generate necessary constraints for the individual robot pose graphs. The proposed approach is thoroughly analyzed and validated using several real-world multi-robot field deployments where we show improvements of the onboard system up to 90%.
△ Less
Submitted 1 March, 2022;
originally announced March 2022.
-
CERBERUS: Autonomous Legged and Aerial Robotic Exploration in the Tunnel and Urban Circuits of the DARPA Subterranean Challenge
Authors:
Marco Tranzatto,
Frank Mascarich,
Lukas Bernreiter,
Carolina Godinho,
Marco Camurri,
Shehryar Khattak,
Tung Dang,
Victor Reijgwart,
Johannes Loeje,
David Wisth,
Samuel Zimmermann,
Huan Nguyen,
Marius Fehr,
Lukas Solanka,
Russell Buchanan,
Marko Bjelonic,
Nikhil Khedekar,
Mathieu Valceschini,
Fabian Jenelten,
Mihir Dharmadhikari,
Timon Homberger,
Paolo De Petris,
Lorenz Wellhausen,
Mihir Kulkarni,
Takahiro Miki
, et al. (16 additional authors not shown)
Abstract:
Autonomous exploration of subterranean environments constitutes a major frontier for robotic systems as underground settings present key challenges that can render robot autonomy hard to achieve. This has motivated the DARPA Subterranean Challenge, where teams of robots search for objects of interest in various underground environments. In response, the CERBERUS system-of-systems is presented as a…
▽ More
Autonomous exploration of subterranean environments constitutes a major frontier for robotic systems as underground settings present key challenges that can render robot autonomy hard to achieve. This has motivated the DARPA Subterranean Challenge, where teams of robots search for objects of interest in various underground environments. In response, the CERBERUS system-of-systems is presented as a unified strategy towards subterranean exploration using legged and flying robots. As primary robots, ANYmal quadruped systems are deployed considering their endurance and potential to traverse challenging terrain. For aerial robots, both conventional and collision-tolerant multirotors are utilized to explore spaces too narrow or otherwise unreachable by ground systems. Anticipating degraded sensing conditions, a complementary multi-modal sensor fusion approach utilizing camera, LiDAR, and inertial data for resilient robot pose estimation is proposed. Individual robot pose estimates are refined by a centralized multi-robot map optimization approach to improve the reported location accuracy of detected objects of interest in the DARPA-defined coordinate frame. Furthermore, a unified exploration path planning policy is presented to facilitate the autonomous operation of both legged and aerial robots in complex underground networks. Finally, to enable communication between the robots and the base station, CERBERUS utilizes a ground rover with a high-gain antenna and an optical fiber connection to the base station, alongside breadcrumbing of wireless nodes by our legged robots. We report results from the CERBERUS system-of-systems deployment at the DARPA Subterranean Challenge Tunnel and Urban Circuits, along with the current limitations and the lessons learned for the benefit of the community.
△ Less
Submitted 18 January, 2022;
originally announced January 2022.
-
Self-supervised Learning of LiDAR Odometry for Robotic Applications
Authors:
Julian Nubert,
Shehryar Khattak,
Marco Hutter
Abstract:
Reliable robot pose estimation is a key building block of many robot autonomy pipelines, with LiDAR localization being an active research domain. In this work, a versatile self-supervised LiDAR odometry estimation method is presented, in order to enable the efficient utilization of all available LiDAR data while maintaining real-time performance. The proposed approach selectively applies geometric…
▽ More
Reliable robot pose estimation is a key building block of many robot autonomy pipelines, with LiDAR localization being an active research domain. In this work, a versatile self-supervised LiDAR odometry estimation method is presented, in order to enable the efficient utilization of all available LiDAR data while maintaining real-time performance. The proposed approach selectively applies geometric losses during training, being cognizant of the amount of information that can be extracted from scan points. In addition, no labeled or ground-truth data is required, hence making the presented approach suitable for pose estimation in applications where accurate ground-truth is difficult to obtain. Furthermore, the presented network architecture is applicable to a wide range of environments and sensor modalities without requiring any network or loss function adjustments. The proposed approach is thoroughly tested for both indoor and outdoor real-world applications through a variety of experiments using legged, tracked and wheeled robots, demonstrating the suitability of learning-based LiDAR odometry for complex robotic applications.
△ Less
Submitted 25 June, 2021; v1 submitted 10 November, 2020;
originally announced November 2020.
-
A Doubly Regularized Linear Discriminant Analysis Classifier with Automatic Parameter Selection
Authors:
Alam Zaib,
Tarig Ballal,
Shahid Khattak,
Tareq Y. Al-Naffouri
Abstract:
Linear discriminant analysis (LDA) based classifiers tend to falter in many practical settings where the training data size is smaller than, or comparable to, the number of features. As a remedy, different regularized LDA (RLDA) methods have been proposed. These methods may still perform poorly depending on the size and quality of the available training data. In particular, the test data deviation…
▽ More
Linear discriminant analysis (LDA) based classifiers tend to falter in many practical settings where the training data size is smaller than, or comparable to, the number of features. As a remedy, different regularized LDA (RLDA) methods have been proposed. These methods may still perform poorly depending on the size and quality of the available training data. In particular, the test data deviation from the training data model, for example, due to noise contamination, can cause severe performance degradation. Moreover, these methods commit further to the Gaussian assumption (upon which LDA is established) to tune their regularization parameters, which may compromise accuracy when dealing with real data. To address these issues, we propose a doubly regularized LDA classifier that we denote as R2LDA. In the proposed R2LDA approach, the RLDA score function is converted into an inner product of two vectors. By substituting the expressions of the regularized estimators of these vectors, we obtain the R2LDA score function that involves two regularization parameters. To set the values of these parameters, we adopt three existing regularization techniques; the constrained perturbation regularization approach (COPRA), the bounded perturbation regularization (BPR) algorithm, and the generalized cross-validation (GCV) method. These methods are used to tune the regularization parameters based on linear estimation models, with the sample covariance matrix's square root being the linear operator. Results obtained from both synthetic and real data demonstrate the consistency and effectiveness of the proposed R2LDA approach, especially in scenarios involving test data contaminated with noise that is not observed during the training phase.
△ Less
Submitted 27 March, 2021; v1 submitted 28 April, 2020;
originally announced April 2020.
-
Pioneer dataset and automatic recognition of Urdu handwritten characters using a deep autoencoder and convolutional neural network
Authors:
Hazrat Ali,
Ahsan Ullah,
Talha Iqbal,
Shahid Khattak
Abstract:
Automatic recognition of Urdu handwritten digits and characters, is a challenging task. It has applications in postal address reading, bank's cheque processing, and digitization and preservation of handwritten manuscripts from old ages. While there exists a significant work for automatic recognition of handwritten English characters and other major languages of the world, the work done for Urdu la…
▽ More
Automatic recognition of Urdu handwritten digits and characters, is a challenging task. It has applications in postal address reading, bank's cheque processing, and digitization and preservation of handwritten manuscripts from old ages. While there exists a significant work for automatic recognition of handwritten English characters and other major languages of the world, the work done for Urdu lan-guage is extremely insufficient. This paper has two goals. Firstly, we introduce a pioneer dataset for handwritten digits and characters of Urdu, containing samples from more than 900 individuals. Secondly, we report results for automatic recog-nition of handwritten digits and characters as achieved by using deep auto-encoder network and convolutional neural network. More specifically, we use a two-layer and a three-layer deep autoencoder network and convolutional neural network and evaluate the two frameworks in terms of recognition accuracy. The proposed framework of deep autoencoder can successfully recognize digits and characters with an accuracy of 97% for digits only, 81% for characters only and 82% for both digits and characters simultaneously. In comparison, the framework of convolutional neural network has accuracy of 96.7% for digits only, 86.5% for characters only and 82.7% for both digits and characters simultaneously. These frameworks can serve as baselines for future research on Urdu handwritten text.
△ Less
Submitted 17 December, 2019;
originally announced December 2019.
-
Vision-Depth Landmarks and Inertial Fusion for Navigation in Degraded Visual Environments
Authors:
Shehryar Khattak,
Christos Papachristos,
Kostas Alexis
Abstract:
This paper proposes a method for tight fusion of visual, depth and inertial data in order to extend robotic capabilities for navigation in GPS-denied, poorly illuminated, and texture-less environments. Visual and depth information are fused at the feature detection and descriptor extraction levels to augment one sensing modality with the other. These multimodal features are then further integrated…
▽ More
This paper proposes a method for tight fusion of visual, depth and inertial data in order to extend robotic capabilities for navigation in GPS-denied, poorly illuminated, and texture-less environments. Visual and depth information are fused at the feature detection and descriptor extraction levels to augment one sensing modality with the other. These multimodal features are then further integrated with inertial sensor cues using an extended Kalman filter to estimate the robot pose, sensor bias terms, and landmark positions simultaneously as part of the filter state. As demonstrated through a set of hand-held and Micro Aerial Vehicle experiments, the proposed algorithm is shown to perform reliably in challenging visually-degraded environments using RGB-D information from a lightweight and low-cost sensor and data from an IMU.
△ Less
Submitted 4 March, 2019;
originally announced March 2019.
-
Visual-Thermal Landmarks and Inertial Fusion for Navigation in Degraded Visual Environments
Authors:
Shehryar Khattak,
Christos Papachristos,
Kostas Alexis
Abstract:
With an ever-widening domain of aerial robotic applications, including many mission critical tasks such as disaster response operations, search and rescue missions and infrastructure inspections taking place in GPS-denied environments, the need for reliable autonomous operation of aerial robots has become crucial. Operating in GPS-denied areas aerial robots rely on a multitude of sensors to locali…
▽ More
With an ever-widening domain of aerial robotic applications, including many mission critical tasks such as disaster response operations, search and rescue missions and infrastructure inspections taking place in GPS-denied environments, the need for reliable autonomous operation of aerial robots has become crucial. Operating in GPS-denied areas aerial robots rely on a multitude of sensors to localize and navigate. Visible spectrum cameras are the most commonly used sensors due to their low cost and weight. However, in environments that are visually-degraded such as in conditions of poor illumination, low texture, or presence of obscurants including fog, smoke and dust, the reliability of visible light cameras deteriorates significantly. Nevertheless, maintaining reliable robot navigation in such conditions is essential. In contrast to visible light cameras, thermal cameras offer visibility in the infrared spectrum and can be used in a complementary manner with visible spectrum cameras for robot localization and navigation tasks, without paying the significant weight and power penalty typically associated with carrying other sensors. Exploiting this fact, in this work we present a multi-sensor fusion algorithm for reliable odometry estimation in GPS-denied and degraded visual environments. The proposed method utilizes information from both the visible and thermal spectra for landmark selection and prioritizes feature extraction from informative image regions based on a metric over spatial entropy. Furthermore, inertial sensing cues are integrated to improve the robustness of the odometry estimation process. To verify our solution, a set of challenging experiments were conducted inside a) an obscurant filed machine shop-like industrial environment, as well as b) a dark subterranean mine in the presence of heavy airborne dust.
△ Less
Submitted 4 March, 2019;
originally announced March 2019.
-
Keyframe-based Direct Thermal-Inertial Odometry
Authors:
Shehryar Khattak,
Christos Papachristos,
Kostas Alexis
Abstract:
This paper proposes an approach for fusing direct radiometric data from a thermal camera with inertial measurements to extend the robotic capabilities of aerial robots for navigation in GPS-denied and visually degraded environments in the conditions of darkness and in the presence of airborne obscurants such as dust, fog and smoke. An optimization based approach is developed that jointly minimizes…
▽ More
This paper proposes an approach for fusing direct radiometric data from a thermal camera with inertial measurements to extend the robotic capabilities of aerial robots for navigation in GPS-denied and visually degraded environments in the conditions of darkness and in the presence of airborne obscurants such as dust, fog and smoke. An optimization based approach is developed that jointly minimizes the re-projection error of 3D landmarks and inertial measurement errors. The developed solution is extensively verified against both ground-truth in an indoor laboratory setting, as well as inside an underground mine under severely visually degraded conditions.
△ Less
Submitted 2 March, 2019;
originally announced March 2019.
-
Marker based Thermal-Inertial Localization for Aerial Robots in Obscurant Filled Environments
Authors:
Shehryar Khattak,
Christos Papachristos,
Kostas Alexis
Abstract:
For robotic inspection tasks in known environments fiducial markers provide a reliable and low-cost solution for robot localization. However, detection of such markers relies on the quality of RGB camera data, which degrades significantly in the presence of visual obscurants such as fog and smoke. The ability to navigate known environments in the presence of obscurants can be critical for inspecti…
▽ More
For robotic inspection tasks in known environments fiducial markers provide a reliable and low-cost solution for robot localization. However, detection of such markers relies on the quality of RGB camera data, which degrades significantly in the presence of visual obscurants such as fog and smoke. The ability to navigate known environments in the presence of obscurants can be critical for inspection tasks especially, in the aftermath of a disaster. Addressing such a scenario, this work proposes a method for the design of fiducial markers to be used with thermal cameras for the pose estimation of aerial robots. Our low cost markers are designed to work in the long wave infrared spectrum, which is not affected by the presence of obscurants, and can be affixed to any object that has measurable temperature difference with respect to its surroundings. Furthermore, the estimated pose from the fiducial markers is fused with inertial measurements in an extended Kalman filter to remove high frequency noise and error present in the fiducial pose estimates. The proposed markers and the pose estimation method are experimentally evaluated in an obscurant filled environment using an aerial robot carrying a thermal camera.
△ Less
Submitted 2 March, 2019;
originally announced March 2019.
-
Visual-Inertial Odometry-enhanced Geometrically Stable ICP for Mapping Applications using Aerial Robots
Authors:
Tung Dang,
Shehryar Khattak,
Christos Papachristos,
Kostas Alexis
Abstract:
This paper presents a visual-inertial odometry-enhanced geometrically stable Iterative Closest Point (ICP) algorithm for accurate mapping using aerial robots. The proposed method employs a visual-inertial odometry framework in order to provide robust priors to the ICP step and calculate the overlap among point clouds derived from an onboard time-of-flight depth sensor. Within the overlapping parts…
▽ More
This paper presents a visual-inertial odometry-enhanced geometrically stable Iterative Closest Point (ICP) algorithm for accurate mapping using aerial robots. The proposed method employs a visual-inertial odometry framework in order to provide robust priors to the ICP step and calculate the overlap among point clouds derived from an onboard time-of-flight depth sensor. Within the overlapping parts of the point clouds, the method samples points such that the distribution of normals among them is as large as possible. As different geometries and sensor trajectories will influence the performance of the alignment process, evaluation of the expected geometric stability of the ICP step is conducted. It is only when this test is successful that the matching, outlier rejection, and minimization of the error metric ICP steps are conducted and the new relative translation and rotational components are estimated, otherwise the system relies on the visual-inertial odometry transformation estimates. The proposed strategy was evaluated within handheld, automated and fully autonomous exploration and mapping missions using a small aerial robot and was shown to provide robust results of superior quality at an affordable increase of the computational load.
△ Less
Submitted 28 January, 2018; v1 submitted 24 January, 2018;
originally announced January 2018.
-
Towards Robotically Supported Decommissioning of Nuclear Sites
Authors:
Frank Mascarich,
Taylor Wilson,
Tung Dang,
Shehryar Khattak,
Christos Papachristos,
Kostas Alexis
Abstract:
This paper overviews certain radiation detection, perception, and planning challenges for nuclearized robotics that aim to support the waste management and decommissioning mission. To enable the autonomous monitoring, inspection and multi-modal characterization of nuclear sites, we discuss important problems relevant to the tasks of navigation in degraded visual environments, localizability-aware…
▽ More
This paper overviews certain radiation detection, perception, and planning challenges for nuclearized robotics that aim to support the waste management and decommissioning mission. To enable the autonomous monitoring, inspection and multi-modal characterization of nuclear sites, we discuss important problems relevant to the tasks of navigation in degraded visual environments, localizability-aware exploration and mapping without any prior knowledge of the environment, as well as robotic radiation detection. Future contributions will focus on each of the relevant problems, will aim to deliver a comprehensive multi-modal mapping result, and will emphasize on extensive field evaluation and system verification.
△ Less
Submitted 17 May, 2017;
originally announced May 2017.
-
Ad-Blocking and Counter Blocking: A Slice of the Arms Race
Authors:
Rishab Nithyanand,
Sheharbano Khattak,
Mobin Javed,
Narseo Vallina-Rodriguez,
Marjan Falahrastegar,
Julia E. Powles,
Emiliano De Cristofaro,
Hamed Haddadi,
Steven J. Murdoch
Abstract:
Adblocking tools like Adblock Plus continue to rise in popularity, potentially threatening the dynamics of advertising revenue streams. In response, a number of publishers have ramped up efforts to develop and deploy mechanisms for detecting and/or counter-blocking adblockers (which we refer to as anti-adblockers), effectively escalating the online advertising arms race. In this paper, we develop…
▽ More
Adblocking tools like Adblock Plus continue to rise in popularity, potentially threatening the dynamics of advertising revenue streams. In response, a number of publishers have ramped up efforts to develop and deploy mechanisms for detecting and/or counter-blocking adblockers (which we refer to as anti-adblockers), effectively escalating the online advertising arms race. In this paper, we develop a scalable approach for identifying third-party services shared across multiple web-sites and use it to provide a first characterization of anti-adblocking across the Alexa Top-5K websites. We map websites that perform anti-adblocking as well as the entities that provide anti-adblocking scripts. We study the modus operandi of these scripts and their impact on popular adblockers. We find that at least 6.7% of websites in the Alexa Top-5K use anti-adblocking scripts, acquired from 12 distinct entities -- some of which have a direct interest in nourishing the online advertising industry.
△ Less
Submitted 20 July, 2016; v1 submitted 17 May, 2016;
originally announced May 2016.
-
Systemization of Pluggable Transports for Censorship Resistance
Authors:
Sheharbano Khattak,
Laurent Simon,
Steven J. Murdoch
Abstract:
An increasing number of countries implement Internet censorship at different scales and for a variety of reasons. In particular, the link between the censored client and entry point to the uncensored network is a frequent target of censorship due to the ease with which a nation-state censor can control it. A number of censorship resistance systems have been developed thus far to help circumvent bl…
▽ More
An increasing number of countries implement Internet censorship at different scales and for a variety of reasons. In particular, the link between the censored client and entry point to the uncensored network is a frequent target of censorship due to the ease with which a nation-state censor can control it. A number of censorship resistance systems have been developed thus far to help circumvent blocking on this link, which we refer to as link circumvention systems (LCs). The variety and profusion of attack vectors available to a censor has led to an arms race, leading to a dramatic speed of evolution of LCs. Despite their inherent complexity and the breadth of work in this area, there is no systematic way to evaluate link circumvention systems and compare them against each other. In this paper, we (i) sketch an attack model to comprehensively explore a censor's capabilities, (ii) present an abstract model of a LC, a system that helps a censored client communicate with a server over the Internet while resisting censorship, (iii) describe an evaluation stack that underscores a layered approach to evaluate LCs, and (iv) systemize and evaluate existing censorship resistance systems that provide link circumvention. We highlight open challenges in the evaluation and development of LCs and discuss possible mitigations.
△ Less
Submitted 29 July, 2016; v1 submitted 23 December, 2014;
originally announced December 2014.
-
Modeling Probability of Path Loss for DSDV, OLSR and DYMO above 802.11 and 802.11p
Authors:
S. N. Mohammad,
S. Wasiq,
W. Arshad,
N. Javaid,
S. Khattak,
M. J. Ashraf
Abstract:
This paper presents path loss model along with framework for probability distribution function for VANETs. Furthermore, we simulate three routing protocols Destination Sequenced Distance Vector (DSDV), Optimized Link State Routing (OLSR) and Dynamic MANET On-demand (DYMO) in NS-2 to evaluate and compare their performance using two Mac-layer Protocols 802.11 and 802.11p. A novel approach of this wo…
▽ More
This paper presents path loss model along with framework for probability distribution function for VANETs. Furthermore, we simulate three routing protocols Destination Sequenced Distance Vector (DSDV), Optimized Link State Routing (OLSR) and Dynamic MANET On-demand (DYMO) in NS-2 to evaluate and compare their performance using two Mac-layer Protocols 802.11 and 802.11p. A novel approach of this work is modifications in existing parameters to achieve high efficiency. After extensive simulations, we observe that DSDV out performs with 802.11p while DYMO gives best performance with 802.11.
△ Less
Submitted 27 July, 2013;
originally announced July 2013.
