Holly Dinkel is pursuing the Ph.D. in aerospace engineering at the University of Illinois Urbana-Champaign where she researches robotic caretaking as a NASA Space Technology Graduate Research Fellow with the NASA Ames Research Center Intelligent Robotics Group and the NASA Johnson Space Center Dexterous Robotics Laboratory. She previously completed the M.S. in aeronautics and astronautics and a certificate in entrepreneurship at Stanford University and the B.S. and B.A. in chemical engineering and music, respectively, at the University of Missouri, Columbia.
This work demonstrates the reduction of Angle Random Walk (ARW), Velocity Random Walk (VRW), and Rate Random Walk (RRW) through sensor fusion of complementary distributed Inertial Measurement Units (IMUs) for robot and satellite navigation. High-fidelity IMUs may violate the tight cost, mass, power, and volume constraints typical of small robot and satellite platforms. Low-cost Micro-Electromechanical System (MEMS) IMUs may have performance characteristics unsuitable for attitude estimation when used alone in a system. In this work, the measurements of multiple MEMS IMUs are fused into a virtual IMU. The fusion algorithm takes time-synchronized measurements from an arbitrary number of sensors and produce one virtual measurement estimate. Allan variance is used to evaluate ARW, VRW, and RRW noise performance for the estimated angular velocity and linear acceleration measurements. The noise performance of the virtual sensor measurement estimates and of the single-sensor measurements is compared. Sensor fusion is demonstrated with the simulated data of up to 18 MEMS IMUs and for real data collected by 5 MEMS IMUs. A tradeoff between number of sensors and noise reduction performance is presented to aid the design of future scalable, distributed inertial measurement systems.
2024
Image-Based Pose Estimation of Sub-Centimeter Industrial Parts for Automated Assembly
This work adapts and integrates existing machine vision techniques to estimate the 6DoF pose of sub-centimeter parts for high-mix, low-volume assembly lines, focusing on the challenges of accurate positioning in real-world scenarios. In this system, the 3D models of each part are input to a BlenderProc2 rendering engine to generate a physically- and photometrically-realistic synthetic image dataset. Synthetic images are used to train a Mask R-CNN model for segmenting individual part instances in a scene, with automatically-generated instance mask labels, eliminating the need for manual labeling. Instance segmentation enables part selection for assembly when multiple parts are present. Additionally, a PVNet model is trained on cropped images of each part instance to estimate their positions and orientations. An additional pose refinement step adjusts PVNet pose estimates by aligning the orientation to the nearest physically-stable configuration on a planar surface and refining the translation using calibrated object-to-camera distances from the workspace. To evaluate robustness, noise is injected into the keypoint detection stage of the PVNet model in an ablation study to assess the impact of sensor noise on pose estimation. Real robot pick-and-place experiments demonstrate the system performance.
Business Innovation in Commercial Space: Culture and Trends in Earth Observation
This work studies technological, business, and cultural development practices in the commercial Earth Observation (EO) sector. Increased access to space combined with growing demands for climate and natural resource monitoring, situational awareness, on-demand analytics, and sovereign data independence motivate the expansion of EO. The financial performance and culture of five public EO companies–iQPS, GOMSpace, Kleos Space, Planet Labs, and Satellogic–are analyzed. Time-series econometrics modeling indicates trends in how financial resources impact earnings and revenue, and the models are used to forecast future earnings and revenue with a high goodness-of-fit. A time-series DuPont analysis investigates how companies operate through conversion of different financial instruments, and corresponding corporate milestones contextualize these financial transactions. Informational interviews of four EO company executives provide clues about company culture and decision-making strategies. Recommendations tailored to the unique challenges and opportunities of the global EO sector are provided to help government partners, investors, and program managers estimate business performance given financial and cultural data.
AstrobeeCD: Change Detection in Microgravity with Free-Flying Robots
This work presents AstrobeeCD, a system for 3D scene change detection toward near-real-time environmental awareness of space outposts using the Astrobee free-flying robot in microgravity. Assistive free-flyer robots autonomously caring for future crewed space habitats must be able to detect day-to-day interior changes to track inventory, detect and diagnose faults, and monitor the outpost status. A set of image and depth data from one time step is used to reconstruct a 3D model of the environment. The 3D model is used as the basis for comparison for free-flyer environment surveys at future time steps, where an image-based change detection algorithm identifies inconsistencies against the 3D model. Change detection is demonstrated using real image and pose data collected by an Astrobee robot in a test environment on Earth at NASA Ames Research Center and from microgravity aboard the International Space Station. Change detection computation time and performance are quantitatively evaluated on the test data captured on Earth, and it identifies scene changes more quickly than a point cloud clustering-based algorithm applied to data from the same surveys.
This work presents KnotDLO, a method for one-handed knot manipulation to transform a Deformable Linear Object (DLO) into knot states. Grasp and target waypoints for future DLO states are planned from the current DLO shape. Grasp poses are computed from indexing the tracked piecewise linear curve representing the DLO state based on the current curve shape and are piecewise continuous. Intermediate waypoints are computed from the geometry of the current DLO state and the desired next state. The perception and manipulation system is robust to occlusion, repeatable for varying rope initial configurations, interpretable for generating motion policies, and requires no human demonstrations or training. The system decouples visual reasoning from control. In 16 trials of knot tying, KnotDLO achieves a 50% success rate in tying an overhand knot from previously unseen configurations.
TrackDLO: Tracking Deformable Linear Objects Under Occlusion with Motion Coherence
The TrackDLO algorithm estimates the shape of a Deformable Linear Object (DLO) under occlusion from a sequence of RGB-D images. TrackDLO is vision-only and runs in real-time. It requires no external state information from physics modeling, simulation, visual markers, or contact as input. The algorithm improves on previous approaches by addressing three common scenarios which cause tracking failure: tip occlusion, mid-section occlusion, and self-occlusion. This is achieved through the application of Motion Coherence Theory to impute the spatial velocity of occluded nodes, the use of the topological geodesic distance to track self-occluding DLOs, and the introduction of a non-Gaussian kernel that only penalizes lower-order spatial displacement derivatives to reflect DLO physics. Improved real-time DLO tracking under mid-section occlusion, tip occlusion, and self-occlusion is demonstrated experimentally. The source code and demonstration data are publicly released.
Unsupervised Change Detection for Space Habitats Using 3D Point Clouds
This work presents an algorithm for scene change detection from point clouds to enable autonomous robotic caretaking in future space habitats. Autonomous robotic systems will help maintain future deep-space habitats, such as the Gateway space station, which will be uncrewed for extended periods. Existing scene analysis software used on the International Space Station (ISS) relies on manually-labeled images for detecting changes. In contrast, the algorithm presented in this work uses raw, unlabeled point clouds as inputs. The algorithm first applies modified Expectation-Maximization Gaussian Mixture Model (GMM) clustering to two input point clouds. It then performs change detection by comparing the GMMs using the Earth Mover’s Distance. The algorithm is validated quantitatively and qualitatively using a test dataset collected by an Astrobee robot in the NASA Ames Granite Lab comprising single frame depth images taken directly by Astrobee and full-scene reconstructed maps built with RGB-D and pose data from Astrobee. The runtimes of the approach are also analyzed in depth. The source code is publicly released to promote further development.
2023
The Impact of Time Step Frequency on the Realism of Robotic Manipulation Simulation for Objects of Different Scales
This work evaluates the impact of time step frequency and component scale on robotic manipulation simulation accuracy. Increasing the time step frequency for small-scale objects is shown to improve simulation accuracy. This simulation, demonstrating pre-assembly part picking for two object geometries, serves as a starting point for discussing how to improve Sim2Real transfer in robotic assembly processes.
TrackDLO: Tracking Deformable Linear Objects Under Occlusion with Motion Coherence
The TrackDLO algorithm estimates the shape of a Deformable Linear Object (DLO) under occlusion from a sequence of RGB-D images. TrackDLO is vision-only and runs in real-time. It requires no external state information from physics modeling, simulation, visual markers, or contact as input. The algorithm improves on previous approaches by addressing three common scenarios which cause tracking failure: tip occlusion, mid-section occlusion, and self-occlusion. This is achieved through the application of Motion Coherence Theory to impute the spatial velocity of occluded nodes, the use of the topological geodesic distance to track self-occluding DLOs, and the introduction of a non-Gaussian kernel that only penalizes lower-order spatial displacement derivatives to reflect DLO physics. Improved real-time DLO tracking under mid-section occlusion, tip occlusion, and self-occlusion is demonstrated experimentally. The source code and demonstration data are publicly released.
Simultaneous Shape Tracking of Multiple Deformable Linear Objects with Global-Local Topology Preservation
This work presents an algorithm for tracking the shape of multiple entangling Deformable Linear Objects (DLOs) from a sequence of RGB-D images. This algorithm runs in real-time and improves on previous single-DLO tracking approaches by enabling tracking of multiple objects. This is achieved using Global-Local Topology Preservation (GLTP). This work uses the geodesic distance in GLTP to define the distance between separate objects and the distance between different parts of the same object. Tracking multiple entangling DLOs is demonstrated experimentally. The source code is publicly released.
Multi-Agent 3D Map Reconstruction and Change Detection in Microgravity with Free-Flying Robots
Assistive free-flyer robots autonomously caring for future crewed outposts—such as NASA’s Astrobee robots on the International Space Station (ISS)—must be able to detect day-to-day interior changes to track inventory, detect and diagnose faults, and monitor the outpost status. This work presents a framework for multi-agent cooperative mapping and change detection to enable robotic maintenance of space outposts. One agent is used to reconstruct a 3D model of the environment from sequences of images and corresponding depth information. Another agent is used to periodically scan the environment for inconsistencies against the 3D model. Change detection is validated \textita posteriori using real image and pose data collected by Astrobee robots in a ground testbed environment and from microgravity aboard the ISS. This work outlines the objectives, requirements, and algorithmic modules for the multi-agent reconstruction system, including recommendations for its use by assistive free-flyers aboard future microgravity outposts.
2022
Vela: A Data-Driven Proposal for Joint Collaboration in Space Exploration
The UN Office of Outer Space Affairs identifies synergy of space development activities and international cooperation through data and infrastructure sharing in their Sustainable Development Goal 17 (SDG17). Current multilateral space exploration paradigms, however, are divided between the Artemis and the Roscosmos-CNSA programs to return to the moon and establish permanent human settlements. As space agencies work to expand human presence in space, economic resource consolidation in pursuit of technologically ambitious space expeditions is the most sensible path to accomplish SDG17. This paper compiles a budget dataset for the top five federally-funded space agencies: CNSA, ESA, JAXA, NASA, and Roscosmos. Using time-series econometric anslysis methods in STATA, this work analyzes each agency’s economic contributions toward space exploration. The dataset results are used to propose a multinational space mission, Vela, for the development of an orbiting space station around Mars in the late 2030s. Distribution of economic resources and technological capabilities by the respective space programs are proposed to ensure programmatic redundancy and increase the odds of success on the given timeline.
Wire Point Cloud Instance Segmentation from RGBD Imagery with Mask R-CNN
Perception of the shapes of deforming objects like wires enables their monitoring and manipulation by autonomous robots. This paper presents detection, classification, and instance segmentation of deformable wires from a cluttered scene in RGBD imagery. This work uses the Detectron2 implementation of Mask R-CNN trained with the PointRend mask head on the UIUCWires dataset as the framework for wire instance segmentation on RGB imagery, a method demonstrated to perform well for the instance segmentation task in previous work. In this work, the instance bitmask is directly used to segment individual object point clouds, an important step toward wire shape representation for manipulation tasks.
Insights from an Industrial Collaborative Assembly Project: Lessons in Research and Collaboration
Significant progress in robotics reveals new opportunities to advance manufacturing. Next-generation industrial automation will require both integration of distinct robotic technologies and their application to challenging industrial environments. This paper presents lessons from a collaborative assembly project between three academic research groups and an industry partner. The goal of the project is to develop a flexible, safe, and productive manufacturing cell for sub-centimeter precision assembly. Solving this problem in a high-mix, low-volume production line motivates multiple research thrusts in robotics. This work identifies new directions in collaborative robotics for industrial applications and offers insight toward strengthening collaborations between institutions in academia and industry on the development of new technologies.
2021
Development of a Private Space Sector in the U.S. and Russia
This work analyzes progress and challenges in the development of commercial space economies in the United States and Russia. Space development progress is characterized through examination of three production indicators for the space economies in each country and econometric analysis of the financial data for three new American private space companies. We also provide case studies of major challenges faced by three new private space companies and conjecture about the future of the private space industry through analysis of space education projects in each country. As leaders of space technological development and exploration, the U.S. and Russia are major stakeholders in the current and future global space economy. We find access to capital is a primary barrier to entrepreneurship. The results will help future founders more effectively navigate the complex financial, regulatory, and technological landscape of the space industry.
2016
Gas Filled RF Resonator Hadron Beam Monitor for Intense Neutrino Beam Experiments
MW-class beam facilities are being considered all over the world to produce an intense neutrino beam for fundamental particle physics experiments. A radiation-robust beam monitor system is required to diagnose the primary and secondary beam qualities in high-radiation environments. We have proposed a novel gas-filled RF-resonator hadron beam monitor in which charged particles passing through the resonator produce ionized plasma that changes the permittivity of the gas. The sensitivity of the monitor has been evaluated in numerical simulation. A signal manipulation algorithm has been designed. A prototype system will be constructed and tested by using a proton beam at the MuCool Test Area at Fermilab.
