Showing 1–41 of 41 results for author: Sridharan, M

Anticipate & Collab: Data-driven Task Anticipation and Knowledge-driven Planning for Human-robot Collaboration

Authors: Shivam Singh, Karthik Swaminathan, Raghav Arora, Ramandeep Singh, Ahana Datta, Dipanjan Das, Snehasis Banerjee, Mohan Sridharan, Madhava Krishna

Abstract: An agent assisting humans in daily living activities can collaborate more effectively by anticipating upcoming tasks. Data-driven methods represent the state of the art in task anticipation, planning, and related problems, but these methods are resource-hungry and opaque. Our prior work introduced a proof of concept framework that used an LLM to anticipate 3 high-level tasks that served as goals f… ▽ More An agent assisting humans in daily living activities can collaborate more effectively by anticipating upcoming tasks. Data-driven methods represent the state of the art in task anticipation, planning, and related problems, but these methods are resource-hungry and opaque. Our prior work introduced a proof of concept framework that used an LLM to anticipate 3 high-level tasks that served as goals for a classical planning system that computed a sequence of low-level actions for the agent to achieve these goals. This paper describes DaTAPlan, our framework that significantly extends our prior work toward human-robot collaboration. Specifically, DaTAPlan planner computes actions for an agent and a human to collaboratively and jointly achieve the tasks anticipated by the LLM, and the agent automatically adapts to unexpected changes in human action outcomes and preferences. We evaluate DaTAPlan capabilities in a realistic simulation environment, demonstrating accurate task anticipation, effective human-robot collaboration, and the ability to adapt to unexpected changes. Project website: https://dataplan-hrc.github.io △ Less

Submitted 4 April, 2024; originally announced April 2024.

Relevance Score: A Landmark-Like Heuristic for Planning

Authors: Oliver Kim, Mohan Sridharan

Abstract: Landmarks are facts or actions that appear in all valid solutions of a planning problem. They have been used successfully to calculate heuristics that guide the search for a plan. We investigate an extension to this concept by defining a novel "relevance score" that helps identify facts or actions that appear in most but not all plans to achieve any given goal. We describe an approach to compute t… ▽ More Landmarks are facts or actions that appear in all valid solutions of a planning problem. They have been used successfully to calculate heuristics that guide the search for a plan. We investigate an extension to this concept by defining a novel "relevance score" that helps identify facts or actions that appear in most but not all plans to achieve any given goal. We describe an approach to compute this relevance score and use it as a heuristic in the search for a plan. We experimentally compare the performance of our approach with that of a state of the art landmark-based heuristic planning approach using benchmark planning problems. While the original landmark-based heuristic leads to better performance on problems with well-defined landmarks, our approach substantially improves performance on problems that lack non-trivial landmarks. △ Less

Submitted 12 March, 2024; originally announced March 2024.

Comments: 12 Pages, 3 figures

ACM Class: I.2.8

Inference of Resource Management Specifications

Authors: Narges Shadab, Pritam Gharat, Shrey Tiwari, Michael D. Ernst, Martin Kellogg, Shuvendu Lahiri, Akash Lal, Manu Sridharan

Abstract: A resource leak occurs when a program fails to free some finite resource after it is no longer needed. Such leaks are a significant cause of real-world crashes and performance problems. Recent work proposed an approach to prevent resource leaks based on checking resource management specifications. A resource management specification expresses how the program allocates resources, passes them around… ▽ More A resource leak occurs when a program fails to free some finite resource after it is no longer needed. Such leaks are a significant cause of real-world crashes and performance problems. Recent work proposed an approach to prevent resource leaks based on checking resource management specifications. A resource management specification expresses how the program allocates resources, passes them around, and releases them; it also tracks the ownership relationship between objects and resources, and aliasing relationships between objects. While this specify-and-verify approach has several advantages compared to prior techniques, the need to manually write annotations presents a significant barrier to its practical adoption. This paper presents a novel technique to automatically infer a resource management specification for a program, broadening the applicability of specify-and-check verification for resource leaks. Inference in this domain is challenging because resource management specifications differ significantly in nature from the types that most inference techniques target. Further, for practical effectiveness, we desire a technique that can infer the resource management specification intended by the developer, even in cases when the code does not fully adhere to that specification. We address these challenges through a set of inference rules carefully designed to capture real-world coding patterns, yielding an effective fixed-point-based inference algorithm. We have implemented our inference algorithm in two different systems, targeting programs written in Java and C#. In an experimental evaluation, our technique inferred 85.5% of the annotations that programmers had written manually for the benchmarks. Further, the verifier issued nearly the same rate of false alarms with the manually-written and automatically-inferred annotations. △ Less

Submitted 21 September, 2023; v1 submitted 20 June, 2023; originally announced June 2023.

CLIPGraphs: Multimodal Graph Networks to Infer Object-Room Affinities

Authors: Ayush Agrawal, Raghav Arora, Ahana Datta, Snehasis Banerjee, Brojeshwar Bhowmick, Krishna Murthy Jatavallabhula, Mohan Sridharan, Madhava Krishna

Abstract: This paper introduces a novel method for determining the best room to place an object in, for embodied scene rearrangement. While state-of-the-art approaches rely on large language models (LLMs) or reinforcement learned (RL) policies for this task, our approach, CLIPGraphs, efficiently combines commonsense domain knowledge, data-driven methods, and recent advances in multimodal learning. Specifica… ▽ More This paper introduces a novel method for determining the best room to place an object in, for embodied scene rearrangement. While state-of-the-art approaches rely on large language models (LLMs) or reinforcement learned (RL) policies for this task, our approach, CLIPGraphs, efficiently combines commonsense domain knowledge, data-driven methods, and recent advances in multimodal learning. Specifically, it (a)encodes a knowledge graph of prior human preferences about the room location of different objects in home environments, (b) incorporates vision-language features to support multimodal queries based on images or text, and (c) uses a graph network to learn object-room affinities based on embeddings of the prior knowledge and the vision-language features. We demonstrate that our approach provides better estimates of the most appropriate location of objects from a benchmark set of object categories in comparison with state-of-the-art baselines △ Less

Submitted 2 June, 2023; originally announced June 2023.

Journal ref: RO-MAN 2023 Conference

Knowledge-based Reasoning and Learning under Partial Observability in Ad Hoc Teamwork

Authors: Hasra Dodampegama, Mohan Sridharan

Abstract: Ad hoc teamwork refers to the problem of enabling an agent to collaborate with teammates without prior coordination. Data-driven methods represent the state of the art in ad hoc teamwork. They use a large labeled dataset of prior observations to model the behavior of other agent types and to determine the ad hoc agent's behavior. These methods are computationally expensive, lack transparency, and… ▽ More Ad hoc teamwork refers to the problem of enabling an agent to collaborate with teammates without prior coordination. Data-driven methods represent the state of the art in ad hoc teamwork. They use a large labeled dataset of prior observations to model the behavior of other agent types and to determine the ad hoc agent's behavior. These methods are computationally expensive, lack transparency, and make it difficult to adapt to previously unseen changes, e.g., in team composition. Our recent work introduced an architecture that determined an ad hoc agent's behavior based on non-monotonic logical reasoning with prior commonsense domain knowledge and predictive models of other agents' behavior that were learned from limited examples. In this paper, we substantially expand the architecture's capabilities to support: (a) online selection, adaptation, and learning of the models that predict the other agents' behavior; and (b) collaboration with teammates in the presence of partial observability and limited communication. We illustrate and experimentally evaluate the capabilities of our architecture in two simulated multiagent benchmark domains for ad hoc teamwork: Fort Attack and Half Field Offense. We show that the performance of our architecture is comparable or better than state of the art data-driven baselines in both simple and complex scenarios, particularly in the presence of limited training data, partial observability, and changes in team composition. △ Less

Submitted 1 June, 2023; originally announced June 2023.

Comments: 17 pages, 3 Figures

RAMP: A Benchmark for Evaluating Robotic Assembly Manipulation and Planning

Authors: Jack Collins, Mark Robson, Jun Yamada, Mohan Sridharan, Karol Janik, Ingmar Posner

Abstract: We introduce RAMP, an open-source robotics benchmark inspired by real-world industrial assembly tasks. RAMP consists of beams that a robot must assemble into specified goal configurations using pegs as fasteners. As such, it assesses planning and execution capabilities, and poses challenges in perception, reasoning, manipulation, diagnostics, fault recovery, and goal parsing. RAMP has been designe… ▽ More We introduce RAMP, an open-source robotics benchmark inspired by real-world industrial assembly tasks. RAMP consists of beams that a robot must assemble into specified goal configurations using pegs as fasteners. As such, it assesses planning and execution capabilities, and poses challenges in perception, reasoning, manipulation, diagnostics, fault recovery, and goal parsing. RAMP has been designed to be accessible and extensible. Parts are either 3D printed or otherwise constructed from materials that are readily obtainable. The design of parts and detailed instructions are publicly available. In order to broaden community engagement, RAMP incorporates fixtures such as April Tags which enable researchers to focus on individual sub-tasks of the assembly challenge if desired. We provide a full digital twin as well as rudimentary baselines to enable rapid progress. Our vision is for RAMP to form the substrate for a community-driven endeavour that evolves as capability matures. △ Less

Submitted 8 November, 2023; v1 submitted 16 May, 2023; originally announced May 2023.

Comments: Project website: https://sites.google.com/oxfordrobotics.institute/ramp

Sequence-Agnostic Multi-Object Navigation

Authors: Nandiraju Gireesh, Ayush Agrawal, Ahana Datta, Snehasis Banerjee, Mohan Sridharan, Brojeshwar Bhowmick, Madhava Krishna

Abstract: The Multi-Object Navigation (MultiON) task requires a robot to localize an instance (each) of multiple object classes. It is a fundamental task for an assistive robot in a home or a factory. Existing methods for MultiON have viewed this as a direct extension of Object Navigation (ON), the task of localising an instance of one object class, and are pre-sequenced, i.e., the sequence in which the obj… ▽ More The Multi-Object Navigation (MultiON) task requires a robot to localize an instance (each) of multiple object classes. It is a fundamental task for an assistive robot in a home or a factory. Existing methods for MultiON have viewed this as a direct extension of Object Navigation (ON), the task of localising an instance of one object class, and are pre-sequenced, i.e., the sequence in which the object classes are to be explored is provided in advance. This is a strong limitation in practical applications characterized by dynamic changes. This paper describes a deep reinforcement learning framework for sequence-agnostic MultiON based on an actor-critic architecture and a suitable reward specification. Our framework leverages past experiences and seeks to reward progress toward individual as well as multiple target object classes. We use photo-realistic scenes from the Gibson benchmark dataset in the AI Habitat 3D simulation environment to experimentally show that our method performs better than a pre-sequenced approach and a state of the art ON method extended to MultiON. △ Less

Submitted 10 May, 2023; originally announced May 2023.

Journal ref: ICRA 2023 conference

PENTACET data -- 23 Million Contextual Code Comments and 250,000 SATD comments

Authors: Murali Sridharan, Leevi Rantala, Mika Mäntylä

Abstract: Most Self-Admitted Technical Debt (SATD) research utilizes explicit SATD features such as 'TODO' and 'FIXME' for SATD detection. A closer look reveals several SATD research uses simple SATD ('Easy to Find') code comments without the contextual data (preceding and succeeding source code context). This work addresses this gap through PENTACET (or 5C dataset) data. PENTACET is a large Curated Context… ▽ More Most Self-Admitted Technical Debt (SATD) research utilizes explicit SATD features such as 'TODO' and 'FIXME' for SATD detection. A closer look reveals several SATD research uses simple SATD ('Easy to Find') code comments without the contextual data (preceding and succeeding source code context). This work addresses this gap through PENTACET (or 5C dataset) data. PENTACET is a large Curated Contextual Code Comments per Contributor and the most extensive SATD data. We mine 9,096 Open Source Software Java projects with a total of 435 million LOC. The outcome is a dataset with 23 million code comments, preceding and succeeding source code context for each comment, and more than 250,000 comments labeled as SATD, including both 'Easy to Find' and 'Hard to Find' SATD. We believe PENTACET data will further SATD research using Artificial Intelligence techniques. △ Less

Submitted 11 August, 2023; v1 submitted 24 March, 2023; originally announced March 2023.

Comments: Accepted in MSR 2023 Tools and Data Showcase

Toward a Framework for Adaptive Impedance Control of an Upper-limb Prosthesis

Authors: Laura Ferrante, Mohan Sridharan, Claudio Zito, Dario Farina

Abstract: Adapting upper-limb impedance (i.e., stiffness, damping, inertia) is essential for humans interacting with dynamic environments for executing grasping or manipulation tasks. On the other hand, control methods designed for state-of-the-art upper-limb prostheses infer motor intent from surface electromyography (sEMG) signals in terms of joint kinematics, but they fail to infer and use the underlying… ▽ More Adapting upper-limb impedance (i.e., stiffness, damping, inertia) is essential for humans interacting with dynamic environments for executing grasping or manipulation tasks. On the other hand, control methods designed for state-of-the-art upper-limb prostheses infer motor intent from surface electromyography (sEMG) signals in terms of joint kinematics, but they fail to infer and use the underlying impedance properties of the limb. We present a framework that allows a human user to simultaneously control the kinematics, stiffness, and damping of a simulated robot through wrist's flexion-extension. The framework includes muscle-tendon units and a forward dynamics block to estimate the motor intent from sEMG signals, and a variable impedance controller that implements the estimated intent on the robot, allowing the user to adapt the robot's kinematics and dynamics online. We evaluate our framework with 8 able-bodied subjects and an amputee during reaching tasks performed in free space, and in the presence of unexpected external perturbations that require adaptation of the wrist impedance to ensure stable interaction with the environment. We experimentally demonstrate that our approach outperforms a data-driven baseline in terms of its ability to adapt to external perturbations, overall controllability, and feedback from participants. △ Less

Submitted 17 December, 2022; v1 submitted 11 September, 2022; originally announced September 2022.

Spatial Relation Graph and Graph Convolutional Network for Object Goal Navigation

Authors: D. A. Sasi Kiran, Kritika Anand, Chaitanya Kharyal, Gulshan Kumar, Nandiraju Gireesh, Snehasis Banerjee, Ruddra dev Roychoudhury, Mohan Sridharan, Brojeshwar Bhowmick, Madhava Krishna

Abstract: This paper describes a framework for the object-goal navigation task, which requires a robot to find and move to the closest instance of a target object class from a random starting position. The framework uses a history of robot trajectories to learn a Spatial Relational Graph (SRG) and Graph Convolutional Network (GCN)-based embeddings for the likelihood of proximity of different semantically-la… ▽ More This paper describes a framework for the object-goal navigation task, which requires a robot to find and move to the closest instance of a target object class from a random starting position. The framework uses a history of robot trajectories to learn a Spatial Relational Graph (SRG) and Graph Convolutional Network (GCN)-based embeddings for the likelihood of proximity of different semantically-labeled regions and the occurrence of different object classes in these regions. To locate a target object instance during evaluation, the robot uses Bayesian inference and the SRG to estimate the visible regions, and uses the learned GCN embeddings to rank visible regions and select the region to explore next. △ Less

Submitted 27 August, 2022; originally announced August 2022.

Comments: CASE 2022 paper

Object Goal Navigation using Data Regularized Q-Learning

Authors: Nandiraju Gireesh, D. A. Sasi Kiran, Snehasis Banerjee, Mohan Sridharan, Brojeshwar Bhowmick, Madhava Krishna

Abstract: Object Goal Navigation requires a robot to find and navigate to an instance of a target object class in a previously unseen environment. Our framework incrementally builds a semantic map of the environment over time, and then repeatedly selects a long-term goal ('where to go') based on the semantic map to locate the target object instance. Long-term goal selection is formulated as a vision-based d… ▽ More Object Goal Navigation requires a robot to find and navigate to an instance of a target object class in a previously unseen environment. Our framework incrementally builds a semantic map of the environment over time, and then repeatedly selects a long-term goal ('where to go') based on the semantic map to locate the target object instance. Long-term goal selection is formulated as a vision-based deep reinforcement learning problem. Specifically, an Encoder Network is trained to extract high-level features from a semantic map and select a long-term goal. In addition, we incorporate data augmentation and Q-function regularization to make the long-term goal selection more effective. We report experimental results using the photo-realistic Gibson benchmark dataset in the AI Habitat 3D simulation environment to demonstrate substantial performance improvement on standard measures in comparison with a state of the art data-driven baseline. △ Less

Submitted 27 August, 2022; originally announced August 2022.

Comments: CASE 2022 paper

Knowledge-based and Data-driven Reasoning and Learning for Ad Hoc Teamwork

Authors: Hasra Dodampegama, Mohan Sridharan

Abstract: We present an architecture for ad hoc teamwork, which refers to collaboration in a team of agents without prior coordination. State of the art methods for this problem often include a data-driven component that uses a long history of prior observations to model the behaviour of other agents (or agent types) and to determine the ad hoc agent's behaviour. In many practical domains, it is challenging… ▽ More We present an architecture for ad hoc teamwork, which refers to collaboration in a team of agents without prior coordination. State of the art methods for this problem often include a data-driven component that uses a long history of prior observations to model the behaviour of other agents (or agent types) and to determine the ad hoc agent's behaviour. In many practical domains, it is challenging to find large training datasets, and necessary to understand and incrementally extend the existing models to account for changes in team composition or domain attributes. Our architecture combines the principles of knowledge-based and data-driven reasoning and learning. Specifically, we enable an ad hoc agent to perform non-monotonic logical reasoning with prior commonsense domain knowledge and incrementally-updated simple predictive models of other agents' behaviour. We use the benchmark simulated multi-agent collaboration domain Fort Attack to demonstrate that our architecture supports adaptation to unforeseen changes, incremental learning and revision of models of other agents' behaviour from limited samples, transparency in the ad hoc agent's decision making, and better performance than a data-driven baseline. △ Less

Submitted 19 October, 2022; v1 submitted 24 August, 2022; originally announced August 2022.

Comments: Presented at the AI-HRI Symposium at AAAI Fall Symposium Series(FSS), 2022 (arXiv:cs/2209.14292)

Report number: AIHRI/2022/7052

BURG-Toolkit: Robot Grasping Experiments in Simulation and the Real World

Authors: Martin Rudorfer, Markus Suchi, Mohan Sridharan, Markus Vincze, Aleš Leonardis

Abstract: This paper presents BURG-Toolkit, a set of open-source tools for Benchmarking and Understanding Robotic Grasping. Our tools allow researchers to: (1) create virtual scenes for generating training data and performing grasping in simulation; (2) recreate the scene by arranging the corresponding objects accurately in the physical world for real robot experiments, supporting an analysis of the sim-to-… ▽ More This paper presents BURG-Toolkit, a set of open-source tools for Benchmarking and Understanding Robotic Grasping. Our tools allow researchers to: (1) create virtual scenes for generating training data and performing grasping in simulation; (2) recreate the scene by arranging the corresponding objects accurately in the physical world for real robot experiments, supporting an analysis of the sim-to-real gap; and (3) share the scenes with other researchers to foster comparability and reproducibility of experimental results. We explain how to use our tools by describing some potential use cases. We further provide proof-of-concept experimental results quantifying the sim-to-real gap for robot grasping in some example scenes. The tools are available at: https://mrudorfer.github.io/burg-toolkit/ △ Less

Submitted 27 May, 2022; originally announced May 2022.

Comments: presented at ICRA 2022 Workshop on Releasing Robots into the Wild: Simulations, Benchmarks, and Deployment. Project page: https://mrudorfer.github.io/burg-toolkit/

Automatic Root Cause Quantification for Missing Edges in JavaScript Call Graphs (Extended Version)

Authors: Madhurima Chakraborty, Renzo Olivares, Manu Sridharan, Behnaz Hassanshahi

Abstract: Building sound and precise static call graphs for real-world JavaScript applications poses an enormous challenge, due to many hard-to-analyze language features. Further, the relative importance of these features may vary depending on the call graph algorithm being used and the class of applications being analyzed. In this paper, we present a technique to automatically quantify the relative importa… ▽ More Building sound and precise static call graphs for real-world JavaScript applications poses an enormous challenge, due to many hard-to-analyze language features. Further, the relative importance of these features may vary depending on the call graph algorithm being used and the class of applications being analyzed. In this paper, we present a technique to automatically quantify the relative importance of different root causes of call graph unsoundness for a set of target applications. The technique works by identifying the dynamic function data flows relevant to each call edge missed by the static analysis, correctly handling cases with multiple root causes and inter-dependent calls. We apply our approach to perform a detailed study of the recall of a state-of-the-art call graph construction technique on a set of framework-based web applications. The study yielded a number of useful insights. We found that while dynamic property accesses were the most common root cause of missed edges across the benchmarks, other root causes varied in importance depending on the benchmark, potentially useful information for an analysis designer. Further, with our approach, we could quickly identify and fix a recall issue in the call graph builder we studied, and also quickly assess whether a recent analysis technique for Node.js-based applications would be helpful for browser-based code. All of our code and data is publicly available, and many components of our technique can be re-used to facilitate future studies. △ Less

Submitted 13 May, 2022; originally announced May 2022.

Comments: Extended version of ECOOP'22 paper (with appendix)

Generating Task-specific Robotic Grasps

Authors: Mark Robson, Mohan Sridharan

Abstract: This paper describes a method for generating robot grasps by jointly considering stability and other task and object-specific constraints. We introduce a three-level representation that is acquired for each object class from a small number of exemplars of objects, tasks, and relevant grasps. The representation encodes task-specific knowledge for each object class as a relationship between a keypoi… ▽ More This paper describes a method for generating robot grasps by jointly considering stability and other task and object-specific constraints. We introduce a three-level representation that is acquired for each object class from a small number of exemplars of objects, tasks, and relevant grasps. The representation encodes task-specific knowledge for each object class as a relationship between a keypoint skeleton and suitable grasp points that is preserved despite intra-class variations in scale and orientation. The learned models are queried at run time by a simple sampling-based method to guide the generation of grasps that balance task and stability constraints. We ground and evaluate our method in the context of a Franka Emika Panda robot assisting a human in picking tabletop objects for which the robot does not have prior CAD models. Experimental results demonstrate that in comparison with a baseline method that only focuses on stability, our method is able to provide suitable grasps for different tasks. △ Less

Submitted 20 March, 2022; originally announced March 2022.

A Survey of Ad Hoc Teamwork Research

Authors: Reuth Mirsky, Ignacio Carlucho, Arrasy Rahman, Elliot Fosong, William Macke, Mohan Sridharan, Peter Stone, Stefano V. Albrecht

Abstract: Ad hoc teamwork is the research problem of designing agents that can collaborate with new teammates without prior coordination. This survey makes a two-fold contribution: First, it provides a structured description of the different facets of the ad hoc teamwork problem. Second, it discusses the progress that has been made in the field so far, and identifies the immediate and long-term open problem… ▽ More Ad hoc teamwork is the research problem of designing agents that can collaborate with new teammates without prior coordination. This survey makes a two-fold contribution: First, it provides a structured description of the different facets of the ad hoc teamwork problem. Second, it discusses the progress that has been made in the field so far, and identifies the immediate and long-term open problems that need to be addressed in ad hoc teamwork. △ Less

Submitted 16 August, 2022; v1 submitted 16 February, 2022; originally announced February 2022.

Comments: European Conference on Multi-Agent Systems (EUMAS), 2022

Combining Commonsense Reasoning and Knowledge Acquisition to Guide Deep Learning in Robotics

Authors: Mohan Sridharan, Tiago Mota

Abstract: Algorithms based on deep network models are being used for many pattern recognition and decision-making tasks in robotics and AI. Training these models requires a large labeled dataset and considerable computational resources, which are not readily available in many domains. Also, it is difficult to explore the internal representations and reasoning mechanisms of these models. As a step towards ad… ▽ More Algorithms based on deep network models are being used for many pattern recognition and decision-making tasks in robotics and AI. Training these models requires a large labeled dataset and considerable computational resources, which are not readily available in many domains. Also, it is difficult to explore the internal representations and reasoning mechanisms of these models. As a step towards addressing the underlying knowledge representation, reasoning, and learning challenges, the architecture described in this paper draws inspiration from research in cognitive systems. As a motivating example, we consider an assistive robot trying to reduce clutter in any given scene by reasoning about the occlusion of objects and stability of object configurations in an image of the scene. In this context, our architecture incrementally learns and revises a grounding of the spatial relations between objects and uses this grounding to extract spatial information from input images. Non-monotonic logical reasoning with this information and incomplete commonsense domain knowledge is used to make decisions about stability and occlusion. For images that cannot be processed by such reasoning, regions relevant to the tasks at hand are automatically identified and used to train deep network models to make the desired decisions. Image regions used to train the deep networks are also used to incrementally acquire previously unknown state constraints that are merged with the existing knowledge for subsequent reasoning. Experimental evaluation performed using simulated and real-world images indicates that in comparison with baselines based just on deep networks, our architecture improves reliability of decision making and reduces the effort involved in training data-driven deep network models. △ Less

Submitted 25 January, 2022; originally announced January 2022.

Comments: 37 pages, 17 figures, 5 tables

The Ninth Advances in Cognitive Systems (ACS) Conference

Authors: Mark Burstein, Mohan Sridharan, David McDonald

Abstract: ACS is an annual meeting for research on the initial goals of artificial intelligence and cognitive science, which aimed to explain the mind in computational terms and to reproduce the entire range of human cognitive abilities in computational artifacts. Many researchers remain committed to this original vision, and Advances in Cognitive Systems provides a place to present recent results and pose… ▽ More ACS is an annual meeting for research on the initial goals of artificial intelligence and cognitive science, which aimed to explain the mind in computational terms and to reproduce the entire range of human cognitive abilities in computational artifacts. Many researchers remain committed to this original vision, and Advances in Cognitive Systems provides a place to present recent results and pose new challenges for the field. The meetings bring together researchers with interests in human-level intelligence, complex cognition, integrated intelligent systems, cognitive architectures, and related topics. △ Less

Submitted 16 January, 2022; originally announced January 2022.

An Adaptive Framework for Reliable Trajectory Following in Changing-Contact Robot Manipulation Tasks

Authors: Saif Sidhik, Mohan Sridharan, Dirk Ruiken

Abstract: We describe a framework for changing-contact robot manipulation tasks that require the robot to make and break contacts with objects and surfaces. The discontinuous interaction dynamics of such tasks make it difficult to construct and use a single dynamics model or control strategy, and the highly non-linear nature of the dynamics during contact changes can be damaging to the robot and the objects… ▽ More We describe a framework for changing-contact robot manipulation tasks that require the robot to make and break contacts with objects and surfaces. The discontinuous interaction dynamics of such tasks make it difficult to construct and use a single dynamics model or control strategy, and the highly non-linear nature of the dynamics during contact changes can be damaging to the robot and the objects. We present an adaptive control framework that enables the robot to incrementally learn to predict contact changes in a changing contact task, learn the interaction dynamics of the piece-wise continuous system, and provide smooth and accurate trajectory tracking using a task-space variable impedance controller. We experimentally compare the performance of our framework against that of representative control methods to establish that the adaptive control and incremental learning components of our framework are needed to achieve smooth control in the presence of discontinuous dynamics in changing-contact robot manipulation tasks. △ Less

Submitted 15 November, 2021; originally announced November 2021.

Comments: 21 pages including references

Towards a Framework for Changing-Contact Robot Manipulation

Authors: Saif Sidhik, Mohan Sridharan, Dirk Ruiken

Abstract: Many robot manipulation tasks require the robot to make and break contact with objects and surfaces. The dynamics of such changing-contact robot manipulation tasks are discontinuous when contact is made or broken, and continuous elsewhere. These discontinuities make it difficult to construct and use a single dynamics model or control strategy for any such task. We present a framework for smooth dy… ▽ More Many robot manipulation tasks require the robot to make and break contact with objects and surfaces. The dynamics of such changing-contact robot manipulation tasks are discontinuous when contact is made or broken, and continuous elsewhere. These discontinuities make it difficult to construct and use a single dynamics model or control strategy for any such task. We present a framework for smooth dynamics and control of such changing-contact manipulation tasks. For any given target motion trajectory, the framework incrementally improves its prediction of when contacts will occur. This prediction and a model relating approach velocity to impact force modify the velocity profile of the motion sequence such that it is $C^\infty$ smooth, and help achieve a desired force on impact. We implement this framework by building on our hybrid force-motion variable impedance controller for continuous contact tasks. We experimentally evaluate our framework in the illustrative context of sliding tasks involving multiple contact changes with transitions between surfaces of different properties. △ Less

Submitted 21 June, 2021; originally announced June 2021.

Comments: Submitted to "Autonomous Robots and Multirobot Systems (ARMS) Workshop" at 20th International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2021

Demanded Abstract Interpretation (Extended Version)

Authors: Benno Stein, Bor-Yuh Evan Chang, Manu Sridharan

Abstract: We consider the problem of making expressive static analyzers interactive. Formal static analysis is seeing increasingly widespread adoption as a tool for verification and bug-finding, but even with powerful cloud infrastructure it can take minutes or hours to get batch analysis results after a code change. While existing techniques offer some demand-driven or incremental aspects for certain class… ▽ More We consider the problem of making expressive static analyzers interactive. Formal static analysis is seeing increasingly widespread adoption as a tool for verification and bug-finding, but even with powerful cloud infrastructure it can take minutes or hours to get batch analysis results after a code change. While existing techniques offer some demand-driven or incremental aspects for certain classes of analysis, the fundamental challenge we tackle is doing both for arbitrary abstract interpreters. Our technique, demanded abstract interpretation, lifts program syntax and analysis state to a dynamically evolving graph structure, in which program edits, client-issued queries, and evaluation of abstract semantics are all treated uniformly. The key difficulty addressed by our approach is the application of general incremental computation techniques to the complex, cyclic dependency structure induced by abstract interpretation of loops with widening operators. We prove that desirable abstract interpretation meta-properties, including soundness and termination, are preserved in our approach, and that demanded analysis results are equal to those computed by a batch abstract interpretation. Experimental results suggest promise for a prototype demanded abstract interpretation framework: by combining incremental and demand-driven techniques, our framework consistently delivers analysis results at interactive speeds, answering 95% of queries within 1.2 seconds. △ Less

Submitted 6 April, 2021; v1 submitted 2 April, 2021; originally announced April 2021.

Comments: extended version of PLDI'21 paper (with appendices)

Data Balancing Improves Self-Admitted Technical Debt Detection

Authors: Murali Sridharan, Mika Mantyla, Leevi Rantala, Maelick Claes

Abstract: A high imbalance exists between technical debt and non-technical debt source code comments. Such imbalance affects Self-Admitted Technical Debt (SATD) detection performance, and existing literature lacks empirical evidence on the choice of balancing technique. In this work, we evaluate the impact of multiple balancing techniques, including Data level, Classifier level, and Hybrid, for SATD detecti… ▽ More A high imbalance exists between technical debt and non-technical debt source code comments. Such imbalance affects Self-Admitted Technical Debt (SATD) detection performance, and existing literature lacks empirical evidence on the choice of balancing technique. In this work, we evaluate the impact of multiple balancing techniques, including Data level, Classifier level, and Hybrid, for SATD detection in Within-Project and Cross-Project setup. Our results show that the Data level balancing technique SMOTE or Classifier level Ensemble approaches Random Forest or XGBoost are reasonable choices depending on whether the goal is to maximize Precision, Recall, F1, or AUC-ROC. We compared our best-performing model with the previous SATD detection benchmark (cost-sensitive Convolution Neural Network). Interestingly the top-performing XGBoost with SMOTE sampling improved the Within-project F1 score by 10% but fell short in Cross-Project set up by 9%. This supports the higher generalization capability of deep learning in Cross-Project SATD detection, yet while working within individual projects, classical machine learning algorithms can deliver better performance. We also evaluate and quantify the impact of duplicate source code comments in SATD detection performance. Finally, we employ SHAP and discuss the interpreted SATD features. We have included the replication package and shared a web-based SATD prediction tool with the balancing techniques in this study. △ Less

Submitted 24 March, 2021; originally announced March 2021.

Comments: 11 pages, 1 figure, for conference

Continual Learning of Knowledge Graph Embeddings

Authors: Angel Daruna, Mehul Gupta, Mohan Sridharan, Sonia Chernova

Abstract: In recent years, there has been a resurgence in methods that use distributed (neural) representations to represent and reason about semantic knowledge for robotics applications. However, while robots often observe previously unknown concepts, these representations typically assume that all concepts are known a priori, and incorporating new information requires all concepts to be learned afresh. Ou… ▽ More In recent years, there has been a resurgence in methods that use distributed (neural) representations to represent and reason about semantic knowledge for robotics applications. However, while robots often observe previously unknown concepts, these representations typically assume that all concepts are known a priori, and incorporating new information requires all concepts to be learned afresh. Our work relaxes this limiting assumption of existing representations and tackles the incremental knowledge graph embedding problem by leveraging the principles of a range of continual learning methods. Through an experimental evaluation with several knowledge graphs and embedding representations, we provide insights about trade-offs for practitioners to match a semantics-driven robotics applications to a suitable continual knowledge graph embedding method. △ Less

Submitted 10 May, 2021; v1 submitted 14 January, 2021; originally announced January 2021.

Comments: 8 pages, 4 figures. Accepted for publication in IEEE Robotics and Automation Letters (RA-L)

Axiom Learning and Belief Tracing for Transparent Decision Making in Robotics

Authors: Tiago Mota, Mohan Sridharan

Abstract: A robot's ability to provide descriptions of its decisions and beliefs promotes effective collaboration with humans. Providing such transparency is particularly challenging in integrated robot systems that include knowledge-based reasoning methods and data-driven learning algorithms. Towards addressing this challenge, our architecture couples the complementary strengths of non-monotonic logical re… ▽ More A robot's ability to provide descriptions of its decisions and beliefs promotes effective collaboration with humans. Providing such transparency is particularly challenging in integrated robot systems that include knowledge-based reasoning methods and data-driven learning algorithms. Towards addressing this challenge, our architecture couples the complementary strengths of non-monotonic logical reasoning, deep learning, and decision-tree induction. During reasoning and learning, the architecture enables a robot to provide on-demand relational descriptions of its decisions, beliefs, and the outcomes of hypothetical actions. These capabilities are grounded and evaluated in the context of scene understanding tasks and planning tasks performed using simulated images and images from a physical robot manipulating tabletop objects. △ Less

Submitted 20 October, 2020; originally announced October 2020.

Comments: 10 pages, 5 figures, AAAI Fall Symposium on Artificial Intelligence for Human-Robot Interaction: Trust & Explainability in Artificial Intelligence for Human-Robot Interaction (2020)

A Survey of Knowledge-based Sequential Decision Making under Uncertainty

Authors: Shiqi Zhang, Mohan Sridharan

Abstract: Reasoning with declarative knowledge (RDK) and sequential decision-making (SDM) are two key research areas in artificial intelligence. RDK methods reason with declarative domain knowledge, including commonsense knowledge, that is either provided a priori or acquired over time, while SDM methods (probabilistic planning and reinforcement learning) seek to compute action policies that maximize the ex… ▽ More Reasoning with declarative knowledge (RDK) and sequential decision-making (SDM) are two key research areas in artificial intelligence. RDK methods reason with declarative domain knowledge, including commonsense knowledge, that is either provided a priori or acquired over time, while SDM methods (probabilistic planning and reinforcement learning) seek to compute action policies that maximize the expected cumulative utility over a time horizon; both classes of methods reason in the presence of uncertainty. Despite the rich literature in these two areas, researchers have not fully explored their complementary strengths. In this paper, we survey algorithms that leverage RDK methods while making sequential decisions under uncertainty. We discuss significant developments, open problems, and directions for future work. △ Less

Submitted 30 June, 2022; v1 submitted 19 August, 2020; originally announced August 2020.

Comments: AI Magazine, Volume 43, Issue 2, Pages 249-266, 2022

Non-monotonic Logical Reasoning Guiding Deep Learning for Explainable Visual Question Answering

Authors: Heather Riley, Mohan Sridharan

Abstract: State of the art algorithms for many pattern recognition problems rely on deep network models. Training these models requires a large labeled dataset and considerable computational resources. Also, it is difficult to understand the working of these learned models, limiting their use in some critical applications. Towards addressing these limitations, our architecture draws inspiration from researc… ▽ More State of the art algorithms for many pattern recognition problems rely on deep network models. Training these models requires a large labeled dataset and considerable computational resources. Also, it is difficult to understand the working of these learned models, limiting their use in some critical applications. Towards addressing these limitations, our architecture draws inspiration from research in cognitive systems, and integrates the principles of commonsense logical reasoning, inductive learning, and deep learning. In the context of answering explanatory questions about scenes and the underlying classification problems, the architecture uses deep networks for extracting features from images and for generating answers to queries. Between these deep networks, it embeds components for non-monotonic logical reasoning with incomplete commonsense domain knowledge, and for decision tree induction. It also incrementally learns and reasons with previously unknown constraints governing the domain's states. We evaluated the architecture in the context of datasets of simulated and real-world images, and a simulated robot computing, executing, and providing explanatory descriptions of plans. Experimental results indicate that in comparison with an ``end to end'' architecture of deep networks, our architecture provides better accuracy on classification problems when the training dataset is small, comparable accuracy with larger datasets, and more accurate answers to explanatory questions. Furthermore, incremental acquisition of previously unknown constraints improves the ability to answer explanatory questions, and extending non-monotonic logical reasoning to support planning and diagnostics improves the reliability and efficiency of computing and executing plans on a simulated robot. △ Less

Submitted 23 September, 2019; originally announced September 2019.

Comments: 28 pages, 15 figures

Towards a Theory of Intentions for Human-Robot Collaboration

Authors: Rocio Gomez, Mohan Sridharan, Heather Riley

Abstract: The architecture described in this paper encodes a theory of intentions based on the the key principles of non-procrastination, persistence, and automatically limiting reasoning to relevant knowledge and observations. The architecture reasons with transition diagrams of any given domain at two different resolutions, with the fine-resolution description defined as a refinement of, and hence tightly… ▽ More The architecture described in this paper encodes a theory of intentions based on the the key principles of non-procrastination, persistence, and automatically limiting reasoning to relevant knowledge and observations. The architecture reasons with transition diagrams of any given domain at two different resolutions, with the fine-resolution description defined as a refinement of, and hence tightly-coupled to, a coarse-resolution description. Non-monotonic logical reasoning with the coarse-resolution description computes an activity (i.e., plan) comprising abstract actions for any given goal. Each abstract action is implemented as a sequence of concrete actions by automatically zooming to and reasoning with the part of the fine-resolution transition diagram relevant to the current coarse-resolution transition and the goal. Each concrete action in this sequence is executed using probabilistic models of the uncertainty in sensing and actuation, and the corresponding fine-resolution outcomes are used to infer coarse-resolution observations that are added to the coarse-resolution history. The architecture's capabilities are evaluated in the context of a simulated robot assisting humans in an office domain, on a physical robot (Baxter) manipulating tabletop objects, and on a wheeled robot (Turtlebot) moving objects to particular places or people. The experimental results indicate improvements in reliability and computational efficiency compared with an architecture that does not include the theory of intentions, and an architecture that does not include zooming for fine-resolution reasoning. △ Less

Submitted 30 July, 2019; originally announced July 2019.

Comments: 25 pages, 4 figures

NullAway: Practical Type-Based Null Safety for Java

Authors: Subarno Banerjee, Lazaro Clapp, Manu Sridharan

Abstract: NullPointerExceptions (NPEs) are a key source of crashes in modern Java programs. Previous work has shown how such errors can be prevented at compile time via code annotations and pluggable type checking. However, such systems have been difficult to deploy on large-scale software projects, due to significant build-time overhead and / or a high annotation burden. This paper presents NullAway, a new… ▽ More NullPointerExceptions (NPEs) are a key source of crashes in modern Java programs. Previous work has shown how such errors can be prevented at compile time via code annotations and pluggable type checking. However, such systems have been difficult to deploy on large-scale software projects, due to significant build-time overhead and / or a high annotation burden. This paper presents NullAway, a new type-based null safety checker for Java that overcomes these issues. NullAway has been carefully engineered for low overhead, so it can run as part of every build. Further, NullAway reduces annotation burden through targeted unsound assumptions, aiming for no false negatives in practice on checked code. Our evaluation shows that NullAway has significantly lower build-time overhead (1.15X) than comparable tools (2.8-5.1X). Further, on a corpus of production crash data for widely-used Android apps built with NullAway, remaining NPEs were due to unchecked third-party libraries (64%), deliberate error suppressions (17%), or reflection and other forms of post-checking code modification (17%), never due to NullAway's unsound assumptions for checked code. △ Less

Submitted 3 July, 2019; originally announced July 2019.

Comments: Pre-print of ESEC/FSE 2019 paper

Generative grasp synthesis from demonstration using parametric mixtures

Authors: Ermano Arruda, Claudio Zito, Mohan Sridharan, Marek Kopicki, Jeremy L. Wyatt

Abstract: We present a parametric formulation for learning generative models for grasp synthesis from a demonstration. We cast new light on this family of approaches, proposing a parametric formulation for grasp synthesis that is computationally faster compared to related work and indicates better grasp success rate performance in simulated experiments, showing a gain of at least 10% success rate (p < 0.05)… ▽ More We present a parametric formulation for learning generative models for grasp synthesis from a demonstration. We cast new light on this family of approaches, proposing a parametric formulation for grasp synthesis that is computationally faster compared to related work and indicates better grasp success rate performance in simulated experiments, showing a gain of at least 10% success rate (p < 0.05) in all the tested conditions. The proposed implementation is also able to incorporate arbitrary constraints for grasp ranking that may include task-specific constraints. Results are reported followed by a brief discussion on the merits of the proposed methods noted so far. △ Less

Submitted 27 June, 2019; originally announced June 2019.

Comments: Accepted for publication in the Robotics: Science and Systems (RSS) workshop on Task-Informed Grasping (TIG-II), 2019

Achieving Throughput via Fine-Grained Path Planning in Small World DTNs

Authors: Dhrubojyoti Roy, Mukundan Sridharan, Satyajeet Deshpande, Anish Arora

Abstract: We explore the benefits of using fine-grained statistics in small world DTNs to achieve high throughput without the aid of external infrastructure. We first design an empirical node-pair inter-contacts model that predicts meetings within a time frame of suitable length, typically of the order of days, with a probability above some threshold, and can be readily computed with low overhead. This temp… ▽ More We explore the benefits of using fine-grained statistics in small world DTNs to achieve high throughput without the aid of external infrastructure. We first design an empirical node-pair inter-contacts model that predicts meetings within a time frame of suitable length, typically of the order of days, with a probability above some threshold, and can be readily computed with low overhead. This temporal knowledge enables effective time-dependent path planning that can be respond to even per-packet deadline variabilities. We describe one such routing framework, REAPER (for Reliable, Efficient and Predictive Routing), that is fully distributed and self-stabilizing. Its key objective is to provide probabilistic bounds on path length (cost) and delay in a temporally fine-grained way, while exploiting the small world structure to entail only polylogarithmic storage and control overhead. A simulation-based evaluation confirms that REAPER achieves high throughput and energy efficiency across the spectrum of ultra-light to heavy network traffic, and substantially outperforms state-of-the-art single copy protocols as well as sociability-based protocols that rely on essentially coarse-grained metrics. △ Less

Submitted 17 February, 2019; originally announced February 2019.

Comments: arXiv admin note: text overlap with arXiv:1310.1162

Safe Stream-Based Programming with Refinement Types

Authors: Benno Stein, Lazaro Clapp, Manu Sridharan, Bor-Yuh Evan Chang

Abstract: In stream-based programming, data sources are abstracted as a stream of values that can be manipulated via callback functions. Stream-based programming is exploding in popularity, as it provides a powerful and expressive paradigm for handling asynchronous data sources in interactive software. However, high-level stream abstractions can also make it difficult for developers to reason about control-… ▽ More In stream-based programming, data sources are abstracted as a stream of values that can be manipulated via callback functions. Stream-based programming is exploding in popularity, as it provides a powerful and expressive paradigm for handling asynchronous data sources in interactive software. However, high-level stream abstractions can also make it difficult for developers to reason about control- and data-flow relationships in their programs. This is particularly impactful when asynchronous stream-based code interacts with thread-limited features such as UI frameworks that restrict UI access to a single thread, since the threading behavior of streaming constructs is often non-intuitive and insufficiently documented. In this paper, we present a type-based approach that can statically prove the thread-safety of UI accesses in stream-based software. Our key insight is that the fluent APIs of stream-processing frameworks enable the tracking of threads via type-refinement, making it possible to reason automatically about what thread a piece of code runs on -- a difficult problem in general. We implement the system as an annotation-based Java typechecker for Android programs built upon the popular ReactiveX framework and evaluate its efficacy by annotating and analyzing 8 open-source apps, where we find 33 instances of unsafe UI access while incurring an annotation burden of only one annotation per 186 source lines of code. We also report on our experience applying the typechecker to two much larger apps from the Uber Technologies Inc. codebase, where it currently runs on every code change and blocks changes that introduce potential threading bugs. △ Less

Submitted 8 August, 2018; originally announced August 2018.

Journal ref: Proceedings of the 2018 33rd ACM/IEEE International Conference on Automated Software Engineering

A Computational Framework for Modelling and Analyzing Ice Storms

Authors: Ranjini Swaminathan, Mohan Sridharan, Katharine Hayhoe

Abstract: Ice storms are extreme weather events that can have devastating implications for the sustainability of natural ecosystems as well as man made infrastructure. Ice storms are caused by a complex mix of atmospheric conditions and are among the least understood of severe weather events. Our ability to model ice storms and characterize storm features will go a long way towards both enabling support sys… ▽ More Ice storms are extreme weather events that can have devastating implications for the sustainability of natural ecosystems as well as man made infrastructure. Ice storms are caused by a complex mix of atmospheric conditions and are among the least understood of severe weather events. Our ability to model ice storms and characterize storm features will go a long way towards both enabling support systems that offset storm impacts and increasing our understanding of ice storms. In this paper, we present a holistic computational framework to answer key questions of interest about ice storms. We model ice storms as a function of relevant surface and atmospheric variables. We learn these models by adapting and applying supervised and unsupervised machine learning algorithms on data with missing or incorrect labels. We also include a knowledge representation module that reasons with domain knowledge to revise the output of the learned models. Our models are trained using reanalysis data and historical records of storm events. We evaluate these models on reanalyis data as well as Global Climate Model (GCM) data for historical and future climate change scenarios. Furthermore, we discuss the use of appropriate bias correction approaches to run such modeling frameworks with GCM data. △ Less

Submitted 13 May, 2018; originally announced May 2018.

Comments: 7 pages including bibliography

Type Inference for Static Compilation of JavaScript (Extended Version)

Authors: Satish Chandra, Colin S. Gordon, Jean-Baptiste Jeannin, Cole Schlesinger, Manu Sridharan, Frank Tip, Youngil Choi

Abstract: We present a type system and inference algorithm for a rich subset of JavaScript equipped with objects, structural subtyping, prototype inheritance, and first-class methods. The type system supports abstract and recursive objects, and is expressive enough to accommodate several standard benchmarks with only minor workarounds. The invariants enforced by the types enable an ahead-of-time compiler to… ▽ More We present a type system and inference algorithm for a rich subset of JavaScript equipped with objects, structural subtyping, prototype inheritance, and first-class methods. The type system supports abstract and recursive objects, and is expressive enough to accommodate several standard benchmarks with only minor workarounds. The invariants enforced by the types enable an ahead-of-time compiler to carry out optimizations typically beyond the reach of static compilers for dynamic languages. Unlike previous inference techniques for prototype inheritance, our algorithm uses a combination of lower and upper bound propagation to infer types and discover type errors in all code, including uninvoked functions. The inference is expressed in a simple constraint language, designed to leverage off-the-shelf fixed point solvers. We prove soundness for both the type system and inference algorithm. An experimental evaluation showed that the inference is powerful, handling the aforementioned benchmarks with no manual type annotation, and that the inferred types enable effective static compilation. △ Less

Submitted 18 October, 2016; v1 submitted 25 August, 2016; originally announced August 2016.

Comments: Extended version of OOPSLA 2016 paper of the same name

Trace Typing: An Approach for Evaluating Retrofitted Type Systems (Extended Version)

Authors: Esben Andreasen, Colin S. Gordon, Satish Chandra, Manu Sridharan, Frank Tip, Koushik Sen

Abstract: Recent years have seen growing interest in the retrofitting of type systems onto dynamically-typed programming languages, in order to improve type safety, programmer productivity, or performance. In such cases, type system developers must strike a delicate balance between disallowing certain coding patterns to keep the type system simple, or including them at the expense of additional complexity a… ▽ More Recent years have seen growing interest in the retrofitting of type systems onto dynamically-typed programming languages, in order to improve type safety, programmer productivity, or performance. In such cases, type system developers must strike a delicate balance between disallowing certain coding patterns to keep the type system simple, or including them at the expense of additional complexity and effort. Thus far, the process for designing retrofitted type systems has been largely ad hoc, because evaluating multiple variations of a type system on large bodies of existing code is a significant undertaking. We present trace typing: a framework for automatically and quantitatively evaluating variations of a retrofitted type system on large code bases. The trace typing approach involves gathering traces of program executions, inferring types for instances of variables and expressions occurring in a trace, and merging types according to merge strategies that reflect specific (combinations of) choices in the source-level type system design space. We evaluated trace typing through several experiments. We compared several variations of type systems retrofitted onto JavaScript, measuring the number of program locations with type errors in each case on a suite of over fifty thousand lines of JavaScript code. We also used trace typing to validate and guide the design of a new retrofitted type system that enforces fixed object layout for JavaScript objects. Finally, we leveraged the types computed by trace typing to automatically identify tag tests --- dynamic checks that refine a type --- and examined the variety of tests identified. △ Less

Submitted 4 May, 2016; originally announced May 2016.

Comments: Samsung Research America Technical Report

Report number: SRA-CSIC-2016-001 ACM Class: F.3.3

REBA: A Refinement-Based Architecture for Knowledge Representation and Reasoning in Robotics

Authors: Mohan Sridharan, Michael Gelfond, Shiqi Zhang, Jeremy Wyatt

Abstract: This paper describes an architecture for robots that combines the complementary strengths of probabilistic graphical models and declarative programming to represent and reason with logic-based and probabilistic descriptions of uncertainty and domain knowledge. An action language is extended to support non-boolean fluents and non-deterministic causal laws. This action language is used to describe t… ▽ More This paper describes an architecture for robots that combines the complementary strengths of probabilistic graphical models and declarative programming to represent and reason with logic-based and probabilistic descriptions of uncertainty and domain knowledge. An action language is extended to support non-boolean fluents and non-deterministic causal laws. This action language is used to describe tightly-coupled transition diagrams at two levels of granularity, with a fine-resolution transition diagram defined as a refinement of a coarse-resolution transition diagram of the domain. The coarse-resolution system description, and a history that includes (prioritized) defaults, are translated into an Answer Set Prolog (ASP) program. For any given goal, inference in the ASP program provides a plan of abstract actions. To implement each such abstract action, the robot automatically zooms to the part of the fine-resolution transition diagram relevant to this action. A probabilistic representation of the uncertainty in sensing and actuation is then included in this zoomed fine-resolution system description, and used to construct a partially observable Markov decision process (POMDP). The policy obtained by solving the POMDP is invoked repeatedly to implement the abstract action as a sequence of concrete actions, with the corresponding observations being recorded in the coarse-resolution history and used for subsequent reasoning. The architecture is evaluated in simulation and on a mobile robot moving objects in an indoor domain, to show that it supports reasoning with violation of defaults, noisy observations and unreliable actions, in complex domains. △ Less

Submitted 21 September, 2018; v1 submitted 16 August, 2015; originally announced August 2015.

Comments: 72 pages, 14 figures

Mixed Logical and Probabilistic Reasoning for Planning and Explanation Generation in Robotics

Authors: Zenon Colaco, Mohan Sridharan

Abstract: Robots assisting humans in complex domains have to represent knowledge and reason at both the sensorimotor level and the social level. The architecture described in this paper couples the non-monotonic logical reasoning capabilities of a declarative language with probabilistic belief revision, enabling robots to represent and reason with qualitative and quantitative descriptions of knowledge and d… ▽ More Robots assisting humans in complex domains have to represent knowledge and reason at both the sensorimotor level and the social level. The architecture described in this paper couples the non-monotonic logical reasoning capabilities of a declarative language with probabilistic belief revision, enabling robots to represent and reason with qualitative and quantitative descriptions of knowledge and degrees of belief. Specifically, incomplete domain knowledge, including information that holds in all but a few exceptional situations, is represented as a Answer Set Prolog (ASP) program. The answer set obtained by solving this program is used for inference, planning, and for jointly explaining (a) unexpected action outcomes due to exogenous actions and (b) partial scene descriptions extracted from sensor input. For any given task, each action in the plan contained in the answer set is executed probabilistically. The subset of the domain relevant to the action is identified automatically, and observations extracted from sensor inputs perform incremental Bayesian updates to a belief distribution defined over this domain subset, with highly probable beliefs being committed to the ASP program. The architecture's capabilities are illustrated in simulation and on a mobile robot in the context of a robot waiter operating in the dining room of a restaurant. △ Less

Submitted 31 July, 2015; originally announced August 2015.

Comments: 11 pages, 3 figures

On the repair time scaling wall for MANETs

Authors: Vinod Kulathumani, Mukundan Sridharan, Anish Arora, Bryan Lemon, Kenneth Parker

Abstract: The inability of practical MANET deployments to scale beyond about 100 nodes has traditionally been blamed on insufficient network capacity for supporting routing related control traffic. However, this paper points out that network capacity is significantly under-utilized by standard MANET routing algorithms at observed scaling limits. Therefore, as opposed to identifying the scaling limit for MAN… ▽ More The inability of practical MANET deployments to scale beyond about 100 nodes has traditionally been blamed on insufficient network capacity for supporting routing related control traffic. However, this paper points out that network capacity is significantly under-utilized by standard MANET routing algorithms at observed scaling limits. Therefore, as opposed to identifying the scaling limit for MANET routing from a capacity stand-point, it is instead characterized as a function of the interaction between dynamics of path failure (caused due to mobility) and path repair. This leads to the discovery of the repair time scaling wall, which is used to explain observed scaling limits in MANETs. The factors behind the repair time scaling wall are identified and techniques to extend the scaling limits are described. △ Less

Submitted 9 September, 2015; v1 submitted 25 September, 2014; originally announced September 2014.

Comments: 10 pages; Index terms: MANET, path failure, repair time, network capacity, link estimation, local routing, neighborhood discovery

Census: Fast, scalable and robust data aggregation in MANETs

Authors: Vinod Kulathumani, Anish Arora, Kenneth Parker, Mukundan Sridharan, Masahiro Nakagawa

Abstract: This paper describes Census, a protocol for data aggregation and statistical counting in MANETs. Census operates by circulating a set of tokens in the network using biased random walks such that each node is visited by at least one token. The protocol is structure-free so as to avoid high messaging overhead for maintaining structure in the presence of node mobility. It biases the random walks of t… ▽ More This paper describes Census, a protocol for data aggregation and statistical counting in MANETs. Census operates by circulating a set of tokens in the network using biased random walks such that each node is visited by at least one token. The protocol is structure-free so as to avoid high messaging overhead for maintaining structure in the presence of node mobility. It biases the random walks of tokens so as to achieve fast cover time; the bias involves short albeit multi-hop gradients that guide the tokens towards hitherto unvisited nodes. Census thus achieves a cover time of O(N/k) and message overhead of O(Nlog(N)/k) where N is the number of nodes and k the number of tokens in the network. Notably, it enjoys scalability and robustness, which we demonstrate via simulations in networks ranging from 100 to 4000 nodes under different network densities and mobility models. △ Less

Submitted 21 September, 2015; v1 submitted 25 September, 2014; originally announced September 2014.

Comments: 25 pages, technical report, index terms:random walk, MANET, statistical aggregation, gossip, local gradients

KR$^3$: An Architecture for Knowledge Representation and Reasoning in Robotics

Authors: Shiqi Zhang, Mohan Sridharan, Michael Gelfond, Jeremy Wyatt

Abstract: This paper describes an architecture that combines the complementary strengths of declarative programming and probabilistic graphical models to enable robots to represent, reason with, and learn from, qualitative and quantitative descriptions of uncertainty and knowledge. An action language is used for the low-level (LL) and high-level (HL) system descriptions in the architecture, and the definiti… ▽ More This paper describes an architecture that combines the complementary strengths of declarative programming and probabilistic graphical models to enable robots to represent, reason with, and learn from, qualitative and quantitative descriptions of uncertainty and knowledge. An action language is used for the low-level (LL) and high-level (HL) system descriptions in the architecture, and the definition of recorded histories in the HL is expanded to allow prioritized defaults. For any given goal, tentative plans created in the HL using default knowledge and commonsense reasoning are implemented in the LL using probabilistic algorithms, with the corresponding observations used to update the HL history. Tight coupling between the two levels enables automatic selection of relevant variables and generation of suitable action policies in the LL for each HL action, and supports reasoning with violation of defaults, noisy observations and unreliable actions in large and complex domains. The architecture is evaluated in simulation and on physical robots transporting objects in indoor domains; the benefit on robots is a reduction in task execution time of 39% compared with a purely probabilistic, but still hierarchical, approach. △ Less

Submitted 5 May, 2014; originally announced May 2014.

Comments: The paper appears in the Proceedings of the 15th International Workshop on Non-Monotonic Reasoning (NMR 2014)

A Little Prediction Goes a Long Way: Routing in Semi-Deterministic Delay Tolerant Networks

Authors: Dhrubojyoti Roy, Mukundan Sridharan, Satyajeet Deshpande, Anish Arora

Abstract: Realizing delay-capacity in intermittently connected mobile networks remains a largely open question, with state-of-the-art routing schemes typically focusing either on delay or on capacity. We show the feasibility of routing with both high goodput and desired delay constraints, with REAPER (for Reliable, Efficient, and Predictive Routing), a fully distributed convergecast routing framework that j… ▽ More Realizing delay-capacity in intermittently connected mobile networks remains a largely open question, with state-of-the-art routing schemes typically focusing either on delay or on capacity. We show the feasibility of routing with both high goodput and desired delay constraints, with REAPER (for Reliable, Efficient, and Predictive Routing), a fully distributed convergecast routing framework that jointly optimizes both path length and path delay. A key idea for efficient instantiation of REAPER is to exploit predictability of mobility patterns, in terms of a semi-deterministic model which appropriately captures several vehicular and human inter-contact patterns. Packets are thus routed using paths that are jointly optimal at their time of arrival, in contrast to extant DTN protocols which use time-average metrics for routing. REAPER is also self-stabilizing to changes in the mobility pattern. A simulation-based evaluation confirms that, across the spectrum of ultra-light to heavy traffics, REAPER achieves up to 135% and 200% higher throughput and up to 250% and 1666% higher energy efficiency than state-of-the-art single-copy protocols MEED-DVR and PROPHET, which optimize a single metric only, specifically, expected delay and path probability respectively. △ Less

Submitted 22 January, 2014; v1 submitted 4 October, 2013; originally announced October 2013.

Comments: This paper has been withdrawn by the authors. Withdrawn since document intended to be anonymous

Combining Answer Set Programming and POMDPs for Knowledge Representation and Reasoning on Mobile Robots

Authors: Shiqi Zhang, Mohan Sridharan

Abstract: For widespread deployment in domains characterized by partial observability, non-deterministic actions and unforeseen changes, robots need to adapt sensing, processing and interaction with humans to the tasks at hand. While robots typically cannot process all sensor inputs or operate without substantial domain knowledge, it is a challenge to provide accurate domain knowledge and humans may not hav… ▽ More For widespread deployment in domains characterized by partial observability, non-deterministic actions and unforeseen changes, robots need to adapt sensing, processing and interaction with humans to the tasks at hand. While robots typically cannot process all sensor inputs or operate without substantial domain knowledge, it is a challenge to provide accurate domain knowledge and humans may not have the time and expertise to provide elaborate and accurate feedback. The architecture described in this paper combines declarative programming and probabilistic reasoning to address these challenges, enabling robots to: (a) represent and reason with incomplete domain knowledge, resolving ambiguities and revising existing knowledge using sensor inputs and minimal human feedback; and (b) probabilistically model the uncertainty in sensor input processing and navigation. Specifically, Answer Set Programming (ASP), a declarative programming paradigm, is combined with hierarchical partially observable Markov decision processes (POMDPs), using domain knowledge to revise probabilistic beliefs, and using positive and negative observations for early termination of tasks that can no longer be pursued. All algorithms are evaluated in simulation and on mobile robots locating target objects in indoor domains. △ Less

Submitted 29 July, 2013; originally announced July 2013.