Supervisors: Thomas C. Henderson Phone: 1-413-545-3280 Address: Computer Science Department 140 Governors Drive University of Massachusetts Amherst Amherst, MA 01003 USA
AN ALGEBRAIC APPROACH TO ABSTRACTION IN REINFORCEMENT LEARNING FEBRUARY 2004 BALARAMAN RAVINDRAN ... more AN ALGEBRAIC APPROACH TO ABSTRACTION IN REINFORCEMENT LEARNING FEBRUARY 2004 BALARAMAN RAVINDRAN B.E., MADURAI-KAMARAJ UNIVERSITY, INDIA M.Sc.(Engg.), INDIAN INSTITUTE OF SCIENCE, BANGALORE, INDIA Ph.D., UNIVERSITY OF MASSACHUSETTS AMHERST Directed by: Professor Andrew G. Barto To operate e#ectively in complex environments learning agents require the ability to form useful abstractions, that is, the ability to selectively ignore irrelevant details. Stated in general terms this is a very di#cult problem. Much of the work in this field is specialized to specific modeling paradigms or classes of problems. In this thesis we introduce an abstraction framework for Markov decision processes (MDPs) based on homomorphisms relating MDPs. We build on classical finite-state automata literature and develop a minimization framework for MDPs that can exploit structure and symmetries to derive smaller equivalent models of the problem. Since employing homomorphisms for minimization requires that the...
In cognitive robotics "ongoing development" refers to the ability to continuously build... more In cognitive robotics "ongoing development" refers to the ability to continuously build on what the system already knows, in an ongoing process, which acquires new skills and knowledge, and achieves more sophisticated levels of behaviour. Human infants are possibly the best known demonstrators of this ability; developmental psychology has many results documenting what infants can and cannot do at various ages, however we know very little about the mechanisms underlying the development. On the robotics side, creating a computational system which displays ongoing development is still an unsolved problem. There are major unsolved questions regarding the mechanisms of ongoing development, in both biological and artificial systems; for example: how to transfer existing skills to a new context, how to build on existing skills, and how to represent knowledge (or skills). The primary aim of the seminar was to bring together researchers from two communities (developmental robotics ...
Primates (including humans) have evolved subtle kinematic adaptations that enhance the capacity o... more Primates (including humans) have evolved subtle kinematic adaptations that enhance the capacity of their hands to perceive the environment as well as to manipulate it. This duality makes sense when we observe that natural environments are dynamic, uncertain, and partially observable. To successfully manipulate objects in this setting requires that uncertainties, specifically those that can cause catastrophic outcomes, are actively suppressed. We expect these fundamental challenges are not specific to biological systems and evolution, but are instead fundamental properties of the manipulation domain that arise due to the wide range of sensitivity that manual interactions can have to context. We propose that these observations suggest that the robotic analogs of manipulation behavior in the animal kingdom must actively control uncertainty in a manner that is objectand context-dependent and, thus, that balances the risk of acting against the cost of gathering additional information fro...
Abstract: This paper describes the design and preliminary implementation of two distributed smart... more Abstract: This paper describes the design and preliminary implementation of two distributed smart camera applications: a fall detector and an object finder. These functions are part of a novel suite of applications being developed to address aging in place health care ...
Proceedings of the AAAI Conference on Artificial Intelligence
We describe a robot system that autonomously acquires skills through interaction with its environ... more We describe a robot system that autonomously acquires skills through interaction with its environment. The robot learns to sequence the execution of a set of innate controllers to solve a task, extracts and retains components of that solution as portable skills, and then transfers those skills to reduce the time required to learn to solve a second task.
In cognitive robotics “ongoing development” refers to the ability to continuously build on what t... more In cognitive robotics “ongoing development” refers to the ability to continuously build on what the system already knows, in an ongoing process, which acquires new skills and knowledge, and achieves more sophisticated levels of behaviour. Human infants are possibly the best known demonstrators of this ability; developmental psychology has many results documenting what infants can and cannot do at various ages, however we know very little about the mechanisms underlying the development. On the robotics side, creating a computational system which displays ongoing development is still an unsolved problem. There are major unsolved questions regarding the mechanisms of ongoing development, in both biological and artificial systems; for example: how to transfer existing skills to a new context, how to build on existing skills, and how to represent knowledge (or skills). The primary aim of the seminar was to bring together researchers from two communities (developmental robotics and infant...
AN ALGEBRAIC APPROACH TO ABSTRACTION IN REINFORCEMENT LEARNING FEBRUARY 2004 BALARAMAN RAVINDRAN ... more AN ALGEBRAIC APPROACH TO ABSTRACTION IN REINFORCEMENT LEARNING FEBRUARY 2004 BALARAMAN RAVINDRAN B.E., MADURAI-KAMARAJ UNIVERSITY, INDIA M.Sc.(Engg.), INDIAN INSTITUTE OF SCIENCE, BANGALORE, INDIA Ph.D., UNIVERSITY OF MASSACHUSETTS AMHERST Directed by: Professor Andrew G. Barto To operate e#ectively in complex environments learning agents require the ability to form useful abstractions, that is, the ability to selectively ignore irrelevant details. Stated in general terms this is a very di#cult problem. Much of the work in this field is specialized to specific modeling paradigms or classes of problems. In this thesis we introduce an abstraction framework for Markov decision processes (MDPs) based on homomorphisms relating MDPs. We build on classical finite-state automata literature and develop a minimization framework for MDPs that can exploit structure and symmetries to derive smaller equivalent models of the problem. Since employing homomorphisms for minimization requires that the...
In cognitive robotics "ongoing development" refers to the ability to continuously build... more In cognitive robotics "ongoing development" refers to the ability to continuously build on what the system already knows, in an ongoing process, which acquires new skills and knowledge, and achieves more sophisticated levels of behaviour. Human infants are possibly the best known demonstrators of this ability; developmental psychology has many results documenting what infants can and cannot do at various ages, however we know very little about the mechanisms underlying the development. On the robotics side, creating a computational system which displays ongoing development is still an unsolved problem. There are major unsolved questions regarding the mechanisms of ongoing development, in both biological and artificial systems; for example: how to transfer existing skills to a new context, how to build on existing skills, and how to represent knowledge (or skills). The primary aim of the seminar was to bring together researchers from two communities (developmental robotics ...
Primates (including humans) have evolved subtle kinematic adaptations that enhance the capacity o... more Primates (including humans) have evolved subtle kinematic adaptations that enhance the capacity of their hands to perceive the environment as well as to manipulate it. This duality makes sense when we observe that natural environments are dynamic, uncertain, and partially observable. To successfully manipulate objects in this setting requires that uncertainties, specifically those that can cause catastrophic outcomes, are actively suppressed. We expect these fundamental challenges are not specific to biological systems and evolution, but are instead fundamental properties of the manipulation domain that arise due to the wide range of sensitivity that manual interactions can have to context. We propose that these observations suggest that the robotic analogs of manipulation behavior in the animal kingdom must actively control uncertainty in a manner that is objectand context-dependent and, thus, that balances the risk of acting against the cost of gathering additional information fro...
Abstract: This paper describes the design and preliminary implementation of two distributed smart... more Abstract: This paper describes the design and preliminary implementation of two distributed smart camera applications: a fall detector and an object finder. These functions are part of a novel suite of applications being developed to address aging in place health care ...
Proceedings of the AAAI Conference on Artificial Intelligence
We describe a robot system that autonomously acquires skills through interaction with its environ... more We describe a robot system that autonomously acquires skills through interaction with its environment. The robot learns to sequence the execution of a set of innate controllers to solve a task, extracts and retains components of that solution as portable skills, and then transfers those skills to reduce the time required to learn to solve a second task.
In cognitive robotics “ongoing development” refers to the ability to continuously build on what t... more In cognitive robotics “ongoing development” refers to the ability to continuously build on what the system already knows, in an ongoing process, which acquires new skills and knowledge, and achieves more sophisticated levels of behaviour. Human infants are possibly the best known demonstrators of this ability; developmental psychology has many results documenting what infants can and cannot do at various ages, however we know very little about the mechanisms underlying the development. On the robotics side, creating a computational system which displays ongoing development is still an unsolved problem. There are major unsolved questions regarding the mechanisms of ongoing development, in both biological and artificial systems; for example: how to transfer existing skills to a new context, how to build on existing skills, and how to represent knowledge (or skills). The primary aim of the seminar was to bring together researchers from two communities (developmental robotics and infant...
Uploads
Papers by Roderic A. Grupen