20th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2021), International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS), May 2021, London (fully virtual event), United Kingdom, 2021
Artificial Intelligence XXXVII 40th SGAI International Conference on Artificial Intelligence, AI 2020, Cambridge, UK, December 15–17, 2020, Proceedings, 12498, pp.317-330, 2020, Lecture Notes in Computer Science book series (LNCS), 2020
38th International Conference on Innovative Techniques and Applications of Artificial Intelligence, 2018, Cambridge, United Kingdom., 2018
Considering Markovian Decision Processes (MDPs), the meaning of an optimal policy depends on the ... more Considering Markovian Decision Processes (MDPs), the meaning of an optimal policy depends on the optimality criterion chosen. The most common approach is to define the optimal policy as the one that maximizes the sum of discounted rewards. The intuitive alternative is to maximize the average reward per step. The former has strong convergence guarantees but suffers from the dependency on a discount factor. The latter has the additional inconvenience of being insensitive to different policies with equivalent average. This paper analyzes the impact of such different criteria on a series of experiments, and then provides a threshold for the discount factor in order to ensure average optimality for discounted-optimal policies in the deterministic case.
... Yves Demazeau Antônio Carlos da Rocha Costa Paulo Martins Engel Georgi Stojanov Yves Demazeau... more ... Yves Demazeau Antônio Carlos da Rocha Costa Paulo Martins Engel Georgi Stojanov Yves Demazeau Antônio Carlos da Rocha Costa ... José Carlos Ferraz Hennemann Vice-Reitor: Prof. Pedro Cezar Dutra Fonseca Pró-Reitora Adjunta de Pós-Graduação: Profa. ...
This paper presents CALM (Constructivist Anticipatory Learning Mechanism), an agent learning mech... more This paper presents CALM (Constructivist Anticipatory Learning Mechanism), an agent learning mechanism based on a constructivist approach of AI. It is designed to deal dynamically and interactively with environments which are at the same time partially deterministic and partially observable. We describe in detail the mechanism, explaining how the learning methods operate, and how it represents a world model using a factored MDP. We analyze the kinds of environmental regularities that CALM is able to discover, trying to show that our proposition follows the way towards the construction of more abstract or high-level representational concepts.
> Context • The advent of a general artificial intelligence mechanism that learns like humans do ... more > Context • The advent of a general artificial intelligence mechanism that learns like humans do would represent the realization of an old and major dream of science. It could be achieved by an artifact able to develop its own cognitive structures following constructivist principles. However, there is a large distance between the descriptions of the intelligence made by constructivist theories and the mechanisms that currently exist. > Problem • The constructivist conception of intelligence is very powerful for explaining how cognitive development takes place. However, until now, no computational model has successfully demonstrated the underlying mechanisms necessary to realize it. In other words, the artificial intelligence (AI) community has not been able to give rise to a system that convincingly imple- ments the principles of intelligence as postulated by constructivism, and that is also capable of dealing with complex environments. > Results • This paper presents the constructivist anticipatory learning mechanism (CALM), an agent learning mechanism based on the constructivist approach of AI. It is designed to deal dynamically and interactively with environments that are at the same time partially deterministic and partially observable. CALM can model the regularities experienced in the interaction with the environment, on the sensorimotor level as well, as by constructing abstract or high-level representational concepts. The created model provides the knowledge necessary to generate the agent behavior. The paper also presents the coupled agent environment system (CAES) meta-architecture, which defines a conception of an autonomous agent, situated in the environment, embodied and intrinsically motivated. > Implications • The paper can be seen as a step towards a computational implementation of constructivist principles, on the one hand suggesting a further perspective of this refreshing movement on the AI field (which is still too steeped in a behaviorist influence and dominated by probabilistic models and narrow applied approaches), and on the other hand bringing some abstract descriptions of the cognitive process into a more concrete dimension, in the form of algorithms. > Constructivist content • The connection of this paper with constructivism is the proposal of a computational and formally described mechanism that implements important aspects of the subjective process of knowledge construction based on key ideas proposed by constructivist theories. > Key words • Factored partially observable Markov decision process (FPOMDP), computational constructivist learning mechanisms, anticipatory learning, model-based learning.
Proceedings of the fifth international joint conference on Autonomous agents and multiagent systems - AAMAS '06, 2006
In this paper we propose a method for solving reinforcement learning problems in non-stationary e... more In this paper we propose a method for solving reinforcement learning problems in non-stationary environments. The basic idea is to create and simultaneously update multiple partial models of the environment dynamics. The learning mechanism is based on the detection of context changes, that is, on the detection of significant changes in the dynamics of the environment. Based on this motivation,
20th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2021), International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS), May 2021, London (fully virtual event), United Kingdom, 2021
Artificial Intelligence XXXVII 40th SGAI International Conference on Artificial Intelligence, AI 2020, Cambridge, UK, December 15–17, 2020, Proceedings, 12498, pp.317-330, 2020, Lecture Notes in Computer Science book series (LNCS), 2020
38th International Conference on Innovative Techniques and Applications of Artificial Intelligence, 2018, Cambridge, United Kingdom., 2018
Considering Markovian Decision Processes (MDPs), the meaning of an optimal policy depends on the ... more Considering Markovian Decision Processes (MDPs), the meaning of an optimal policy depends on the optimality criterion chosen. The most common approach is to define the optimal policy as the one that maximizes the sum of discounted rewards. The intuitive alternative is to maximize the average reward per step. The former has strong convergence guarantees but suffers from the dependency on a discount factor. The latter has the additional inconvenience of being insensitive to different policies with equivalent average. This paper analyzes the impact of such different criteria on a series of experiments, and then provides a threshold for the discount factor in order to ensure average optimality for discounted-optimal policies in the deterministic case.
... Yves Demazeau Antônio Carlos da Rocha Costa Paulo Martins Engel Georgi Stojanov Yves Demazeau... more ... Yves Demazeau Antônio Carlos da Rocha Costa Paulo Martins Engel Georgi Stojanov Yves Demazeau Antônio Carlos da Rocha Costa ... José Carlos Ferraz Hennemann Vice-Reitor: Prof. Pedro Cezar Dutra Fonseca Pró-Reitora Adjunta de Pós-Graduação: Profa. ...
This paper presents CALM (Constructivist Anticipatory Learning Mechanism), an agent learning mech... more This paper presents CALM (Constructivist Anticipatory Learning Mechanism), an agent learning mechanism based on a constructivist approach of AI. It is designed to deal dynamically and interactively with environments which are at the same time partially deterministic and partially observable. We describe in detail the mechanism, explaining how the learning methods operate, and how it represents a world model using a factored MDP. We analyze the kinds of environmental regularities that CALM is able to discover, trying to show that our proposition follows the way towards the construction of more abstract or high-level representational concepts.
> Context • The advent of a general artificial intelligence mechanism that learns like humans do ... more > Context • The advent of a general artificial intelligence mechanism that learns like humans do would represent the realization of an old and major dream of science. It could be achieved by an artifact able to develop its own cognitive structures following constructivist principles. However, there is a large distance between the descriptions of the intelligence made by constructivist theories and the mechanisms that currently exist. > Problem • The constructivist conception of intelligence is very powerful for explaining how cognitive development takes place. However, until now, no computational model has successfully demonstrated the underlying mechanisms necessary to realize it. In other words, the artificial intelligence (AI) community has not been able to give rise to a system that convincingly imple- ments the principles of intelligence as postulated by constructivism, and that is also capable of dealing with complex environments. > Results • This paper presents the constructivist anticipatory learning mechanism (CALM), an agent learning mechanism based on the constructivist approach of AI. It is designed to deal dynamically and interactively with environments that are at the same time partially deterministic and partially observable. CALM can model the regularities experienced in the interaction with the environment, on the sensorimotor level as well, as by constructing abstract or high-level representational concepts. The created model provides the knowledge necessary to generate the agent behavior. The paper also presents the coupled agent environment system (CAES) meta-architecture, which defines a conception of an autonomous agent, situated in the environment, embodied and intrinsically motivated. > Implications • The paper can be seen as a step towards a computational implementation of constructivist principles, on the one hand suggesting a further perspective of this refreshing movement on the AI field (which is still too steeped in a behaviorist influence and dominated by probabilistic models and narrow applied approaches), and on the other hand bringing some abstract descriptions of the cognitive process into a more concrete dimension, in the form of algorithms. > Constructivist content • The connection of this paper with constructivism is the proposal of a computational and formally described mechanism that implements important aspects of the subjective process of knowledge construction based on key ideas proposed by constructivist theories. > Key words • Factored partially observable Markov decision process (FPOMDP), computational constructivist learning mechanisms, anticipatory learning, model-based learning.
Proceedings of the fifth international joint conference on Autonomous agents and multiagent systems - AAMAS '06, 2006
In this paper we propose a method for solving reinforcement learning problems in non-stationary e... more In this paper we propose a method for solving reinforcement learning problems in non-stationary environments. The basic idea is to create and simultaneously update multiple partial models of the environment dynamics. The learning mechanism is based on the detection of context changes, that is, on the detection of significant changes in the dynamics of the environment. Based on this motivation,
Uploads
Papers by Filipo S Perotto
realization of an old and major dream of science. It could be achieved by an artifact able to develop its own cognitive
structures following constructivist principles. However, there is a large distance between the descriptions of the intelligence made by constructivist theories and the mechanisms that currently exist.
> Problem • The constructivist conception of intelligence is very powerful for explaining how cognitive development takes place. However, until now, no computational model has successfully demonstrated the underlying mechanisms necessary to realize it. In other words, the artificial intelligence (AI) community has not been able to give rise to a system that convincingly imple-
ments the principles of intelligence as postulated by constructivism, and that is also capable of dealing with complex
environments.
> Results • This paper presents the constructivist anticipatory learning mechanism (CALM), an agent learning mechanism based on the constructivist approach of AI. It is designed to deal dynamically and interactively with environments that are at the same time partially deterministic and partially observable. CALM can model the regularities experienced in the interaction with the environment, on the sensorimotor level as well, as by constructing abstract or high-level representational concepts. The created model provides the knowledge necessary to generate the agent behavior. The paper also presents the coupled agent environment system (CAES) meta-architecture, which defines a conception of an autonomous agent, situated in the environment, embodied and intrinsically motivated.
> Implications • The paper can be seen as a step towards a computational implementation of constructivist principles,
on the one hand suggesting a further perspective of this refreshing movement on the AI field (which is still too
steeped in a behaviorist influence and dominated by probabilistic models and narrow applied approaches), and on
the other hand bringing some abstract descriptions of the cognitive process into a more concrete dimension, in the
form of algorithms.
> Constructivist content • The connection of this paper with constructivism is the proposal of a computational and formally described mechanism that implements important aspects of the subjective process of knowledge construction based on key ideas proposed by constructivist theories.
> Key words • Factored partially observable Markov
decision process (FPOMDP), computational constructivist learning mechanisms, anticipatory learning, model-based learning.
realization of an old and major dream of science. It could be achieved by an artifact able to develop its own cognitive
structures following constructivist principles. However, there is a large distance between the descriptions of the intelligence made by constructivist theories and the mechanisms that currently exist.
> Problem • The constructivist conception of intelligence is very powerful for explaining how cognitive development takes place. However, until now, no computational model has successfully demonstrated the underlying mechanisms necessary to realize it. In other words, the artificial intelligence (AI) community has not been able to give rise to a system that convincingly imple-
ments the principles of intelligence as postulated by constructivism, and that is also capable of dealing with complex
environments.
> Results • This paper presents the constructivist anticipatory learning mechanism (CALM), an agent learning mechanism based on the constructivist approach of AI. It is designed to deal dynamically and interactively with environments that are at the same time partially deterministic and partially observable. CALM can model the regularities experienced in the interaction with the environment, on the sensorimotor level as well, as by constructing abstract or high-level representational concepts. The created model provides the knowledge necessary to generate the agent behavior. The paper also presents the coupled agent environment system (CAES) meta-architecture, which defines a conception of an autonomous agent, situated in the environment, embodied and intrinsically motivated.
> Implications • The paper can be seen as a step towards a computational implementation of constructivist principles,
on the one hand suggesting a further perspective of this refreshing movement on the AI field (which is still too
steeped in a behaviorist influence and dominated by probabilistic models and narrow applied approaches), and on
the other hand bringing some abstract descriptions of the cognitive process into a more concrete dimension, in the
form of algorithms.
> Constructivist content • The connection of this paper with constructivism is the proposal of a computational and formally described mechanism that implements important aspects of the subjective process of knowledge construction based on key ideas proposed by constructivist theories.
> Key words • Factored partially observable Markov
decision process (FPOMDP), computational constructivist learning mechanisms, anticipatory learning, model-based learning.