Hierarchical Frames-of-References in Learning Classifier Systems

Perceptual aliasing is one of the most important problems in Robot-navigation as a robot cannot distinguish its state via its immediate observations leading to poor decision-making. Frames-of-References-based XCS learns policies comprising of constituent-level paths (that may be aliased) integrated into a holistic-level path that has unique patterns of the environment leading to improved policy performance. However, unique references are required to identify an aliased state such that is impractical to learn a policy in an environment with multiple, sequential, aliased states, e.g., a long, uniform, corridor. This paper introduces methods for hierarchical references, which concatenate sequential aliased states to form a discrete unique state and references them using the end-of-the-series state. The experiments investigated the performance of the proposed system in partially observable environments with complex aliasing patterns, including sequential aliased states. The results showed that the proposed system overcomes the state-of-the-art system's issues in the tested problems; learning in sequential aliased states, unlike the previous system.