Planning Rational Behavior of Cognitive Semiotic Agents in a Dynamic Environment
Authors: 
G. S. Osipov, A. I. PanovAuthors Info & Claims
G. S. Osipov, A. I. PanovAuthors Info & ClaimsPages 502 - 516
Abstract
Abstract
This paper presents a general architecture of a cognitive semiotic agent acting in a dynamic environment. A new implementation and integration of the agent planning and learning modules are proposed to solve the symbol grounding problem. We suggest a new approach to describing the semantic level of components of the sign-based world view. The approach is used as basic for generating rational agent behavior. A formal definition of the behavior script and its use in generating the action plan of a rational agent are proposed. In conclusion, we describe a model experiment that showcases the work of a semiotic agent in a game environment.
References
[1]
Pospelov, D.A., Ten hot topics in AI studies, Iskusstv. Intellekt Prinyatie Reshenii, 2019, no. 4, pp. 3–9.
[2]
Osipov, G.S., Metody iskusstvennogo intellekta (Methods of Artificial Intelligence). M.: Fizmatlit, 2015.
[3]
Schwarting W., Alonso-Mora J., and Rus D. Planning and decision-making for autonomous vehicles, Ann. Rev. Control, Rob Auton. Syst. 2018 1 187-210
[4]
Ghallab M., Nau D., and Traverso P. Automated Planning and Acting 2016 Cambridge Cambridge Univ. Press
[5]
Rankooh M. and Ghassem-Sani G. ITSAT: An efficient SAT-based temporal planner J. Artif. Intell. Res. 2015 53 541-632
[6]
Richter S. and Westphal M. The LAMA planner: Guiding cost-based anytime planning with landmarks J. Artif. Intell. Res. 2010 39 127-177
[7]
Alford, R., Shivashankar, V., Roberts, M., Frank, J., and Aha, D., Hierarchical planning: relating task and goal decomposition with task sharing, Proc. of the Twenty-Fifth Int. Joint Conf. on Artificial Intelligence, New York, 2016, pp. 3022–3028.
[8]
Cardoso, R. and Bordini, R., Decentralised planning for multi-agent programming platforms, Proc. of the 18th Int. Conf. on Autonomous Agents and MultiAgent Systems, Montreal, 2019, pp. 799–807.
[9]
Kiselev, G.A. and Panov, A.I., Sign-based approach to the task of role distribution in the coalition of cognitive agents, Tr. St. Petersburg Inst. Inf. Ross. Akad. Nauk, 2018, no. 57, pp. 161–187.
[10]
Borrajo D., Roubíčková A., and Serina I. Progress in case-based planning ACM Comput. Surv. 2015 47 35
[11]
G.V. Rybina and Blokhin, Yu.M., Modern automated planning methods and tools and their use for control of process of integrated expert systems construction, Iskusstv. Intellekt Prinyatie Reshenii, 2015, no. 1, pp. 75–93.
[12]
Kim B., Wang Z., Kaelbling L.P., and Lozano-Pérez T. Learning to guide task and motion planning using score-space representation Int. J. Rob. Res. 2019 38 793-812
[13]
Harnad, S., The symbol grounding problem, Phys. D (Amsterdam, Neth.), 1990, vol. 42, no. 1–3, pp. 335–346.
[14]
Besold T.R. and Kühnberger K.-U. Towards integrated neural–symbolic systems for human-level AI: Two research programs helping to bridge the gaps Biol. Inspired Cognit. Archit. 2015 14 97-110
[15]
Kaelbling L.P. and Lozano-Pérez T. Integrated task and motion planning in belief space Int. J. Rob. Res. 2013 32 1194-1227
[16]
Tarasov, V., Ot mnogoagentnykh sistem k intellektual’nym organizatsiyam. Filosofiya, psihologiya, informatika (From Multi-Agent Systems to Intellectual Organizations), Moscow: Editorial URSS, 2002.
[17]
Karpov V.E. and Tarasov V.B. From collaborative robotics to social robots for assisting persons with reduced mobility: New development directions of using intellectual agents, Intellektual’nye tekhnologii i sredstva reabilitatsii i abilitatsii lyudei s ogranichennymi vozmozhnostyami (ITSR-2018) (Intellectual Technologies for Rehabilitation and Habilitation of Persons with Reduced Mobility), Moscow 2018 Moscow Mosk. Gos. Gumanitarno-Ekonomicheskii Univ.
[18]
Dorri A. Kanhere, S.S., and Jurdak, R., Multi-agent systems: A survey IEEE Access 2018 6 28573-28593
[19]
Snaider J. and Franklin S. Vector LIDA Procedia Comput. Sci. 2014 41 188-203
[20]
Fernandes L.C., Souza J.R., Pessin G., Shinzato P.Y., Sales D., Mendes C., Prado M., Klaser R., Chaves Magalhães A., Hata A., Pigatto D., Castelo Branco K., Grassi V., Osorio F.S., and Wolf D.F. CaRINA Intelligent Robotic Car: Architectural design and applications J. Syst. Archit. 2014 60 372-392
[21]
Goertzel B., Pennachin, C., and Geisweiller, N., The OpenCog framework, in Engineering General Intelligence, Part 2: The CogPrime Architecture for Integrative, Embodied AGI, Atlantis Thinking Machines, vol. 6., Paris: Atlantis Press, 2014, pp. 3–29.
[22]
Laird J. The Soar Cognitive Architecture 2012
[23]
Bothell D. ACT-R 7 Reference Manual 2015
[24]
Hélie S. and Sun R. Autonomous learning in psychologically-oriented cognitive architectures: A survey New Ideas Psychol. 2014 34 37-55
[25]
Samsonovich A. Emotional biologically inspired cognitive architecture Biol. Inspired Cognit. Archit. 2013 6 109-125
[26]
George D. and Hawkins J. Towards a mathematical theory of cortical micro-circuits PLoS Comput. Biol. 2009 5 e1000532
[27]
Hawkins J., Ahmad S., and Cui Y. A theory of how columns in the neocortex enable learning the structure of the world Front. Neural Circuits 2017 11 81
[28]
George, D., Lehrach, W., Kansky, K., Lázaro-Gredilla, M., Laan, C., Marthi, B., Lou, X., Meng, Z., Liu, Y., Wang, H., Lavin, A., and Scott Phoenix, D., A generative vision model that trains with high data efficiency and breaks text-based CAPTCHAs, Science, 2017, vol. 358, no. 6368, p. eaag2612.
[29]
Schmidhuber J. Deep learning in neural networks: An overview Neural Networks 2015 61 85-117
[30]
Manhaeve, R., Dumančić, S., Kimmig, A., Demeester, T., and De Raedt, L., DeepProbLog : Neural probabilistic logic programming, Advances in Neural Information Processing Systems, Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R., Eds., Curran Associates, 2018, vol. 31. arXiv:1805.10872v2 [cs.AI]
[31]
Besold, T., d’Avila Garcez, A., Bader, S., Bowman, H., Domingos, P., Hitzler, P., Kuehnberger, K.-U., Lamb, L.C., Lowd, D., Lima, P.M.V.L., de Penning, L., Pinkas, G., Poon, H., and Zaverucha, G., Neural-symbolic learning and reasoning: A survey and interpretation. arXiv:1711.03902 [cs.AI]
[32]
Ghidini C. and Serafini L. Distributed first order logic Artif. Intell. 2017 253 1-39
[33]
Schaul, T., Horgan, D., Gregor, K., and Silver, D., Universal value function approximators, Proc. of the 32nd Int. Conf. on Machine Learning, Lille, 2015, vol. 37, pp. 1312–1320.
[34]
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M., Playing Atari with deep reinforcement learning. arXiv:1312.5602 [cs.LG]
[35]
Vinyals O., Babuschkin I., Czarnecki W.M., Mathieu M., Dudzik A., Chung J., Choi D.H., Powell R., Edwalds T., and Georgiev P. Grandmaster level in StarCraft II using multi-agent reinforcement learning Nature 2019 575 350-354
[36]
Silver D., Hubert T., Schrittwieser J., Antonoglou I., Lai M., Guez A., Lanctot M., Sifre L., Kumaran D., Graepel T., Lillicrap T., Simonyan K., and Hassabis D. A general reinforcement learning algorithm that masters chess, shogi, and go through self-play Science 2018 362 1140-1144
[37]
Schrittwieser J., Antonoglou I., Hubert T., Simonyan K., Sifre L., Schmitt S., Guez A., Lockhart E., Hassabis D., Graepel T., Lillicrap T., and Silver D. Mastering Atari, Go, chess and shogi by planning with a learned model Nature 2020 588 604-609
[38]
Kuznetsova, Yu., Osipov, G., Panov, A., Petrov, A., and Chudova, N., Modeling behavior controlled by consciousness, Sistemnyi analiz i informatsionnye tekhnologii. Tr. Chetvertoi Mezhdunarodnoi konf. (Systems Analysis and Information Technologies: Theses of the 4th Int. Conf.), Abzakovo, Russia, 2011, Chelyabinsk: Chelyabinsk Gos. Univ., 2011, vol. 1, pp. 6–13.
[39]
Osipov, G.S., Panov, A.I., Chudova, N.V., and Kuznetsova, Yu.M., Znakovaya kartina mira sub”ekta povedeniya (Sign World View of a Behaver), Moscow: Fizmatlit, 2018.
[40]
Osipov G.S., Panov A.I., and Chudova N.V. Behavior control as a function of consciousness. I. World model and goal setting J. Comput. Syst. Sci. Int. 2014 53 517-529
[41]
Chudova, N.V., Model of the world conceptualizing for the purpose of deliberate behavior simulation, Iskusstv. Intellekt Prinyatie Reshenii, 2012, no. 2, pp. 51–62.
[42]
Paraense A.L.O., Raizer K., and Gudwin R.R. A machine consciousness approach to urban traffic control Biol. Inspired Cognit. Archit. 2016 15 61-73
[43]
Madl T., Franklin S., Chen K., and Trappl R. A computational cognitive framework of spatial memory in brains and robots Cognit. Syst. Res. 2018 47 147-172
[44]
Osipov G.S. Intelligent dynamic systems Sci. Tech. Inf. Process. 2010 37 259-264
[45]
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O., Proximal policy optimization algorithms. arXiv:1707.06347 [cs.LG]
[46]
Haarnoja, T., Zhou, A., Hartikainen, K., Tucker, G., Ha, S., Tan, J., Kumar, V., Zhu, H., Gupta, A., Abbeel, P., and Levine, S. Soft actor-critic algorithms and applications. arXiv:1812.05905 [cs.LG]
[47]
Choi D. and Langley P. Evolution of the Icarus cognitive architecture Cognit. Syst. Res. 2018 48 25-38
[48]
Wu, Yi, Wu, Yu., Tamar, A., Russell, S., Gkioxari, G., and Tian, Y., Learning and planning with a semantic model. arXiv:1809.10842 [cs.LG]
[49]
Francois-Lavet, V., Bengio, Y., Precup, D., and Pineau, J., Combined reinforcement learning via abstract representations, Proc. AAAI Conf. Artif. Intell., 2019, vol. 33, no. 1, pp. 3582–3589.
[50]
Minsky, M.L., Frame-system theory, Thinking, Johnson-Laird, P.N. and Wason, P.C., Eds., Readings in Cognitive Science, Cambridge, Mass.: Cambridge Univ. Press, 1977.
[51]
Pichotta, K. and Mooney, R.J., Learning statistical scripts with LSTM recurrent neural networks, Proc. AAAI Conf. Artif. Intell., vol. 30, no. 1, pp. 2800–2806. https://ojs.aaai.org/index.php/AAAI/article/view/10347.
[52]
Donadello, I., Serafini, L., and d’Avilla Garcez, A., Logic tensor networks for semantic image interpretation, Proc. of the Twenty-Sixth Int. Conf. on Artificial Intelligence, Melbourne, 2017, pp. 1596–1602.
[53]
Kleyko D., Rahimi A., Rachkovskij D.A., Osipov E., and Rabaey J. Classification and recall with binary hyperdimensional computing: Tradeoffs in choice of density and mapping characteristics IEEE Trans. Neural Networks Learn. Syst. 2018 29 5880-5898
[54]
Leont’ev, A.N., Deyatel’nost’. Soznanie. Lichnost’ (Activity. Consciousness. Personality). Moscow: Politizdat, 1977.
[55]
Vygotskij L.S. Thought and Speech, Psikhologiya razvitiya cheloveka (Psychology of Personal Growth) 2005 Moscow Eksmo
[56]
Chudova, N.V., Some pertinent problems of modeling goal-setting in sign-based world models: A psychologist’s perspective, Sci. Tech. Inf. Process., 2021, vol. 48, no. 6, pp. 423–429. https://doi.org/10.3103/S0147688221060010
[57]
Chudova, N.V., Psychological aspects of planning in sign world view, Shestnadtsataya natsional’naya konferentsiya po iskusstvennomu intellektu s mezhdunarodnym uchastiem KII-2018 (16th National Conf. on Artificial Intelligence with Int. Participation), 2018, pp. 88–95.
[58]
Panov, A.I. and Yakovlev, K.S., On interaction of strategic and tactical planning for the coalition of agents in dynamic environment, Iskusstv. Intellekt Prinyatie Reshenii, 2016, no. 4, pp. 68–78.
[59]
Kiselev G. and Panov A. Hierarchical psychologically inspired planning for human-robot interaction tasks, Interactive Collaborative Robotics. ICR 2019 2019 Cham Springer
[60]
Osipov G.S., Panov A.I., and Chudova N.V. Behavior control as a function of consciousness. II. Synthesis of a behavior plan J. Comput. Syst. Sci. Int. 2015 54 882-896
[61]
Panov A.I. Behavior planning of intelligent agent with sign world model Biol. Inspired Cognit. Archit. 2017 19 21-31
[62]
Chudova, N.V. and Kuznetsova, Yu.M., A conceptual model of self-consciousness for the sign world view of an intellectual agent, Sci. Tech. Inf. Process., 2019, vol. 46, no. 6, pp. 367–373. https://doi.org/10.3103/S0147688219060017
[63]
Osipov, G.S. and Pospelov, D.A., Applied semiotics, Novosti Iskusstv. Intellekta, 1999, no. 1, pp. 9–35.
[64]
Panov, A.I., Formation of an image component of knowledge of the cognitive agent with a sign-based model of worldview, Inf. Tekhnol. Vychislitel’nye Sist., 2018, no. 4, pp. 84–96.
[65]
Osipov, G.S., Sign-based representation and word model of actor, IEEE 8th Int. Conf. on Intelligent Systems (IS), 2016, Sofia, pp. 22–26.
[66]
Osipov, G.S., Signs-based vs. symbolic models, Advances in Artificial Intelligence and Soft Computing, Sidorov, G. and Galicia-Haro, S., Eds., Lecture Notes in Computer Science, vol. 9413, Cham: Springer, 2015, pp. 3–11.
[67]
Osipov G.S. and Panov A.I. Relationships and operations in a sign-based world model of the actor Sci. Tech. Inf. Process. 2018 45 317-330
[68]
George, D., How the brain might work: a hierarchical and temporal model for learning and recognition, PhD Dissertation, Stanford: Stanford University, 2008.
[69]
Hengst, B., Hierarchical approaches, Reinforcement Learning, Wiering, M. and van Otterlo, M., Eds., Adaptation, Learning, and Optimization, vol. 12, Berlin: Springer, 2012, pp. 293–323.
[70]
Levy, A., Platt, R., and Saenko, K., Hierarchical actor-critic. arXiv:1712.00948v3 [cs.AI]
[71]
Bacon, P.-L., Harb, J., and Precup, D., The option-critic architecture, Proc. AAAI Conf. Artif. Intell., 2017, vol. 31, no. 1. https://ojs.aaai.org/index.php/AAAI/article/view/10916.
[72]
Suvorova, M.I., Kobozeva, M.V., Sokolova, E.G., and Toldova, S.Yu., Extraction of schema knowledge from text documents: Part I. Problem formulation and method overview, Sci. Tech. Inf. Process., 2021, vol. 48, no. 6, pp. 517–523. https://doi.org/10.3103/S0147688221060125
[73]
Zolotova, G.A., Onipenko, N.K., and Sidorova, M.Yu., Kommunikativnaya grammatika russkogo yazyka (Communicative Grammatic of Russian Language), Moscow: Inst. Russkogo Yazyka Vinogradova Ross. Akad. Nauk, 2004.
[74]
Gorodetskiy A., Shlychkova A., and Panov A.I. Delta Schema Network in model-based reinforcement learning, Artificial General Intelligence. AGI 2020 2020 Cham Springer
[75]
Albus J.S. and Barbera A.J. RCS: A cognitive architecture for intelligent multi-agent systems Ann. Rev. Control 2005 29 87-99
[76]
Fedunov, B.E., “Electronic pilot”: point of no return will not be passed. Onboard real-time advisory expert systems of tactical level for manned aerial vehicles, Aviapanorama, 2016, no. 1, p. 9.
[77]
Fedunov B.E. Artificial intelligence agents in the knowledge databases of onboard real-time advisory expert systems for the typical situations of the functioning of an anthropocentric object J. Comput. Syst. Sci. Int. 2019 58 932-944
Recommendations
Action and planning in embedded agents
Embedded agents are computer systems that sense and act on their environments, monitoring complex dynamic conditions and affecting the environment in goal-directed ways. This paper briefly reviews the situated automata approach to agent design and ...
Normative KGP agents
We extend the logical model of agency known as the KGP model, to support agents with normative concepts, based on the roles an agent plays and the obligations and prohibitions that result from playing these roles. The proposed framework illustrates how ...
Comments
Information & Contributors
Information
Published In
© Allerton Press, Inc. 2021. ISSN 0147-6882, Scientific and Technical Information Processing, 2021, Vol. 48, No. 6, pp. 502–516. © Allerton Press, Inc., 2021. Russian Text © The Author(s), 2020, published in Iskusstvennyi Intellekt i Prinyatie Reshenii, 2020, No. 4, pp. 80–100.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 01 December 2021
Received: 01 September 2020
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 14 Jan 2025