research-article

Open access

Agents United: An Open Platform for Multi-Agent Conversational Systems

IVA '21: Proceedings of the 21st ACM International Conference on Intelligent Virtual Agents

Pages 17 - 24

https://doi.org/10.1145/3472306.3478352

Published: 14 September 2021 Publication History

Abstract

The development of applications with intelligent virtual agents (IVA) often comes with integration of multiple complex components. In this article we present the Agents United Platform: an open source platform that researchers and developers can use as a starting point to setup their own multi-IVA applications.

The new platform provides developers with a set of integrated components in a sense-remember-think-act architecture. Integrated components are a sensor framework, memory component, Topic Selection Engine, interaction manager (Flipper), two dialogue execution engines, and two behaviour realisers (ASAP and GRETA) of which the agents can seamlessly interact with each other.

This article discusses the platform and its individual components. It also highlights some of the novelties that arise from the integration of components and elaborates on directions for future work.

References

[1]

T. Beinema, H. op den Akker, and H. Hermens. 2018. Creating an artificial coaching engine for multi-domain conversational coaches in eHealth applications. In AAMAS Workshop on Intelligent Conversation Agents in Home and Geriatric Care Applications (ICA-HoGeCa), E. André, T.W. Bickmore, S. Vrochidis, and L. Wanner (Eds.), Vol. 2338. CEUR Workshop Proceedings, Aachen, 35--39.

[2]

T. Beinema, H. op den Akker, L. van Velsen, and H. Hermens. 2021. Tailoring coaching strategies to users' motivation in a multi-agent health coaching application. Computers in Human Behavior 121 (2021), 106787. https://doi.org/10.1016/j.chb.2021.106787

[3]

R.J. Beun, S. Fitrianie, F. Griffioen-Both, S. Spruit, C. Horsch, J. Lancee, and W. P. Brinkman. 2017. Talk and tools: The best of both worlds in mobile user interfaces for e-coaching. Personal and Ubiquitous Computing 21, 4 (2017), 661--674. https://doi.org/10.1007/s00779-017-1021-5

Digital Library

[4]

F. Bex, J. Lawrence, and C. Reed. 2014. Generalising argument dialogue with the Dialogue Game Execution Platform. In 5th International Conference on Computational Models of Argument (COMMA '14). IOS Press BV, Amsterdam, Netherlands, 141--152. https://doi.org/10.3233/978-1-61499-436-7-141

[5]

A. Cafaro, M. Bruijnes, J. van Waterschoot, C. Pelachaud, M. Theune, and D. Heylen. 2017. Selecting and expressing communicative functions in a SAIBA-compliant agent framework. In 17th ACM International Conference on Intelligent Virtual Agents (IVA '17). Springer, Switzerland, 73--82.

[6]

G. Calbris. 2011. Elements of meaning in gesture. Vol. 5. John Benjamins Publishing, Amsterdam, The Netherlands.

[7]

A. Cienki. 2005. Image schemas and gesture. From Perception to Meaning: Image Schemas in Cognitive Linguistics 29 (2005), 421--442.

[8]

R.A.J. de Vries, K.P. Truong, C. Zaga,J.Li, and V. Evers. 2017. A word of advice: How to tailor motivational text messages based on behavior change theory to personality and gender. Personal and Ubiquitous Computing 21, 4 (2017), 675--687. https://doi.org/10.1007/s00779-017-1025-1

Digital Library

[9]

M. Elvir, A. Gonzalez, C. Walls, and B. Wilder. 2017. Remembering a Conversation - A Conversational Memory Architecture for Embodied Conversational Agents. Journal of Intelligent Systems 26, 1 (2017), 1--21. https://doi.org/10.1515/jisys-2015-0094

[10]

J. Kolkmeier, M. Bruijnes, D. Reidsma, and D. Heylen. 2017. An ASAP Realizer-Unity3D bridge for virtual and mixed reality applications. In 17th ACM International Conference on Intelligent Virtual Agents (IVA '17), Vol. LNCS-10498. Springer, Berlin, Heidelberg, 227--230. https://doi.org/10.1007/978-3-319-67401-8_27

[11]

K. Konsolakis, H. Hermens, and O. Banos. 2019. A novel framework for the holistic monitoring and analysis of human behaviour. In 13th International Conference on Ubiquitous Computing and Ambient Intelligence (UCAmI '19), Vol. 31. MDPI, Basel, Switzerland, 43.

[12]

K. Konsolakis, H. Hermens, C. Villalonga, M. Vollenbroek-Hutten, and O. Banos. 2018. Human behaviour analysis through smartphones. In 12th International Conference on Ubiquitous Computing and Ambient Intelligence (UCAmI '18), Vol. 2. MDPI, Basel, Switzerland, 1243.

[13]

S. Kopp, B. Krenn, S. Marsella, A.N. Marshall, C. Pelachaud, H. Pirker, K.R. Thórisson, and H. Vilhjálmsson. 2006. Towards a common framework for multimodal generation: The behavior markup language. In 6th ACM International Conference on Intelligent Virtual Agents (IVA '06), Vol. LNCS-4133. Springer, Berlin, Heidelberg, 205--217. https://doi.org/10.1007/11821830_17

Digital Library

[14]

J. Lawrence, M. Snaith, B. Konat, K. Budzynska, and C. Reed. 2017. Debating technology for dialogical argument: Sensemaking, engagement and analytics. ACM Transactions on Internet Technology 17, 3 (2017), 24:1-24:23.

Digital Library

[15]

M. Mancini and C. Pelachaud. 2008. Distinctiveness in multimodal behaviors. In 7th International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS '08). International Foundation for Autonomous Agents and Multiagent Systems, Liverpool, United Kingdom, 159--166.

Digital Library

[16]

M. Mancini and C. Pelachaud. 2008. The FML-APML language. In Proceedings of the Workshop on FML at AAMAS, Vol. 8. International Foundation for Autonomous Agents and Multiagent Systems, Liverpool, United Kingdom, 1--5.

[17]

R. Niewiadomski, E. Bevacqua, M. Mancini, and C. Pelachaud. 2009. Greta: An interactive expressive ECA system. 8th International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS '09))2 (2009), 1427--1428.

Digital Library

[18]

H. op den Akker, V.M. Jones, and H.J. Hermens. 2014. Tailoring real-time physical activity coaching systems: A literature survey and model. User Modeling and User-Adapted Interaction 24, 5 (2014), 351--392. https://doi.org/10.1007/s11257-014-9146-y

Digital Library

[19]

H. op den Akker, R. op den Akker, T. Beinema, O. Banos, D. Heylen, A. Pease, C. Pelachaud, V. Traver Salcedo, S. Kyriazakos, and H. Hermens. 2018. Council of Coaches: A novel holistic behavior change coaching approach. In 4th International Conference on Information and Communication Technologies for Ageing Well and e-Health (ICT4AWE 2018). SciTePress, Setúbal, Portugal, 978--989. https://doi.org/10.5220/0006787702190226

[20]

P. Pace, R. Gravina, G. Aloi, G. Fortino, A. Fides-Valero, G. Ibanez-Sanchez, V. Traver, C.E. Palau, and D.C. Yacchirema. 2017. IoT platforms interoperability for active and assisted living healthcare services support. In 2017 Global Internet of Things Summit (GIoTS). 1--6. https://doi.org/10.1109/GIOTS.2017.8016250

[21]

H. Prakken. 2010. An abstract framework for argumentation with structured arguments. Argument & Computation 1, 2 (2010), 93--124.

[22]

B. Ravenet, C. Clavel, and C. Pelachaud. 2018. Automatic nonverbal behavior generation from image schemas. In 17th International Conference on Autonomous Agents and Multiagent Systems (AAMAS '18). International Foundation for Autonomous Agents and Multiagent Systems, Liverpool, United Kingdom, 1667--1674.

Digital Library

[23]

B. Ravenet, C. Pelachaud, C. Clavel, and S. Marsella. 2018. Automating the production of communicative gestures in embodied characters. Frontiers in Psychology 9 (2018), 1144.

[24]

D. Reidsma and H. van Welbergen. 2013. AsapRealizer in practice - A modular and extensible architecture for a BML Realizer. Entertainment Computing 4, 3 (aug 2013), 157--169. https://doi.org/10.1016/j.entcom.2013.05.001

[25]

D. Richards and K. Bransky. 2014. ForgetMeNot: What and how users expect intelligent virtual agents to recall and forget personal conversational content. International Journal of Human-Computer Studies 72, 5 (2014), 460--476.

[26]

C.L. Sidner, T. Bickmore, B. Nooraie, C. Rich, L. Ring, M. Shayganfar, and L. Vardoulakis. 2018. Creating new technologies for companionable agents to support isolated older adults. ACM Transactions on Interactive Intelligent Systems (TiiS) 8, 3 (2018), 1--27.

Digital Library

[27]

M. Snaith. 2020. An argument-based framework for selecting dialogue move types and content. In 8th International Conference on Computational Models of Argument (COMMA '20). IOS Press BV, Amsterdam, Netherlands, 355--362.

[28]

M. Snaith, N. Conway, T. Beinema, D. De Franco, A. Pease, R. Kantharaju, M. Janier, G. Huizing, C. Pelachaud, and H. op den Akker. 2021. A multimodal corpus of simulated consultations between a patient and multiple healthcare professionals. Language Resources and Evaluation (2021). https://doi.org/10.1007/s10579-020-09526-0

[29]

M. Snaith, D. De Franco, T. Beinema, H. op den Akker, and A. Pease. 2018. A dialogue game for multi-party goal-setting in health coaching. In 7th International Conference on Computational Models of Argument (COMMA '18). IOS Press BV, Amsterdam, Netherlands, 337--344.

[30]

M. Snaith, J. Lawrence, A. Pease, and C. Reed. 2020. A modular platform for argument and dialogue. 9th International Conference on Computational Models of Argument (COMMA '20), 473--474. https://doi.org/10.3233/FAIA200540

[31]

M.R. Tazari, F. Furfari, A. Fides-Valero, S. Hanke, O. Höftberger, D. Kehagias, M. Mosmondor, R. Wichert, and P. Wolf. 2012. The universAAL Reference Model for AAL. In Handbook of Ambient Assisted Living: Technology for Healthcare, Rehabilitation and Well-being, J.C. Augusto, M. Huch, A. Kameas, J. Maitland, P. McCullagh, J. Roberts, A. Sixsmith, and R. Wichert (Eds.). IOS Press BV, Amsterdam, Netherlands, 610--625.

[32]

M. ter Maat and D. Heylen. 2011. Flipper: An information state component for spoken dialogue systems. In 11th ACM International Workshop on Intelligent Virtual Agents (IVA '11). Springer, Berlin, Heidelberg, 470--472.

Digital Library

[33]

J. van Waterschoot, M. Bruijnes, J. Flokstra, D. Reidsma, D. Davison, M. Theune, and D. Heylen. 2018. Flipper 2.0: A pragmatic dialogue engine for embodied conversational agents. In 18th ACM International Conference on Intelligent Virtual Agents (IVA '18). ACM, New York, NY, United States, 43--50.

Digital Library

[34]

H. van Welbergen, D. Reidsma, and S. Kopp. 2012. An incremental multimodal realizer for behavior co-articulation and coordination. In 12th ACM International Conference on Intelligent Virtual Agents (IVA '12), Vol. LNCS-7502. Springer, Berlin, Heidelberg, 175--188. https://doi.org/10.1007/978-3-642-33197-8_18

Digital Library

[35]

H. van Welbergen, D. Reidsma, Z.M. Ruttkay, and J. Zwiers. 2010. Elckerlyc - A BML Realizer for continuous, multimodal interaction with a Virtual Human. Journal on Multimodal User Interfaces 3, 4 (2010), 271--284. https://doi.org/10.1007/s12193-010-0051-3

[36]

H. van Welbergen, D. Reidsma, and J. Zwiers. 2013. Multimodal plan representation for adaptable BML scheduling. Autonomous Agents and Multi-Agent Systems 27, 2 (2013), 305--327. https://doi.org/10.1007/s10458-012-9217-6

[37]

H. van Welbergen, R. Yaghoubzadeh, and S. Kopp. 2014. AsapRealizer 2.0: The next steps in fluent behavior realization for ECAs. In 14th ACM International Conference on Intelligent Virtual Agents (IVA '14), Vol. LNCS-8637. Springer, Cham, 449--462. https://doi.org/10.1007/978-3-319-09767-1_56

[38]

S. Wells and C.A. Reed. 2012. A domain specific language for describing diverse systems of dialogue. Journal of Applied Logic 10, 4 (2012), 309--329.

[39]

J. Wohlfahrt-Laymann, H. Hermens, C. Villalonga, M. Vollenbroek-Hutten, and O. Banos. 2018. Enabling remote assessment of cognitive behaviour through mobile experience sampling. In 2018 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops). IEEE, Athens, Greece, 794--799.

[40]

Y.Xu, C. Pelachaud, and S. Marsella. 2014. Compound gesture generation: A model based on ideational units. In 14th ACM International Conference on Intelligent Virtual Agents (IVA '14), Vol. LNCS-8637. Springer, Switzerland, 477--491.

[41]

F. Yunus, C. Clavel, and C. Pelachaud. 2021. Sequence-to-Sequence Predictive Model: From Prosody To Communicative Gestures. In Digital Human Modeling and Applications in Health, Safety, Ergonomics and Risk Management. Human Body, Motion and Behavior (HCII '21). Springer, Berlin, Heidelberg, 1--14.

Cited By

Wang PMiller MQueiroz ABailenson J(2024)Socially Late, Virtually Present: The Effects of Transforming Asynchronous Social Interactions in Virtual RealityProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642244(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642244
Al Owayyed MTielman MHartholt ASpecht MBrinkman W(2024)Agent-based social skills training systems: the ARTES architecture, interaction characteristics, learning theories and future outlooksBehaviour & Information Technology10.1080/0144929X.2024.2374891(1-28)Online publication date: 20-Jul-2024
https://doi.org/10.1080/0144929X.2024.2374891
Knob PPinho GSilva GMontanha RPeres VAraujo VMusse S(2024)Surveying the evolution of virtual humans expressiveness toward real humansComputers & Graphics10.1016/j.cag.2024.104034123(104034)Online publication date: Oct-2024
https://doi.org/10.1016/j.cag.2024.104034
Show More Cited By

Index Terms

Agents United: An Open Platform for Multi-Agent Conversational Systems
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Natural language interfaces
    2. Interactive systems and tools
      1. User interface toolkits
  2. Interaction design
    1. Systems and tools for interaction design
2. Information systems
  1. Information systems applications
    1. Collaborative and social computing systems and tools
      1. Open source software

Recommendations

The Application of ZEUS Agents in Manufacturing Environments
DEXA '01: Proceedings of the 12th International Workshop on Database and Expert Systems Applications

Abstract: The concept of Holonic Manufacturing Systems (HMS) is the key to meet future demands of highly dynamic production environments. A hierarchy of holons, which are autonomous and cooperative entities, represents the internal structure of a HMS, ...
Cooperating Agents for Holonic Manufacturing
Proceedings of the 9th ECCAI-ACAI/EASSS 2001, AEMAS 2001, HoloMAS 2001 on Multi-Agent-Systems and Applications II-Selected Revised Papers

In a Cooperating Knowledge Based System (CKBS) an agent is modelled as an autonomous and cooperative knowledge based system, capable of executing some tasks in the most effective way, using an engineering paradigm. The paper presents an abstract CKBS ...
Towards a distributed tool platform based on mobile agents
MATES'05: Proceedings of the Third German conference on Multiagent System Technologies

Nowadays many software development (SD) projects are placed in a distributed setting, concerning both the software itself and the resources, processes and actors needed to create or maintain the software. Therefore, tools and methods to support software ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

IVA '21: Proceedings of the 21st ACM International Conference on Intelligent Virtual Agents

September 2021

238 pages

ISBN:9781450386197

DOI:10.1145/3472306

Copyright © 2021 Owner/Author.

This work is licensed under a Creative Commons Attribution-NonCommercial International 4.0 License.

Sponsors

SIGAI: ACM Special Interest Group on Artificial Intelligence

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 September 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Horizon 2020 Framework Programme

Conference

IVA '21

Sponsor:

SIGAI

IVA '21: ACM International Conference on Intelligent Virtual Agents

September 14 - 17, 2021

Virtual Event, Japan

Acceptance Rates

Overall Acceptance Rate 53 of 196 submissions, 27%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
686
Total Downloads

Downloads (Last 12 months)262
Downloads (Last 6 weeks)42

Reflects downloads up to 01 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wang PMiller MQueiroz ABailenson J(2024)Socially Late, Virtually Present: The Effects of Transforming Asynchronous Social Interactions in Virtual RealityProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642244(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642244
Al Owayyed MTielman MHartholt ASpecht MBrinkman W(2024)Agent-based social skills training systems: the ARTES architecture, interaction characteristics, learning theories and future outlooksBehaviour & Information Technology10.1080/0144929X.2024.2374891(1-28)Online publication date: 20-Jul-2024
https://doi.org/10.1080/0144929X.2024.2374891
Knob PPinho GSilva GMontanha RPeres VAraujo VMusse S(2024)Surveying the evolution of virtual humans expressiveness toward real humansComputers & Graphics10.1016/j.cag.2024.104034123(104034)Online publication date: Oct-2024
https://doi.org/10.1016/j.cag.2024.104034
Langener SKolkmeier JVanDerNagel JKlaassen Rvan Manen JHeylen D(2023)Development of an Alcohol Refusal Training in Immersive Virtual Reality for Patients With Mild to Borderline Intellectual Disability and Alcohol Use Disorder: Cocreation With Experts in Addiction CareJMIR Formative Research10.2196/425237(e42523)Online publication date: 26-Apr-2023
https://doi.org/10.2196/42523
Brinkschulte LSchlögl SMonz ASchöttle PJanetschek M(2022)Perspectives on Socially Intelligent Conversational AgentsMultimodal Technologies and Interaction10.3390/mti60800626:8(62)Online publication date: 25-Jul-2022
https://doi.org/10.3390/mti6080062
Beinema Top den Akker HHofs Dvan Schooten B(2022)The WOOL Dialogue Platform: Enabling Interdisciplinary User-Friendly Development of Dialogue for Conversational AgentsOpen Research Europe10.12688/openreseurope.14279.12(7)Online publication date: 13-Jan-2022
https://doi.org/10.12688/openreseurope.14279.1
Hartholt AMozgai S(2022)Platforms and Tools for SIA Research and DevelopmentThe Handbook on Socially Interactive Agents10.1145/3563659.3563668(261-304)Online publication date: 27-Oct-2022
https://dl.acm.org/doi/10.1145/3563659.3563668
Mahfoudi MMeyer AGaudin TBuendia ABouakaz S(2022)Emotion Expression in Human Body Posture and Movement: A Survey on Intelligible Motion Factors, Quantification and ValidationIEEE Transactions on Affective Computing10.1109/TAFFC.2022.322625214:4(2697-2721)Online publication date: 2-Dec-2022
https://dl.acm.org/doi/10.1109/TAFFC.2022.3226252
Beinema Top den Akker HHermens Hvan Velsen L(2022)What to Discuss?—A Blueprint Topic Model for Health Coaching Dialogues With Conversational AgentsInternational Journal of Human–Computer Interaction10.1080/10447318.2022.204188439:1(164-182)Online publication date: 9-May-2022
https://doi.org/10.1080/10447318.2022.2041884

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

Affiliations

Tessa Beinema

University of Twente, Enschede, the Netherlands and Roessingh Research and Development, Enschede, the Netherlands

Daniel Davison

University of Twente, Enschede, the Netherlands

Dennis Reidsma

University of Twente, Enschede, the Netherlands

Oresti Banos

University of Twente, Enschede, the Netherlands and University of Granada, Granada, Spain

Merijn Bruijnes

University of Twente, Enschede, the Netherlands and Delft University of Technology, Delft, the Netherlands

Brice Donval

Sorbonne Université, Paris, France

Álvaro Fides Valero

Universitat Politècnica de València, València, Spain

Dirk Heylen

University of Twente, Enschede, the Netherlands

Dennis Hofs

Roessingh Research and Development, Enschede, the Netherlands

Gerwin Huizing

University of Twente, Enschede, the Netherlands

Reshamashree B. Kantharaju

Sorbonne Université, Paris, France

Randy Klaassen

University of Twente, Enschede, the Netherlands

Jan Kolkmeier

University of Twente, Enschede, the Netherlands

Kostas Konsolakis

University of Twente, Enschede, the Netherlands

Alison Pease

University of Dundee, Dundee, United Kingdom

Catherine Pelachaud

CNRS-ISIR, Paris, France

Donatella Simonetti

University of Twente, Enschede, the Netherlands and Sorbonne Université, Paris, France

Mark Snaith

University of Dundee, Dundee, United Kingdom and Robert Gordon University, Aberdeen, United Kingdom

Vicente Traver

Universitat Politècnica de València, València, Spain

Jorien van Loon

University of Twente, Enschede, the Netherlands

Jacky Visser

University of Dundee, Dundee, United Kingdom

Marcel Weusthof

University of Twente, Enschede, the Netherlands

Fajrian Yunus

Sorbonne Université, Paris, France

Hermie Hermens

University of Twente, Enschede, the Netherlands and Roessingh Research and Development, Enschede, the Netherlands

Harm op den Akker

University of Twente, Enschede, the Netherlands, Roessingh Research and Development, Enschede, the Netherlands and Innovation Sprint, Brussels, Belgium

View Table of Contents