research-article

Scenario space: characterizing coverage, quality, and failure of steering algorithms

Authors:

Petros FaloutsosAuthors Info & Claims

SCA '11: Proceedings of the 2011 ACM SIGGRAPH/Eurographics Symposium on Computer Animation

Pages 53 - 62

https://doi.org/10.1145/2019406.2019414

Published: 05 August 2011 Publication History

Get Access

Abstract

Navigation and steering in complex dynamically changing environments is a challenging research problem, and a fundamental aspect of immersive virtual worlds. While there exist a wide variety of approaches for navigation and steering, there is no definitive solution for evaluating and analyzing steering algorithms. Evaluating a steering algorithm involves two major challenges: (a) characterizing and generating the space of possible scenarios that the algorithm must solve, and (b) defining evaluation criteria (metrics) and applying them to the solution. In this paper, we address both of these challenges. First, we characterize and analyze the complete space of steering scenarios that an agent may encounter in dynamic situations. Then, we propose the representative scenario space and a sampling method that can generate subsets of the representative space with good statistical properties. We also propose a new set of metrics and a statistically robust approach to determining the coverage and the quality of a steering algorithm in this space. We demonstrate the effectiveness of our approach on three state of the art techniques. Our results show that these methods can only solve 60% of the scenarios in the representative scenario space.

Supplementary Material

JPG File (p53-kapadia.jpg)

Download
20.34 KB

MOV File (p53-kapadia.mov)

Download
24.60 MB

References

[1]

{BH97} Brogan D. C., Hodgins J. K.: Group behaviors for systems with significant dynamics. Auton. Robots 4, 1 (1997), 137--153.

Digital Library

Google Scholar

[2]

{BMOB03} Braun A., Musse S. R., Oliveira L. P. L. d., Bodmann B. E. J.: Modeling individual behaviors in crowd simulation. In CASA '03: Proceedings of the 16th International Conference on Computer Animation and Social Agents (CASA 2003) (Washington, DC, USA, 2003), IEEE Computer Society, p. 143.

Digital Library

Google Scholar

[3]

{Feu00} Feurtey F.: Simulating the Collision Avoidance Behavior of Pedestrians. Master's thesis, The University of Tokyo, School of Engineering, 2000.

Google Scholar

[4]

{GCC*10} Guy S. J., Chhugani J., Curtis S., Dubey P., Lin M., Manocha D.: Pledestrians: a least-effort approach to crowd simulation. In Proceedings of the 2010 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (Aire-la-Ville, Switzerland, Switzerland, 2010), SCA '10, Eurographics Association, pp. 119--128.

Digital Library

Google Scholar

[5]

{HBJW05} Helbing D., Buzna L., Johansson A., Werner T.: Self-organized pedestrian crowd dynamics: Experiments, simulations, and design solutions. Transportation Science 39, 1 (2005), 1--24.

Digital Library

Google Scholar

[6]

{Hen71} Henderson L. F.: The statistics of crowd fluids. Nature 229, 5284 (February 1971), 381--383.

Google Scholar

[7]

{HFV00} Helbing D., Farkas I., Vicsek T.: Simulating dynamical features of escape panic. NATURE 407 (2000), 487.

Crossref

Google Scholar

[8]

{KSA*09} Kapadia M., Singh S., Allen B., Reinman G., Faloutsos P.: Steerbug: an interactive framework for specifying and detecting steering behaviors. In SCA '09: Proceedings of the 2009 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (2009), ACM, pp. 209--216.

Digital Library

Google Scholar

[9]

{KSHF09} Kapadia M., Singh S., Hewlett W., Faloutsos P.: Egocentric affordance fields in pedestrian steering. In I3D '09: Proceedings of the 2009 symposium on Interactive 3D graphics and games (2009), ACM, pp. 215--223.

Digital Library

Google Scholar

[10]

{LCHL07} Lee K. H., Choi M. G., Hong Q., Lee J.: Group behavior from video: a data-driven approach to crowd simulation. In SCA '07: Proceedings of the 2007 ACM SIGGRAPH/Eurographics symposium on Computer animation (Aire-la-Ville, Switzerland, Switzerland, 2007), Eurographics Association, pp. 109--118.

Digital Library

Google Scholar

[11]

{LCL07} Lerner A., Chrysanthou Y., Lischinski D.: Crowds by example. Computer Graphics Forum 26, 3 (September 2007), 655--664.

Crossref

Google Scholar

[12]

{LCSCO10} Lerner A., Chrysanthou Y., Shamir A., Cohen-Or D.: Context-dependent crowd evaluation. Comput. Graph. Forum 29, 7 (2010), 2197--2206.

Crossref

Google Scholar

[13]

{LD04} Lamarche F., Donikian S.: Crowd of virtual humans: a new approach for real time navigation in complex and structured environments. In Computer Graphics Forum 23. (2004).

Google Scholar

[14]

{LMM03} Loscos C., Marchal D., Meyer A.: Intuitive crowd behaviour in dense urban environments using local laws. In TPCG '03: Proceedings of the Theory and Practice of Computer Graphics 2003 (Washington, DC, USA, 2003), IEEE Computer Society, p. 122.

Digital Library

Google Scholar

[15]

{Lov94} Lovas G.: Modeling and simulation of pedestrian traffic flow. In Transportation Research Record (1994), pp. 429--443.

Google Scholar

[16]

{LS02} Llopis N., Sharp B.: By the Books: Solid Software Engineering for Games, 2002. Games Developers Conference, Round Table.

Google Scholar

[17]

{McF06} McFadden C.: Improving the QA Process, 2006. Games Developers Conference, Round Table.

Google Scholar

[18]

{MRHA98} Milazzo J., Rouphail N., Hummer J., Allen D.: The effect of pedestrians on the capacity of signalized intersections. In Transportation Research Record (1998), pp. 37--46.

Crossref

Google Scholar

[19]

{PAB07} Pelechano N., Allbeck J. M., Badler N. I.: Controlling individual agents in high-density crowd simulation. In SCA '07: Proceedings of the 2007 ACM SIGGRAPH/Eurographics symposium on Computer animation (Aire-la-Ville, Switzerland, Switzerland, 2007), Eurographics Association, pp. 99--108.

Digital Library

Google Scholar

[20]

{PAB08} Pelechano N., Allbeck J. M., Badler N. I.: Virtual Crowds: Methods, Simulation, and Control. Synthesis Lectures on Computer Graphics and Animation.Morgan & Claypool Publishers, 2008.

Digital Library

Google Scholar

[21]

{PPD07} Paris S., Pettré J., Donikian S.: Pedestrian reactive navigation for crowd simulation: a predictive approach. In EUROGRAPHICS 2007 (2007), vol. 26, pp. 665--674.

Crossref

Google Scholar

[22]

{PSAB08} Pelechano N., Stocker C., Allbeck J., Badler N.: Being a part of the crowd: towards validating vr crowds using presence. In Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems - Volume 1 (2008), AAMAS '08, pp. 136--142.

Digital Library

Google Scholar

[23]

{Rey87} Reynolds C. W.: Flocks, herds and schools: A distributed behavioral model. In SIGGRAPH '87: Proceedings of the 14th annual conference on Computer graphics and interactive techniques (1987), ACM, pp. 25--34.

Digital Library

Google Scholar

[24]

{Rey99} Reynolds C.: Steering behaviors for autonomous characters, 1999.

Google Scholar

[25]

{RMH05} Rudomín I., Millán E., Hernández B.: Fragment shaders for agent animation using finite state machines. Simulation Modelling Practice and Theory 13, 8 (2005), 741--751.

Crossref

Google Scholar

[26]

{RP07} Reitsma P. S. A., Pollard N. S.: Evaluating motion graphs for character animation. ACM Trans. Graph. 26 (October 2007).

Digital Library

Google Scholar

[27]

{SGA*07} Sud A., Gayle R., Andersen E., Guy S., Lin M., Manocha D.: Real-time navigation of independent agents using adaptive roadmaps. In VRST '07: Proceedings of the 2007 ACM symposium on Virtual reality software and technology (2007), ACM, pp. 99--106.

Digital Library

Google Scholar

[28]

{SKFR09} Singh S., Kapadia M., Faloutsos P., Reinman G.: An open framework for developing, evaluating, and sharing steering algorithms. In Proceedings of the 2nd International Workshop on Motion in Games (Berlin, Heidelberg, 2009), MIG '09, Springer-Verlag, pp. 158--169.

Digital Library

Google Scholar

[29]

{SKHF11} Singh S., Kapadia M., Hewlett W., Faloutsos P.: A modular framework for adaptive agent-based steering. In Proceedings of the 2011 symposium on Interactive 3D graphics and games (2011), I3D '11, ACM.

Digital Library

Google Scholar

[30]

{SKN*09} Singh S., Kapadia M., Naik M., Reinman G., Faloutsos P.: SteerBench: A Steering Framework for Evaluating Steering Behaviors. Computer Animation and Virtual Worlds (2009). http://dx.doi.org/10.1002/cav.277.

Digital Library

Google Scholar

[31]

{SKRF11} Singh S., Kapadia M., Reinman G., Faloutsos P.: Footstep navigation for dynamic crowds. In Symposium on Interactive 3D Graphics and Games (New York, NY, USA, 2011), I3D '11, ACM, pp. 203--203.

Digital Library

Google Scholar

[32]

{ST05} Shao W., Terzopoulos D.: Autonomous pedestrians. In SCA '05: Proceedings of the 2005 ACM SIGGRAPH/Eurographics symposium on Computer animation (2005), ACM, pp. 19--28.

Digital Library

Google Scholar

[33]

{TCP06} Treuille A., Cooper S., Popović Z.: Continuum crowds. ACM Trans. Graph. 25, 3 (2006), 1160--1168.

Digital Library

Google Scholar

[34]

{vdBLM08} van den Berg J., Lin M. C., Manocha D.: Reciprocal velocity obstacles for real-time multi-agent navigation. In Proceedings of ICRA (2008), IEEE, pp. 1928--1935.

Crossref

Google Scholar

[35]

{vdBPS*08} van den Berg J., Patil S., Sewall J., Manocha D., Lin M.: Interactive navigation of multiple agents in crowded environments. In SI3D '08: Proceedings of the 2008 symposium on Interactive 3D graphics and games (2008), ACM, pp. 139--147.

Digital Library

Google Scholar

Cited By

View all

Shen CMao SXu BWang ZZhang XYan SDing H(2024)Spiral complete coverage path planning based on conformal slit mapping in multi-connected domainsThe International Journal of Robotics Research10.1177/02783649241251385Online publication date: 10-May-2024
https://doi.org/10.1177/02783649241251385
Charalambous PPettre JVassiliades VChrysanthou YPelechano N(2023)GREIL-Crowds: Crowd Simulation with Deep Reinforcement Learning and ExamplesACM Transactions on Graphics10.1145/359245942:4(1-15)Online publication date: 26-Jul-2023
https://dl.acm.org/doi/10.1145/3592459
Hu KHaworth BBerseth GPavlovic VFaloutsos PKapadia M(2023)Heterogeneous Crowd Simulation Using Parametric Reinforcement LearningIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2021.313903129:4(2036-2052)Online publication date: 1-Apr-2023
https://doi.org/10.1109/TVCG.2021.3139031
Show More Cited By

Index Terms

Scenario space: characterizing coverage, quality, and failure of steering algorithms
1. Computing methodologies

Recommendations

Scenario Networks for Software Specification and Scenario Management
Classifying Scenarios with Differential Scenario
Proceedings of the 2008 conference on Knowledge-Based Software Engineering: Proceedings of the Eighth Joint Conference on Knowledge-Based Software Engineering

A visually classification method of scenarios using differential information between normal scenarios is presented. Behaviors of normal scenarios of similar purposes belonging to the same problem domain resemble each other. We derive the differential ...
Fitness Prediction Techniques for Scenario-Based Design Space Exploration

Modern embedded systems are becoming increasingly multifunctional. The dynamism in multifunctional embedded systems manifests itself with more dynamic applications and the presence of multiple applications executing on a single embedded system. This ...

Reviews

Reviewer: George Popescu

"Navigation and steering in complex dynamically changing environments is a challenging research problem, and a fundamental aspect of immersive virtual worlds." In this paper, the authors consider possible scenarios that steering algorithms handle in order to define overall evaluation metrics and understand key concepts such as coverage, quality, and failure. The paper identifies important requirements for evaluating steering techniques while referring to the limitations of previous studies. It then includes some metrics that can be normalized to become scenario-independent, as well as computational examples. Scenarios are generated randomly by considering obstacles and cells in a grid. Then, their number and the agent's goal are considered. The essential evaluation metrics are scenario completion, path length, and total time. The comparison results focus on the previously defined metrics and help deduce scenario classes in which some steering algorithms perform better than others. The essential contributions of the paper are its comparison of the performance of four algorithms-reaction, prediction, and planning; reciprocal velocity obstacles; egocentric; and reactive-and its identification of failure scenarios. Pictures and explanations nicely supplement the developed theoretical framework. Online Computing Reviews Service

Access critical reviews of Computing literature here

Become a reviewer for Computing Reviews.

Comments

Information & Contributors

Information

Published In

SCA '11: Proceedings of the 2011 ACM SIGGRAPH/Eurographics Symposium on Computer Animation

August 2011

297 pages

ISBN:9781450309233

DOI:10.1145/2019406

Conference Chairs:
Karen Liu
Georgia Institute of Technology
,
Robert Bridson
University of British Columbia
,
Editor:
Stephen N. Spencer
University of Washington
,
Program Chairs:
Adam Bargteil
University of Utah
,
Michiel van de Panne
University of British Columbia

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 August 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Conference

SCA '11

Sponsor:

SIGGRAPH
EUROGRAPHICS

SCA '11: The ACM SIGGRAPH / Eurographics Symposium on Computer Animation 2011

August 5 - 7, 2011

British Columbia, Vancouver, Canada

Acceptance Rates

Overall Acceptance Rate 183 of 487 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

43
Total Citations
View Citations
395
Total Downloads

Downloads (Last 12 months)16
Downloads (Last 6 weeks)0

Reflects downloads up to 12 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Shen CMao SXu BWang ZZhang XYan SDing H(2024)Spiral complete coverage path planning based on conformal slit mapping in multi-connected domainsThe International Journal of Robotics Research10.1177/02783649241251385Online publication date: 10-May-2024
https://doi.org/10.1177/02783649241251385
Charalambous PPettre JVassiliades VChrysanthou YPelechano N(2023)GREIL-Crowds: Crowd Simulation with Deep Reinforcement Learning and ExamplesACM Transactions on Graphics10.1145/359245942:4(1-15)Online publication date: 26-Jul-2023
https://dl.acm.org/doi/10.1145/3592459
Hu KHaworth BBerseth GPavlovic VFaloutsos PKapadia M(2023)Heterogeneous Crowd Simulation Using Parametric Reinforcement LearningIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2021.313903129:4(2036-2052)Online publication date: 1-Apr-2023
https://doi.org/10.1109/TVCG.2021.3139031
Panayiotou AKyriakou TLemonari MChrysanthou YCharalambous P(2022)CCP: Configurable Crowd ProfilesACM SIGGRAPH 2022 Conference Proceedings10.1145/3528233.3530712(1-10)Online publication date: 27-Jul-2022
https://dl.acm.org/doi/10.1145/3528233.3530712
Cabrero‐Daniel BMarques RHoyet LPettré JBlat J(2022)Dynamic Combination of Crowd Steering Policies Based on ContextComputer Graphics Forum10.1111/cgf.1446941:2(209-219)Online publication date: 24-May-2022
https://doi.org/10.1111/cgf.14469
Daniel BMarques RHoyet LPettré JBlat J(2021)A Perceptually-Validated Metric for Crowd Trajectory Quality EvaluationProceedings of the ACM on Computer Graphics and Interactive Techniques10.1145/34801364:3(1-18)Online publication date: 27-Sep-2021
https://dl.acm.org/doi/10.1145/3480136
van Toll WPettré J(2021)Algorithms for Microscopic Crowd Simulation: Advancements in the 2010sComputer Graphics Forum10.1111/cgf.14266440:2(731-754)Online publication date: 4-Jun-2021
https://doi.org/10.1111/cgf.142664
Kremer MHaworth BKapadia MFaloutsos P(2020)Watch Out! Modelling Pedestrians with Egocentric DistractionsProceedings of the 13th ACM SIGGRAPH Conference on Motion, Interaction and Games10.1145/3424636.3426910(1-10)Online publication date: 16-Oct-2020
https://dl.acm.org/doi/10.1145/3424636.3426910
Haworth BBerseth GMoon SFaloutsos PKapadia M(2020)Deep Integration of Physical Humanoid Control and Crowd NavigationProceedings of the 13th ACM SIGGRAPH Conference on Motion, Interaction and Games10.1145/3424636.3426894(1-10)Online publication date: 16-Oct-2020
https://dl.acm.org/doi/10.1145/3424636.3426894
Chao QDeng ZXiao YHe DMiao QJin X(2020)Dictionary-based Fidelity Measure for Virtual TrafficIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2018.287369526:3(1490-1501)Online publication date: 1-Mar-2020
https://doi.org/10.1109/TVCG.2018.2873695
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Scenario Networks for Software Specification and Scenario Management

Classifying Scenarios with Differential Scenario

Fitness Prediction Techniques for Scenario-Based Design Space Exploration

Reviews

Access critical reviews of Computing literature here