SIERRA: A Modular Framework for Research Automation
Pages 1905 - 1907
Abstract
Modern intelligent systems researchers form hypotheses about system behavior, and then run experiments using one or more independent variables to test their hypotheses. We present SIERRA, a novel framework structured around that idea for accelerating research developments and improving reproducibility of results. SIERRA makes it easy to quickly specify the independent variable(s) for an experiment, generate experimental inputs, automatically run the experiment, and process the results to generate deliverables such as graphs and videos. SIERRA provides reproducible automation independent of the execution environment (HPC hardware, real robots, etc.) and targeted platform (arbitrary simulator or real robots), enabling exact experiment replication (up to the limit of the execution environment and platform). It employs a deeply modular approach that allows easy customization and extension of automation for the needs of individual researchers, thereby eliminating manual experiment configuration and result processing via throw-away scripts.
References
[1]
Robin Amsters and Peter Slaets. 2020. Turtlebot 3 as a Robotics Education Platform. In Robotics in Education, Munir Merdan, Wilfried Lepuschitz, Gottfried Koppensteiner, Richard Balogh, and David Obdrvzálek (Eds.). Springer International Publishing, Cham, 170--181.
[2]
Matt Franchi. 2021. Webots.HPC: A Parallel Robotics Simulation Pipeline for Autonomous Vehicles on High Performance Computing. showeprint[arXiv]2108.00485
[3]
John Harwell and Maria Gini. 2019. Swarm Engineering Through Quantitative Measurement of Swarm Robotic Principles in a 10,000 Robot Swarm. In Proc. Twenty-Eighth Int'l Joint Conference on Artificial Intelligence, IJCAI-19. 336--342.
[4]
John Harwell and Maria Gini. 2021. Improved Swarm Engineering: Aligning Intuition and Analysis. IEEE Transactions on Robotics (2021).
[5]
John Harwell, London Lowmanstone, and Maria Gini. 2020. Demystifying Emergent Intelligence And Its Effect On Performance In Large Robot Swarms. In Proc. Autonomous Agents and Multi-agent Systems (AAMAS). 474--482.
[6]
John Harwell, Angel Sylvester, and Maria Gini. 2021. Characterizing The Limits of Linear Modeling of Non-Linear Swarm Behaviors. arxiv: 2110.12307 [cs.RO]
[7]
Nathan Koenig and Andrew Howard. 2004. Design and use paradigms for gazebo, an open-source multi-robot simulator. In 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vol. 3. IEEE, 2149--2154.
[8]
O. Michel. 2004. Webots: Professional Mobile Robot Simulation. Journal of Advanced Robotics Systems, Vol. 1, 1 (2004), 39--42.
[9]
Daniel Pickem, Paul Glotfelter, Li Wang, Mark Mote, Aaron Ames, Eric Feron, and Magnus Egerstedt. 2016. The Robotarium: A remotely accessible swarm robotics research testbed. arxiv: 1609.04730 [cs.RO]
[10]
Carlo Pinciroli et al. 2012. ARGoS: a Modular, Parallel, Multi-Engine Simulator for Multi-Robot Systems. Swarm Intelligence, Vol. 6 (12 2012), 271--295.
[11]
Morgan Quigley, Ken Conley, Brian Gerkey, Josh Faust, Tully Foote, Jeremy Leibs, Rob Wheeler, and Andrew Ng. 2009. ROS: an open-source Robot Operating System. ICRA Workshop on Open Source Software.
[12]
Andy B. Yoo, Morris A. Jette, and Mark Grondona. 2003. SLURM: Simple Linux Utility for Resource Management. In Job Scheduling Strategies for Parallel Processing, Dror Feitelson, Larry Rudolph, and Uwe Schwiegelshohn (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 44--60.
Index Terms
- SIERRA: A Modular Framework for Research Automation
Recommendations
A Database for Reproducible Computational Research
WSCAD-SSC '12: Proceedings of the 2012 13th Symposium on Computing SystemsThe ability to reproduce the experiments of a scientific research is one of the fundamental principles of the scientific method, as failing to reproduce it and obtain results equal (or very similar) to the original might imply that the later were wrong. ...
High-Throughput Neuroanatomy and Trigger-Action Programming: A Case Study in Research Automation
AI-Science'18: Proceedings of the 1st International Workshop on Autonomous Infrastructure for ScienceExponential increases in data volumes and velocities are overwhelming finite human capabilities. Continued progress in science and engineering demands that we automate a broad spectrum of currently manual research data manipulation tasks, from data ...
I Am a Scientist, Not a Philosopher!
To evaluate anything we can't prove using pure mathematics or logical syllogism, we must test hypotheses by performing controlled experiments to generate measurable, empirical data. But today's computer security researchers often claim "proof" without ...
Comments
Information & Contributors
Information
Published In
May 2022
1990 pages
ISBN:9781450392136
- General Chairs:
- Catherine Pelachaud,
- Matthew E. Taylor,
- Program Chairs:
- Piotr Faliszewski,
- Viviana Mascardi
Sponsors
Publisher
International Foundation for Autonomous Agents and Multiagent Systems
Richland, SC
Publication History
Published: 09 May 2022
Check for updates
Author Tags
Qualifiers
- Demonstration
Conference
AAMAS ' 22
Sponsor:
AAMAS ' 22: International Conference on Autonomous Agents and Multi-Agent Systems
May 9 - 13, 2022
Virtual Event, New Zealand
Acceptance Rates
Overall Acceptance Rate 1,155 of 5,036 submissions, 23%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 19Total Downloads
- Downloads (Last 12 months)1
- Downloads (Last 6 weeks)0
Reflects downloads up to 21 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in