research-article

Open access

APIstic: A Large Collection of OpenAPI Metrics

Authors:

Souhaila Serbout,

Cesare PautassoAuthors Info & Claims

MSR '24: Proceedings of the 21st International Conference on Mining Software Repositories

Pages 265 - 277

https://doi.org/10.1145/3643991.3644932

Published: 02 July 2024 Publication History

Abstract

In the rapidly evolving landscape of web services, the significance of efficiently designed and well-documented APIs is paramount. In this paper, we present APIstic an API analytics dataset and exploration tool to navigate and segment APIs based on an extensive set of pre-computed metrics extracted from OpenAPI specifications, sourced from GitHub, SwaggerHub, BigQuery and APIs.guru. These pre-computed metrics are categorized into structure, data model, natural language description, and security metrics. The extensive dataset of varied API metrics provides crucial insights into API design and documentation for both researchers and practitioners. Researchers can use APIstic as an empirical resource to extract refined samples, analyze API design trends, best practices, smells, and patterns. For API designers, it serves as a benchmarking tool to assess, compare, and improve API structures, data models, and documentation using metrics to select points of references among 1,275,568 valid OpenAPI specifications. The paper discusses potential use cases of the collected data and presents a descriptive analysis of selected API analytics metrics.

The dataset available at: http://openapi.inf.usi.ch/

References

[1]

Public APIs --- A directory of free and public apis. https://publicapis.io/.

[2]

Rapid API Hub. https://rapidapi.com/hub.

[3]

APIs.Guru. https://github.com/APIs-guru/openAPI-directory https://APIs.guru/.

[4]

Contextual Content Discovery: You've forgotten about the API endpoints. https://blog.assetnote.io/2021/04/05/contextual-content-discovery/.

[5]

Google BigQuery API. https://cloud.google.com/bigquery/docs/reference/rest/.

[6]

GitHub API. https://docs.github.com/en/rest, .

[7]

GitHub Archive. http://www.gharchive.org/, .

[8]

OpenAPI Initiative. https://www.openAPIs.org/.

[9]

RAML. https://raml.org/. Accessed: 2021-06-01.

[10]

Semantic Versioning. https://semver.org/.

[11]

SwaggerHub. https://swagger.io/tools/swaggerhub/, .

[12]

SwaggerHub Registry API. https://app.swaggerhub.com/APIs-docs/swagger-hub/registry-API/, .

[13]

Swagger Validator. https://github.com/APIDevTools/swagger-parser.

[14]

Emad Aghajani, Csaba Nagy, Olga Lucero Vega-Márquez, Mario Linares-Vásquez, Laura Moreno, Gabriele Bavota, and Michele Lanza. Software documentation issues unveiled. In 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE), pages 1199--1210. IEEE, 2019.

Digital Library

[15]

Tarfah Alrashed, Jumana Almahmoud, Amy X Zhang, and David R Karger. Scrapir: making web data apis accessible to end users. In Proceedings of the 2020 CHI conference on human factors in computing systems, pages 1--12, 2020.

Digital Library

[16]

Dionysis Athanasopoulos, Apostolos V Zarras, George Miskos, Valerie Issarny, and Panos Vassiliadis. Cohesion-driven decomposition of service interfaces without access to source code. IEEE Transactions on Services Computing, 8(4): 550--562, 2014.

[17]

David Bermbach and Erik Wittern. Benchmarking web api quality-revisited. Journal of Web Engineering, 19(5-6):603--646, 2020.

[18]

Matthias Biehl. Webhooks-Events for RESTful APIs, volume 4. API-University Press, 2017.

[19]

Justus Bogner, Stefan Wagner, and Alfred Zimmermann. Collecting service-based maintainability metrics from restful api descriptions: static analysis and threshold derivation. In European Conference on Software Architecture, pages 215--227. Springer, 2020.

[20]

Justus Bogner, Sebastian Kotstein, and Timo Pfaff. Do restful api design rules have an impact on the understandability of web apis? Empirical Software Engineering, 28(6):1--35, 2023.

Digital Library

[21]

Pierre Bourhis, Juan L Reutter, Fernando Suárez, and Domagoj Vrgoč. Json: data model, query languages and schema specification. In Proceedings of the 36th ACM SIGMOD-SIGACT-SIGAI symposium on principles of database systems, pages 123--135, 2017.

Digital Library

[22]

Engin Bozdag, Ali Mesbah, and Arie van Deursen. A comparison of push and pull techniques for ajax. In Proceedings of the 2007 9th IEEE International Workshop on Web Site Evolution, WSE '07, page 15--22, 2007. ISBN 9781424414505.

Digital Library

[23]

Steven Bucaille, Javier Luis Cánovas Izquierdo, Hamza Ed-Douibi, and Jordi Cabot. An openapi-based testing framework to monitor non-functional properties of rest apis. In International Conference on Web Engineering, pages 533--537. Springer, 2020.

Digital Library

[24]

Carmen Cheh and Binbin Chen. Analyzing openapi specifications for security design issues. In 2021 IEEE Secure Development Conference (SecDev), pages 15--22. IEEE, 2021.

[25]

Hsiao-Jung Chen, Shang-Pin Ma, and Hsueh-Cheng Lu. Collaborative security annotation and online testing for web apis. In 2021 IEEE International Conference on e-Business Engineering (ICEBE), pages 9--15. IEEE, 2021.

[26]

Meri Coleman and Ta Lin Liau. A computer readability formula designed for machine scoring. Journal of Applied Psychology, 60(2):283, 1975.

[27]

Leonardo da Rocha Araujo, Guillermo Rodríguez, Santiago Vidal, Claudia Marcos, and Rodrigo Pereira dos Santos. Empirical analysis on openapi topic exploration and discovery to support the developer community. Computing and Informatics, 40(6):1345--1369, 2021.

[28]

Ozren Dabic, Emad Aghajani, and Gabriele Bavota. Sampling projects in github for msr studies. In 2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR), pages 560--564. IEEE, 2021.

[29]

Alan De Renzis, Martin Garriga, Andres Flores, Alejandra Cechich, Cristian Mateos, and Alejandro Zunino. A domain independent readability metric for web service descriptions. Computer Standards & Interfaces, 50:124--141, 2017.

Digital Library

[30]

Hamza Ed-Douibi, Javier Luis Cánovas Izquierdo, and Jordi Cabot. Automatic generation of test cases for rest apis: A specification-based approach. In 2018 IEEE 22nd international enterprise distributed object computing conference (EDOC), pages 181--190. IEEE, 2018.

[31]

Tiago Espinha, Andy Zaidman, and Hans-Gerhard Gross. Web api growing pains: Stories from client developers and their code. In 2014 Software Evolution Week - IEEE Conference on Software Maintenance, Reengineering, and Reverse Engineering (CSMR-WCRE), pages 84--93, 2014.

[32]

Sérgio Fernandes and Jorge Bernardino. What is bigquery? In Proceedings of the 19th International Database Engineering & Applications Symposium, pages 202--203, 2015.

Digital Library

[33]

Pascal Gadient, Mohammad Ghafari, Marc-Andrea Tarnutzer, and Oscar Nierstrasz. Web apis in android through the lens of security. In 2020 IEEE 27th international conference on software analysis, evolution and reengineering (SANER), pages 13--22. IEEE, 2020.

[34]

Patric Genfer and Uwe Zdun. Avoiding excessive data exposure through microservice apis. In European Conference on Software Architecture, pages 3--18. Springer, 2022.

Digital Library

[35]

Marc J. Hadley. Web application description language (wadl). Technical report, USA, 2006.

[36]

Florian Haupt, Frank Leymann, Anton Scherer, and Karolina Vukojevic-Haupt. A framework for the structural analysis of rest apis. In Proc. IEEE International Conference on Software Architecture (ICSA), pages 55--58, 2017.

[37]

J. Higginbotham. Principles of Web API Design: Delivering Value with APIs and Microservices. Addison-Wesley Signature Series. Pearson Education (US), 2021.

[38]

Sergio Inzunza, Reyes Juárez-Ramírez, and Samantha Jiménez. Api documentation: A conceptual evaluation model. In Proc. of World Conference on Information Systems and Technologies (WorkdCIST'18): Trends and Advances in Information Systems and Technologies, pages 229--239. Springer, 2018.

[39]

Stefan Karlsson, Adnan Čaušević, and Daniel Sundmark. Quickrest: Property-based test generation of openapi-described restful apis. In 2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST), pages 131--141. IEEE, 2020.

[40]

Rick Kazman, Roberto Tonelli, and Cesare Pautasso. An empirical basis for software architecture research. In Software Architecture: Research Roadmaps from the Community, pages 87--100. Springer, 2023.

[41]

Myeongsoo Kim, Davide Corradini, Saurabh Sinha, Alessandro Orso, Michele Pasqua, Rachel Tzoref-Brill, and Mariano Ceccato. Enhancing rest api testing with nlp techniques. In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis, pages 1232--1243, 2023.

Digital Library

[42]

Myeongsoo Kim, Saurabh Sinha, and Alessandro Orso. Adaptive rest api testing with reinforcement learning. In 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 446--458. IEEE, 2023.

Digital Library

[43]

Rediana Koçi, Xavier Franch, Petar Jovanovic, and Alberto Abelló. A data-driven approach to measure the usability of web apis. In 2020 46th Euromicro Conference on Software Engineering and Advanced Applications (SEAA), pages 64--71. IEEE, 2020.

[44]

Rediana Koçi, Xavier Franch, Petar Jovanovic, and Alberto Abelló. Web api evolution patterns: A usage-driven approach. Journal of Systems and Software, 198:111609, 2023.

Digital Library

[45]

Valliappa Lakshmanan and Jordan Tigani. Google Bigquery: the definitive guide: data warehousing, analytics, and machine learning at scale. O'Reilly Media, 2019.

[46]

Arnaud Lauret. The design of web APIs. Simon and Schuster, 2019.

[47]

Fabio Di Lauro, Souhaila Serbout, and Cesare Pautasso. A large-scale empirical assessment of web api size evolution. Journal of Web Engineering, 21:1937--1980, November 2022.

[48]

Daniel Lübke, Olaf Zimmermann, Cesare Pautasso, Uwe Zdun, and Mirko Stocker. Interface evolution patterns: Balancing compatibility and extensibility across service life cycles. In Proceedings of the 24th European Conference on Pattern Languages of Programs (EuroPLoP), pages 1--24, 2019.

Digital Library

[49]

Shang-Pin Ma, Ming-Jen Hsu, Hsiao-Jung Chen, and Yu-Sheng Su. Api prober-a tool for analyzing web api features and clustering web apis. ICEBE 2019: Advances in E-Business Engineering for Ubiquitous Computing, 2019.

[50]

Alberto Martin-Lopez, Sergio Segura, and Antonio Ruiz-Cortés. Inter-parameter dependencies in real-world web apis: The idea dataset. In Research and Evidence in Software Engineering, pages 101--106. Auerbach Publications, 2021.

[51]

Mehdi Medjaoui, Erik Wilde, Ronnie Mitra, and Mike Amundsen. Continuous API management. O'Reilly, 2021.

[52]

Andy Neumann, Nuno Laranjeiro, and Jorge Bernardino. An analysis of public rest web service apis. IEEE Transactions on Services Computing, 14(4):957--970, 2018.

[53]

Chris Parnin and Christoph Treude. Measuring api documentation on the web. In Proceedings of the 2nd international workshop on Web 2.0 for software engineering, pages 25--30, 2011.

Digital Library

[54]

Mikhail Perepletchikov, Caspar Ryan, and Keith Frampton. Cohesion metrics for predicting maintainability of service-oriented software. In Seventh International Conference on Quality Software (QSIC 2007), pages 328--335. IEEE, 2007.

[55]

Julian Aron Prenner and Romain Robbes. Making the most of small software engineering datasets with modern machine learning. IEEE Transactions on Software Engineering, 48(12):5050--5067, 2021.

[56]

Yuanbo Qiu. The openness of open application programming interfaces. Information, Communication & Society, 20(11):1720--1736, 2017.

[57]

Carlos Rodríguez, Marcos Baez, Florian Daniel, Fabio Casati, Juan Carlos Trabucco, Luigi Canali, and Gianraffaele Percannella. Rest apis: A large-scale analysis of compliance with principles and best practices. In Proc. 16th International Conference on Web Engineering (ICWE), pages 21--39. Springer, 2016.

[58]

Jéssica S Santos, Leonardo G Azevedo, Elton Soares, Raphael Thiago, and Viviane T Silva. Analysis of tools for rest contract specification in swagger/openapi. In International Conference on Enterprise Information Systems. SciTePress, 2020.

[59]

Souhaila Serbout and Cesare Pautasso. An empirical study of web api versioning practices. In Proc. 23rd International Conference on Web Engineering (ICWE). Springer, 2023. ISBN 978-3-031-34443-5.

Digital Library

[60]

Souhaila Serbout, Fabio Di Lauro, and Cesare Pautasso. Web apis structures and data models analysis. In 2022 IEEE 19th International Conference on Software Architecture Companion (ICSA-C), pages 84--91. IEEE, 2022.

[61]

Souhaila Serbout, Amine El Malki, Cesare Pautasso, and Uwe Zdun. Api rate limit adoption - a pattern collection. In Proc. 28th European Conference on Pattern Languages of Programs (EuroPLoP 2023), Kloster Irsee, Germany, July 2023. ACM, ACM.

Digital Library

[62]

Edgar A Smith and RJ Senter. Automated readability index. Number AD0667273. Aerospace Medical Research Laboratories, November 1967. https://apps.dtic.mil/sti/citations/AD0667273.

[63]

Mirko Stocker and Olaf Zimmermann. Api refactoring to patterns - catalog, template and tools for remote interface evolution. In Proc. 28th European Conference on Pattern Languages of Programs (EuroPLoP), 2023.

Digital Library

[64]

Mirko Stocker, Olaf Zimmermann, Uwe Zdun, Daniel Lübke, and Cesare Pautasso. Interface quality patterns: Communicating and improving the quality of microservices apis. In Proceedings of the 23rd European conference on pattern languages of programs, pages 1--16, 2018.

Digital Library

[65]

Christoph Treude, Justin Middleton, and Thushari Atapattu. Beyond accuracy: assessing software documentation quality. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE, page 1509--1512, 2020. ISBN 9781450370431.

Digital Library

[66]

Jim Webber, Savas Parastatidis, and Ian Robinson. REST in practice: Hypermedia and systems architecture. O'Reilly, 2010.

[67]

Chamila Wijayarathna and Nalin AG Arachchilage. An empirical usability analysis of the google authentication apis. In Proceedings of the 23rd International Conference on Evaluation and Assessment in Software Engineering, pages 268--274, 2019.

Digital Library

[68]

Mohammad-Ali Yaghoub-Zadeh-Fard and Boualem Benatallah. Api2can: a dataset & service for canonical utterance generation for rest apis. BMC Research Notes, 14:1--3, 2021.

[69]

Rieko Yamamoto, Kyoko Ohashi, Masahiro Fukuyori, Kosaku Kimura, Atsuji Sekiguchi, Ryuichi Umekawa, Tadahiro Uehara, and Mikio Aoyama. A quality model and its quantitative evaluation method for web apis. In 2018 25th Asia-Pacific Software Engineering Conference (APSEC), pages 598--607. IEEE, 2018.

[70]

Uwe Zdun, Mirko Stocker, Olaf Zimmermann, Cesare Pautasso, and Daniel Lübke. Guiding architectural decision making on quality aspects of microservice apis. In Proc. 16th International Conference on Service-Oriented Computing (ICSOC 2018), volume 11236, pages 73--89, Hangzhou, Zhejiang, China, November 2018. Springer, Springer.

Digital Library

[71]

Neng Zhang, Ying Zou, Xin Xia, Qiao Huang, David Lo, and Shanping Li. Web apis: Features, issues, and expectations-a large-scale empirical study of web apis from two publicly accessible registries using stack overflow and a user survey. IEEE Transactions on Software Engineering, 49(2):498--528, 2022.

[72]

Shixiang Zhou, Heejin Jeong, and Paul A Green. How consistent are the best-known readability equations in estimating the readability of design standards? IEEE Transactions on Professional Communication, 60(1):97--111, 2017.

[73]

Xin-Yu Zhou, Wei Chen, Guo-Quan Wu, and Wei Jun. Rest api design analysis and empirical study. Journal of Software, 33(9):3271--3296, 2021.

[74]

Olaf Zimmermann, Mirko Stocker, Daniel Lubke, Uwe Zdun, and Cesare Pautasso. Patterns for API design: simplifying integration with loosely coupled message exchanges. Addison-Wesley Professional, 2022.

Cited By

Karlsson SJongeling RČaušević ASundmark D(2024)Exploring behaviours of RESTful APIs in an industrial settingSoftware Quality Journal10.1007/s11219-024-09686-032:3(1287-1324)Online publication date: 3-Jul-2024
https://doi.org/10.1007/s11219-024-09686-0
Serbout SPautasso C(2024)How Many Web APIs Evolve Following Semantic Versioning?Web Engineering10.1007/978-3-031-62362-2_25(344-359)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1007/978-3-031-62362-2_25

Recommendations

To Deprecate or to Simply Drop Operations? An Empirical Study on the Evolution of a Large OpenAPI Collection
Software Architecture
Abstract
OpenAPI is a language-agnostic standard used to describe Web APIs which supports the explicit deprecation of interface features. To assess how APIs evolve over time and observe how their developers handle the introduction of breaking changes, we ...
OpenAPItoUML: A Tool to Generate UML Models from OpenAPI Definitions
Web Engineering
Abstract
REpresentational State Transfer (REST) has become the prominent architectural style for designing Web APIs. This increasing adoption has triggered the creation of languages to formally describe REST APIs, thus facilitating and promoting their ...
Portable and accurate collection of calling-context-sensitive bytecode metrics for the Java virtual machine
PPPJ '11: Proceedings of the 9th International Conference on Principles and Practice of Programming in Java

Calling-context profiles and dynamic metrics at the bytecode level are important for profiling, workload characterization, program comprehension, and reverse engineering. Prevailing tools for collecting calling-context profiles or dynamic bytecode ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MSR '24: Proceedings of the 21st International Conference on Mining Software Repositories

April 2024

788 pages

ISBN:9798400705878

DOI:10.1145/3643991

Chair:
Diomidis Spinellis,
Program Chair:
Alberto Bacchelli,
Program Co-chair:
Eleni Constantinou

This work is licensed under a Creative Commons Attribution-NonCommercial International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 July 2024

Check for updates

Qualifiers

Research-article

Funding Sources

Swiss National Science Foundation

Conference

MSR '24

Sponsor:

SIGSOFT

MSR '24: 21st International Conference on Mining Software Repositories

April 15 - 16, 2024

Lisbon, Portugal

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
117
Total Downloads

Downloads (Last 12 months)117
Downloads (Last 6 weeks)16

Reflects downloads up to 13 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Karlsson SJongeling RČaušević ASundmark D(2024)Exploring behaviours of RESTful APIs in an industrial settingSoftware Quality Journal10.1007/s11219-024-09686-032:3(1287-1324)Online publication date: 3-Jul-2024
https://doi.org/10.1007/s11219-024-09686-0
Serbout SPautasso C(2024)How Many Web APIs Evolve Following Semantic Versioning?Web Engineering10.1007/978-3-031-62362-2_25(344-359)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1007/978-3-031-62362-2_25

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents