Mark Harman
Website: https://research.facebook.com/people/harman-mark/
Inna Dvortsova
Website: https://research.facebook.com/people/dvortsova-inna/
Automated Testing as Production Simulation: Research Opportunities and Challenges
Abstract: Testing has always been about simulation, although often implicit. For example, when a software tester mocks out the behaviour of a code unit, they are effectively stimulating its behaviour. When a test tool creates a test user account in order to log in and run a test it is essentially simulating a potential user behaviour. Traditionally, software engineers, and the research that supports them, has tended to apply greater focus to problems and challenges associated with test mock techniques and applications. However, as software is increasingly deployed to user communities that interact through the software platform, effective techniques for simulating user behaviour are becoming increasingly important. In this talk we outline engineering trade-offs between test (simulation) realism, which we seek to maximise, and various testing costs we seek to minimise. We present an overview of and results from the deployment, at Meta, of two simulation-inspired testing technologies: Sapienz and Web Enabled Simulation, both of which aim to realistically simulate user behaviour, on client side and server side respectively. We show how a simulation-based testing approach opens new perspectives on new and existing automated software engineering challenges and opportunities.
Mark Harman is a full-time Software Engineer at Meta Platforms, working in the Simulation-Based Testing team. The team has developed and deployed both the Sapienz and WW platforms for client- and server- side testing. Simulation-based testing is helping to tackle challenging technical problems in software reliability, performance, safety and privacy. These simulation-based testing technologies have been deployed to test systems of over 100 million lines of lines of code, daily relied upon by over 2.88 billion people for communications, business, social media and community building. Sapienz grew out of Majicke (a start up Mark co-founded) that was acquired by Facebook (now Meta Platforms) in 2017. Prior to working at Meta Platforms, Mark was head of Software Engineering and director of its CREST centre at UCL, where he remains a part time professor. In his more purely scientific work, he co-founded the field Search Based Software Engineering (SBSE) in 2001, now the subject of active research in over 40 countries worldwide. He received the IEEE Harlan Mills Award and the ACM Outstanding Research Award in 2019 for his work, and was awarded a fellowship of the Royal Academy of Engineering in 2020.
Inna Dvortsova graduated from Taras Shevchenko National University of Kyiv with a masters in mathematics in 2013. She worked as a practicing software engineer at several companies in the IT sector before moving to take up a post as tech lead at Barclays in 2015, where she led data analysis, algorithms and deployment of stress testing. In 2018 she moved to Spotify, where she developed and deployed systems that supported and optimised the company’s revenue streams. More recently in 2020 she started work at Facebook where she has been a lead engineer on the technical development, deployment and optimisation of Facebook’s WW Web Enabled Simulation system. In 2022 she was a founder member of the wider Meta Simulation-Based Testing team, which is developing the WW and Sapienz technologies for automated testing. She and her co-authors have published results from this work at the 8th Genetic Improvement Workshop (GI 2020) and the 43rd International Conference on Software Engineering (ICSE 2021). She has keynote papers at the 9th International workshop on Realising Artificial Intelligence synergies in Software Engineering (RAISE 2021), and the 15th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM 2021). In 2021 she gave keynotes at both academic and industry conferences: the 25th International Conference on Evaluation and Assessment in Software Engineering (EASE 2021) and the UK National Software Testing Conference.
Lionel C. Briand
University of Ottawa, Canada
University of Luxembourg, Luxembourg
Website: https://www.lbriand.info
Autonomous Systems: How to address the Dilemma between Autonomy and Safety
Abstract: The need for more autonomy in many systems (e.g., automotive, aerospace) leads to increasingly complex systems that are becoming harder to verify and certify. This problem is compounded by the rising use of AI to enable such autonomy in many critical functions. Such situation has a significant impact on software development practice and makes it particularly difficult to assess risks and meet industry standards. This talk will focus on automated techniques, relying on various AI disciplines, to enable the testing and risk analysis of such systems, in ways that are practical and scalable. The talk will report and reflect on various research projects carried out in collaboration with the automotive and satellite industries.
Bio: Lionel C. Briand is professor of software engineering and has shared appointments between (1) The University of Ottawa, Canada and (2) The SnT centre for Security, Reliability, and Trust, University of Luxembourg. In collaboration with colleagues, over 25 years, he has run many collaborative research projects with companies in the automotive, satellite, aeropsace, energy, financial, and legal domains. Lionel was elevated to the grades of IEEE Fellow and ACM Fellow for his work on software testing and verification. He was granted the IEEE Computer Society Harlan Mills award, the ACM SIGSOFT outstanding research award, and the IEEE Reliability Society engineer-of-the-year award, respectively in 2012, 2022, and 2013. He received an ERC Advanced grant in 2016 — on the topic of modelling and testing cyber-physical systems — which is the most prestigious individual research award in the European Union. He currently holds a Canada Research Chair (Tier 1) on "Intelligent Software Dependability and Compliance". His research interests include: software testing and verification, applications of AI in software engineering, model-driven software development, requirements engineering, and empirical software engineering.
From Automating Software Development to Empowering Software Developers
Abstract: Machines today can write software, compose music, create art, predict events, and listen and learn from humans. Notably, automation also plays an essential role in high performing software development teams by automating tasks and improving developer productivity. But automation can’t (yet) replace human imagination and the intelligence that arises when multiple great minds work together to solve the complex problems that are inherent in software and systems design. In this talk, we will review how automation in modern software development has evolved and the many benefits it has brought. We will then explore how a deeper understanding of the developer experience points to untapped possibilities for innovating automation for software engineering, focusing on how they can: support developers to manage the cognitive complexity of today’s systems, ease and enhance collaboration by speeding up feedback loops, and help developers to get in and stay in a state of flow when developing. We will conclude by discussing how we can measure the impact of new innovations on the developer experience, and how doing so will drive actionable change and empower developers to do their best work joyfully.
Bio: Dr. Margaret-Anne Storey is a Professor of Computer Science. She holds a Canada Research Chair in Human and Social Aspects of Software Engineering and is a member of the Royal Society of Canada’s College of New Scholars, Artists and Scientists. Together with her students and collaborators, she seeks to understand how software tools, communication media, data visualizations, and social theories can be leveraged to improve how software engineers and knowledge workers explore, understand, analyze and share complex information and knowledge. She has published widely on these topics and collaborates extensively with high-tech companies and non-profit organizations to ensure real-world applicability of her research contributions and tools. Dr. Storey currently consults with Microsoft and getDX to understand and improve developer productivity and developer experience.