Quality Assurance 3.0
Quality Assurance 3.0
▪ Traditional Software Testing has transformed ▪ Non-functional and Emerging Technology ▪ 2 key segments of players exist in the QA
into Digital Quality Assurance & Engineering
testing are the high focus, high-growth areas ecosystem – competing on a nimble footing ▪ High growth (>20%), High EBITDA profitability
▪ Key contributing factors are increasing CX in NextGen QA for the NextGen QA market-share (>30%) are desired financial metrics
expectations and a leap in digital technology ▪ Long-standing relationships with US /
▪ Other high growth areas include DevOps and
Canada enterprise clients is preferred
automation within testing
▪ Higher revenue from specialized non-
▪ Quality Assurance is a highly scalable functional testing & emerging tech testing
business as the market for software testing ▪ Pure-play QA companies such as Qualitest,
cuts across industry verticals and technology ▪ Non-functional QA has become critical in TestingXperts, Quality Kiosk, A1QA, QA
horizontals today’s era of digital transformation Consultants are capturing a major chunk of
▪ Worldwide QA industry growth is led by the ‘high-value QA’ market
▪ Existence of niche players in the market with
growth in cloud, digital transformation ▪ Majority of demand comes from the end- ability to scale through acquisitions and
spends, proliferation of emerging tech users of applications on areas such as UI / ▪ 61% of IT decision makers are considering make rapid strides in NextGen tech
UX, security, performance and reliability- pure-play service providers for QA needs ▪ The unique business model of QA that could
▪ North America is the most important market
for QA with a ~$19Bn size under-load lead to high growth & profitability
▪ BFSI is the largest vertical segment … are key factors that make it a lucrative
contributing ~25% of testing spends segment for investments & acquisitions
▪ Companies like TCS, Infosys, Cognizant have
sizeable testing practices and are investing
▪ Connected ecosystems / IoT systems (30%+ in NextGen QA solutions
▪ Winners In Digital QA are able to deliver CAGR), Immersive Tech, Cloud apps (12% ▪ The testing products segment has seen high
requirements of new-age customers enabled CAGR) are the core emerging technologies
▪ Areas such as emerging technology testing & fundraising over the last 2 years, buoyed by
by cutting-edge technology automation are assuming high priority within increasing number of end use-cases for tools
that have high growth potential
these organizations
~40 The share of spend on quality assurance was 26% of total IT spend in 2019
is expected to grow to 35% by 2022 owing to growing digital transformation
initiatives like automation, IoT
2021 2030
1960s – 1970s
1990s – 2000s
6 Source: Nagarro, KPMG Software Testing Report 2022, Techtarget Report on QE 1. Agile is an iterative approach to software development that delivers value to customers faster 2. DevOps is a set of practices that combine software development and operations
Significant rise in demand for user-friendly, responsive and Rapidly evolving technology is putting pressure on Testing has evolved from Manual using waterfall approach
secure software has prompted a surge in the requirement of firms to develop quality products quickly and efficiently in the 1980s to Autonomous1 using technologies like
bug-free and error-free environments using highly effective processes such as Agile AI/ML, today
7 Sources: Software Testing to Digital Quality Assurance, 2019, NASSCOM; Industry Research, American Society for Quality; Nagarro; 1. Autonomous testing is the ability to create tests autonomously without users’ intervention
PLAN RELEASE
DEPLOY
BUILD
▪ Shift Left testing focuses on continuous testing in ▪ Shift-right testing involves doing more testing in the ▪ AI / ML are helping to prioritize test cases and also
DevOps, and involves testing at every stage of the immediate pre-release and post-release phases of helping in automating maintenance of test code
development life-cycle application lifecycle ▪ In the future, AI / ML will also help in analyzing test results
▪ The goal is to evaluate the quality of the software as a ▪ Allows developers to discover new and unexpected and creating new test cases
part of a continuous delivery process by testing early and scenarios that have remained undetected in development ▪ Overall testing lifecycle will be shortened due to the
often environments creation of automated scripts
▪ Key Outcomes: ▪ Key Outcomes: ▪ Key Outcomes:
▪ Quicker releases: Deploying multiple times ▪ Helps gauge and enhance customer experience basis ▪ Improved accuracy vis-à-vis manual testing
a day feedback received from customers ▪ Faster test creation: Intelligent test recorders help at
▪ Aids security: Keeps application safe from unexpected ▪ Allows developers to add, delete or change features creating test scripts faster
changes and attacks based on feedback ▪ Reduces costs & time-to-market
8 Sources: Cigniiti, Tech Target, BrowserStack; CI: Continuous Integration and CD: Continuous Development
Subset of QC Subset of SDLC
Validates the product against specification Ensure that processes and procedures are in place to achieve quality
End user experience not considered End user experience or feedback key
9 Source: Avendus Research. QC- Quality Control, SDLC- Software Development Lifecycle
Through Data analytics,
data science and AI/ML
Through adoption of DevOps, Low-code/
No-code platforms and low code
test automation platforms “
To stay ahead in new normal, focus is on strengthening the
quality of software products with QA and focusing on efforts
on testing in Agile and DevOps. At the same time supporting
digital transformations with quality engineering, using AI to
boost QA efficiency.
”
– a1qa
10 Sources: Software Testing to Digital Quality Assurance, 2019, NASSCOM; KPMG-NASSCOM Report on Software Testing, May 2022
▪ A new role called Software Development Engineers in Test (SDETs) has been created in the QA industry
QA resources with programming skills are critical contributors
▪ Professionals who can both write code as well as test based on requirements are called SDETs to a QA function / organization
▪ They help improve the code quality by performing strict and detailed source code reviews along with checking for the testability
of the code
Winners in Digital QA are hiring QA professionals and training
them on digital technologies as well as programming skills
“
areas such as Automation approach that is being
testing, proficiency in increasingly adopted by QA
programming languages has practices, as it helps widen the
become a must for a QA Engineer net for bug-catching
80% of the job ads you’d find if searching for
as automation testing involves
▪ It helps improve security of the
jobs in software QA or Test are asking for
thread management, exception
system, as a wider pool of testers
handling, logging, integration with
contribute to it programming skills
DevOps tools
▪ Large bounties are today given to
– Elisabeth Hendrickson of Agile Alliance
▪ Agile methodology has become a promote the open-source testing
must-have knowledge base community
▪ Functional testing is getting more commoditized, and the market is shifting towards high
end specialized automated services which have higher realization
40% of people abandon a website that takes
▪ In 2012, 75% of the testing was functional out of which 60% was manual and rest automated more than 3 seconds to load
Specialized Specialized
Services Services Average B2B company loses $260 Bn due to
poor usability
New New
offerings offerings
▪ Performance and reliability-under-load are examples of non-functional ▪ It requires innovative test solutions and automation with a high order of
parameters that make-or-break user experience (UX), which is a critical sophistication and efficiency in testing
parameter for digital applications post the pandemic
Performance AR / VR
Testing
Security AI Systems
Testing
Big Data
User Applications1
Interface
Testing
Usability Cloud-
Testing based
Applications
Distributed
Compatibility Ledger
Testing Technologies
API Testing Connected Devices
6 ▪ Removes Blinders
“ The global Compatibility Testing Service market is estimated to attain noticeable growth
over next 6-7 years, owing to the digital transformation taking place across several
services such as R&D & Testing, IT, Telecom and Internet.
”
▪ Validates Data – Crifax
2 ▪ Provides User insights
▪ Enhances UX
▪ Measures Engagement
2021 2030
“
1,862
▪ Attributes covered by security testing include: Authentication, Authorization, Confidentiality,
1,506
Availability, Integrity, Non-Repudiation, and Resilience 1,279
1,175 1,108 Cybercrime is estimated
▪ Areas covered by security testing include System software security, Network security, Server-side 1,385 1,543
to cost $10 Trillion Annually
application security, Client-side application security 1,013 1,084
882 By 2025
”
2017 2018 2019 2020 2021 Executive
Data Breaches Sensitive Records Exposed
17 Source: Allied Market Research, Globe newswire, Gartner, Deloitte, AustCyber: CSP, Javatpoint, Avendus Research .
▪
▪
UI testing helps ensure that all the components on the UI
function as desired
”
carried out using the keyboard, mouse, and other input 4% ‒ Elementor
Mobile users are Functional
devices
more likely to abandon a task if 68%
▪
“
Provides a wide range of benefits including cost reduction, the website isn’t optimized for
opportunity cost reduction, speed, reproducibility, and mobile
consistency Visual Bugs account for 17% of the total bugs
”
‒ Applause
▪ Growth of API economy as API-based applications are more scalable when compared to traditional
API/Service Layer
applications/software, allowing easy integration with other systems Business Rules Automate at the
Functional Test story level
”
added to the Big Data verify it again at multiple integrity is maintained in the – Gartner
storage system nodes. target system
Data Ingestion Data Data Storage Data Migration Data Loading & Outcomes ▪ 100% data validation and quality control
Testing Processing Testing Testing Throughput Delivered ▪ Over 15% reduction due to automated data validation
Testing Testing
▪ Data explosion has led to an increase in AI-led use cases that understand patterns in data Issues Solution
using complex algorithms thereby fueling the need for robust AI systems ▪ Self-driving car killed a pedestrian as a Aspects of AI systems testing
result of software failure. The car’s that could solve these
▪ AI systems testing involves a fundamental shift from output conformance to input validation to
sensors did detect the victim, but problems:
verify the robustness of AI system
unfortunately did not identify her as a ✓ Data Validation
trigger for applying the brakes
✓ Output Validation
▪ Eminent Social Media website faced
✓ Core Algorithm Testing
glitches while understanding and
interpreting customer preferences and ✓ Data Transformation Testing
▪ Natural Language ▪ Domain Specific Data thereby served wrong recommendations
Processing ▪ Contextual data clusters ✓ System Integration Testing
▪ Image Processing ▪ Data denoising/labelling
▪ ML/ Deep Learning
22 Source: Gartner, Infosys Insights, Future market insights, Avendus Research, Wipro Holmes, Zion Market Research.
▪ Cloud testing is the process of using the cloud computing resources of a third-party service ▪ AR / VR blends the physical and virtual worlds for the experience to be immersive for the end-user
provider to test software applications which gives birth to a new quality paradigm that can’t be programmatically tested
called Immersiveness testing
▪ Gaining significant traction because of the speed with which cloud environment is set up
▪ Increasing prominence propelled by user expectation of immersive experiences in gaming,
▪ Assesses physical as well as virtual elements such as storage, web application, network healthcare, leisure, retail, e-commerce, travel, and hospitality
performance, security, scalability, and reliability, interoperability, availability, and multitenancy ▪ AR/VR testing comprises full-stack functionality, compatibility, reliability, usability, security,
testing maintainability, performance, and a new factor “Immersiveness”
“ “
19 455
More than 85% of organizations
are embracing a cloud-first strategy by Almost 70% of respondents believe
8 2025 and 95% of new digital workloads AR/VR will become mainstream within
being deployed on cloud-native platforms, next five years
up from 30% in 2021 15
– Jabil
– Gartner
2020 2028 2020 2030
▪ Test case Creation ▪ Smart Contract Testing Node Addition: Helps test each new block that comes in to make sure that there is no
▪ Test Data Creation Report leakage in the block addition system
▪ Environment Setup ▪ Performance testing Report
▪ Performance Metrics ▪ Project Summary Report
“
137
▪ SRE is the process that leverages operations data and software engineering functions to automate IT operations tasks, to accelerate ▪ Robotic process automation (RPA) is the process to automate
software delivery while minimizing IT risk repetitive business processes
▪ Focused on improving software reliability across categories including availability, performance, latency, efficiency & incident response ▪ Wider and adaptive in scope compared to test automation
with a focus on business processes unlike any specific
feature/ test case
▪ Key Outcomes:
▪ Facilitates GUI test automation, as the RPA tool can
▪ Chaos engineering involves ▪ Auto-Remediation involves ▪ Error budget tracks the maximum capture repetitive IT system interactions
deliberately introducing faulty rectification capabilities that amount of time a system can fail
▪ Reduces error that occurs during manual testing
scenarios in software to verify correct issues automatically as it or underperform without violating
resiliency during disruptions detects them the contractual terms of SLA ▪ Involvement of RPA makes User Acceptance Testing more
productive and accurate
▪ Helps understand expectations ▪ Next step to chaos engineering ▪ Helps reconcile the company's
about what a baseline involving “self-healing” features pace of innovation with its service
looks like reliability
▪ Useful for all test-related activities such as managing test suites, test runs
ALM
and test results etc
▪ No-code test automation tools enable business teams to write and manage
tests without coding skills. These platforms will evolve and widen their scope
with the help of powerful technologies like AI
▪ Test Infrastructure refers to systems/platforms used for carry out the testing,
such as different browsers and mobile devices for compatibility testing
▪ Well-matured QA practice and wider digital transformation, data, and engineering capabilities
Robust testing cycles for functional and non-functional performance will mitigate
failure rates and reduce maintenance costs. Additionally, it will eliminate HR and ▪ Pure-play providers not only help in meeting quality assurance
management-related costs requirement but also contribute significantly towards improving
the quality and agility of business output
Highly experienced and skilled teams integrated into the projects from the beginning
to end results in a fluid testing cycle which means faster turnaround and a quicker time
to market ▪ 52% of firms employ an integrated approach of establishing in-
“
house testing skills while engaging pure-play providers for
specific functions
The focus on quality and speed of testing capabilities requires dedicated and catered expertise from
their providers. As a result, pure-play providers have the potential to play an important role in
▪ Decision-makers use a hybrid strategy, relying on both
meeting these specialized testing needs
specialist pure-plays for customized knowledge and major
”
– Forrester
▪ System Integrators (SIs) for size and established ecosystems
▪ Offers innovative solutions in the next-generation QE space, digital assurance, service virtualization, AI-based test automation, and platform-led
Quality Engineering
3,000+
▪ Industry Leaders in the BFSI domain by catering to QA/ QE needs of marquee logos in the financial services space
▪ Extended expertise in areas such as RPA, DevOps & SRE, Application monitoring & CX tracking apart from QA
▪ Built 10 accelerators focused on automation testing, non-functional testing, testing of chatbots, DevOps
▪ Increased focus on next-generation Quality Engineering technologies by establishing a Center of Excellence in AI testing, RPA for domain-specific
1,000+
capabilities
▪ Global footprint with business across all major markets, including US, Canada, UK, Netherlands, UAE, Singapore, and India
▪ Scaled player with 1000+ software testing professionals working across various challenging projects on new technologies and domain
1,000+
▪ Gained significant expertise in crowd testing and managed QA services
▪ Specialized play with a Test automation framework Tenjin Enterprise and Tenjin Online’ for codeless test automation
500+
▪ Significant expertise in the BFSI domain by catering to the QA needs of marquee names in the banking space
▪ Strong foothold in Next-gen testing services including connected devices testing, AI/ML powered testing, AR-VR testing, and IoT testing
▪ Had setup an emerging technologies practice focused on quality engineering solutions for connected and autonomous vehicles, artificial 350+
intelligence (AI), Internet of Things (IoT), and blockchain
▪ Specialized in digital transformation testing having built a unified test automation framework across web, mobile apps, APIs & web-services and
data pipelines
300+
▪ Strong focus on emerging testing capabilities including AI/ ML, Internet of Things (IoT), big data, and blockchain testing
▪ Strong domain-level expertise in verticals like E-commerce, fintech, and SaaS
▪ Focused on enterprise application testing, which is one of the cornerstones of digital transformation
▪ Next gen testing capabilities include IoT testing: (smart watches, car infotainment systems, smart bulbs), testing
200+
chatbots using AI
▪ Strong domain-level expertise across Insurance, Healthcare, and Technology space with marquee fortune 500 clients
▪ Specialized in niche areas including testing of print system testing, 3D Printer testing, Fax over IP testing, and Smart
grid testing 150+
▪ Strong foothold in North America with clients ranging from Media and Entertainment to Healthcare
▪ Carved its niche in test automation services with most of the Fortune 500 and mid-sized companies as their clients
100+
▪ Strong geographical presence in North America across all verticals including BFSI, Defence, Public Services, Healthcare, Energy, and Manufacturing
▪ Demonstrated continuous innovation and significant shift towards quality ▪ Leveraging ML and NLP to enable intelligent test design automation for
engineering for business through AI-/ML-driven bots, codeless platforms, business process testing and generating automation test scripts NA
and AI-infused accelerators ▪ Automated connected-device testing: Has created a tool called TESBOT to
test for interoperability between firmware, devices, and OS
Wipro
HCL Technologies TCS Accenture
Major Contenders IBM
Capgemini Infosys
DXC Technology
LTI
EPAM
Qualitest
Virtusa Tech Mahindra
Softtek Infogain
Cigniti Atos
UST Mindtree
Cybage Tavant QualityKiosk
ValueLabs
Mphasis Zensar
Sonata Software Aspire TestingXperts
Jade Global Systems
QA Consultants
Mastek
TO THE NEW
Low Aspirants
Low Leaders Major Contenders Aspirants Aspirants High
▪ Focusing more on talent upskilling and adding next- ▪ Contenders are a mix of end-to-end QA service providers, ▪ Leaders include larger Sis backed by technical expertise
generation QA services in the portfolio digital engineering firms, and independent QA service and co-innovation efforts through continued investment in
providers technology & talent development
▪ Transitioning from Aspirants to Contenders will require
more strategic partnerships to expand across depth and ▪ Actively investing in next-generation services like drone ▪ Stiff competition from contenders due to active
breadth of services testing, connected devices testing by building capabilities investment in next-gen quality engineering
or through smaller tuck-in acquisitions
36 Sources: Everest
▪ Revenue growth >20% ▪ Enterprise preferred to mid-tier ▪ Higher share of specialized testing revenue – test
automation, security testing, performance testing,
▪ EBITDA margin >30% ▪ Long-standing relationships preferred
mobile testing, Big Data and IoT testing
▪ High visibility into revenue though Signed Contracts ▪ High spend client across BFSI, Hi-Tech preferred
▪ Revenue from testing of emerging areas like AI,
(SoWs) and multi year MSAs
Cloud
▪ US/Canada deemed most attractive given the ▪ Repeat/recurring revenue preferred to project- ▪ Technology IP and proprietary data are key
depth of market and size/breadth of client based revenue differentiators
accounts
▪ Managed services help increase wallet share ▪ Impact on valuation higher if IP is monetized
▪ Developed markets-focus is the next and revenue visibility
top priority
Mar 2021 ▪ Australia based provider of software testing services 1,300 300 NA NA
Acquisition to acquire Acquisition to extend Acquisition to expand One of the largest QA Acquisition to expand Acquisition to expand global Acquisition to expand
capabilities in niche geographic footprint to quality engineering and offshore assets based quality assurance organizational presence in Quality Assurance
high-growth areas DACH market and digital engineering out of India. Acquisition presence specifically in India coupled with adding capabilities in New
including blockchain expanded capabilities in footprint across the to expand global right- Pharma and Digital existing services including Zealand and strengthen
testing, connected digital engineering and legal industry shoring quality Healthcare verticals functional, automation, and its presence in ANZ
devices testing cyber security assurance offerings performance testing services region
41 Source: MergerMarket
Acquirer/Investor Target Target Description
Deal size: NA
▪ Cloud-hosted, web and mobile application automated testing
Jun’22 platform company
Deal size: NA
▪ US based provider of Next Generation Test Automation Software (Visual test
May’21 automation software)
Deal size: NA
▪ Cloud-hosted, web and mobile application automated testing
TPG – Mar’21 platform company
Riverwood - Mar’19
EV: $1,700Mn
▪ Delivers tools for application performance monitoring, software development,
software testing, API testing and API management
Oct’20
Deal size: NA
▪ Quality platform designed for Agile Testing and DevOps Teams
Aug’20
42 Source: MergerMarket
✓ QA has already undergone a transformation due to CX requirements and a quantum leap in technology
✓ Testing is left-shifting, involves more of AI/ML and more of continuous testing to support CI / CD
✓ AI / ML will take the lead in software testing ✓ User-centered QA and CX will be a huge trend
✓ Automation will flourish even more ✓ Security testing will demand attention in the next two years
✓ Focus of winners will be on flexibility, market data analytics, predicting customer behavior, speed and Time-to-market, security
✓ Pure-play vendors will emerge as market leaders, charged by next-gen technology capabilities, delivering a diversified set of offerings & top-of-the-line economics
✓ Need for QA specialists with programming skills is at an all-time high and set to increase
✓ Winners will continue to invest in high-growth segments such as emerging technology and non-functional testing which have 15%+ growth rates
✓ Winners are focusing on developing IP in the form of frameworks & accelerators across the testing spectrum mainly prompted by needs of automation and DevOps
✓ Quality Assurance is a highly scalable business as the market for software testing cuts across industry verticals and technology horizontals
✓ Business model of testing companies continues to allow for high growth & EBITDA profitability which makes it a lucrative sector
45 Source: MergerMarket
▪ World’s largest pure play testing services company focusing on AI-powered quality assurance and quality engineering services
▪ Backed by Bridgepoint, a leading global PE firm with USD 40 Billion in AUM
▪ Service delivery platform spans US, UK, Israel, Romania, and India
▪ 17 years of robust experience in providing quality engineering enabled ▪ 18+ years of experience in experience in delivery end-to-end testing services
software testing services to product companies ▪ Exposure to software dev. in web, mobile & cloud in existing client accounts
▪ One of the largest independent offshore testing companies with long standing ▪ High domain expertise in verticals- BFSI, Healthcare, Retail, Education & Tech
relationship with clients in the eLearning and publishing verticals ▪ Focus on IP-creation with Centers of Excellence (CoE) in various Next Gen
▪ Global delivery footprint spread across India and US testing services
▪ Positioned QA Infotech as a pure-play offshore QE services provider with ▪ Positioned ZenQ as a must-have asset providing Next Gen QA services such
strong test automation capabilities and a pedigree of marquee, long term as Blockchain, IoT, Big Data and drone technologies testing
client relationships ▪ Negotiated transaction structure & definitive docs to ensure efficient
▪ Helped arrive at the right structure for the transaction with two legs – India & US outcomes for sellers
▪ Navigated the cross-border transaction amid global lockdown. Conducted ▪ Helped negotiate key governance principles and conducted management
management meetings in Dubai in Nov’20 to expedite the process meetings between the parties for an efficient integration
www.avendus.com