Jesse Reed, QA Director at Questar, and Kyle McMeekin discuss how Questar made the switch to qTest and the key factors you should consider in test case management and exploratory testing.
Report
Share
Report
Share
1 of 41
Download to read offline
More Related Content
Quality Jam 2017: Jesse Reed & Kyle McMeekin "Test Case Management & Exploratory Testing Best Practices"
2. Agenda
- Introductions
- Challenges
- Why the move to qTest
- Results of implementation
- Current best practices of Questar
- Future of Questar testing
- Exploratory Testing
- Q & A
4. INSERT PHOTO ON TOP
Jesse Reed
Title: Director of QA
Where: Questar Assessment Inc.
How long: IT industry for 15
How long: QA industry for 8
Where from: Minnesota Grown
Things I like to do: Travel
Other hobbies: Entrepreneur
5. INSERT PHOTO ON TOP
Kyle McMeekin
Title: Senior Sales Engineer
Where: QASymphony
How long: 2 years
How long: 5 years
Where from: Maryland
Things I like to do: Travel and Ski
Other hobbies: Video Games
7. Questar Assessment is an
educational assessment
company focused on
developing and delivering
meaningful measures of
student progress. It is not just
one of the things we do. It’s all
we do.
- Scoring and reporting
- Items and forms
- Test delivery system and
Admin
- Paper and Pencil, Item and
Form Development
- Offshore partners –
MentorMate (Bulgarian)
- Atlassian Suite (Jira/Confluence)
- Microsoft shop mainly using C#
.Net, MS SQL 2012, 2016
- Jenkins, GIT
- MAQS Framework (Selenium
Webdriver,Grid, Appium, .Net
service, MS SQL)
- Jmeter
- Workfront
K-12
Assessment QA Team Technology
8. • Rapidly growing company
• Poor planning
• Lack of process/SDLC/requirements
• Limited automation on legacy systems
• Newly created platform
9. • Overworked and frustrated team
• Limited test cases/No traceability
• No formal QA processes
• Excel and gSheets holders of test cases
• Lack of knowledge of Jira and Confluence
15. Quality Assurance Strategy
• Continuous improvement of Process
• Centralized test case repository
• Traceability of work
• Requirements
• Test cases
• Bugs
Test Planning
•Provide clarity of testing focus
•Assignment of testing duties
•Test coverage
•Transparency
•Reporting
Testing Techniques
•Functional and integration
•System and End-to-end
•Ad hoc/Exploratory
•Regression
•Automation
Technology focused testing
•Utilizing automated strategies
•Continuously increase efficiencies and coverage
Quality
15
Why a Test Management Tool?
28. Questar best practices for qTest
Transparency
is Key
• Large software projects define our
projects
• Test plans are created with 2 week
sprints in mind
• Test case approval from peers
• Import of requirements or stories
from Jira
• Linking of test cases to
requirements or stories
• Hierarchical test case structure
• Modular test cases
• Intuitive test case naming
30. Questar’s plans for the future
Jenkins
integration of
qTest API
Continuous
Integration
Exploratory, Ad
hoc, Session
based
Custom
Reporting
Efficiency
32. PROBLEMS IN TRADITIONAL TESTING
32
Prescribing detailed test steps,
and test planning, can cloud
interactions.
Focus on compressive documentation
blocks cognitive thinking abilities
Customer collaboration in testing
remains in silo or nonexistent.
Comprehensive test plans create a
tunnel vision effect
Individuals and interactions over
processes and tools
Working software over
comprehensive documentation
Customer collaboration over
contract negotiation
Responding to change over
following a plan
33. WHAT IS EXPLORATORY TESTING?
33
1. Parallel test planning, test design, and test
execution
1. Specific yet flexible
1. Aligned towards investigation of potential
opportunities
1. Values depth and attention to detail during testing
1. Fosters knowledge sharing and accountability
34. PARALLEL PLANNING, DESIGNING, EXECUTION
34
Unlike traditional testing techniques, planning, design, and execution happen concurrently,
allowing efficiencies of time as well as flexibility in approach
PLAN
DESIGNEXECUTE
REPORT
PLAN
DESIGN
EXECUTE
REPORT
Traditional Scripted Testing Exploratory Testing
35. Exploratory Testing provides a specific lens through which to perform testing –
whether that be a user person, functionality, criteria (i.e. Localization), etc.
However, it allows testers to use the tool as an end user would, not necessarily as the
product owner envisioned it.
SPECIFIC YET FLEXIBLE
35
I tested the application as the
end user would
I tested the application as
the script prescribed
36. INVESTIGATE OPPORTUNITIES
36
Exploratory testing rewards testers who identify unknown areas of “opportunity”
within the application, as they are essential in maintaining a backlog of future test
charters
Traditional Scripted Testing Exploratory Testing
37. HOW TO STRUCTURE ET: SESSION BASED
37
Session Based Test Management is a popular framework, because it tracks important
data on the testing, while also adding some much needed structure.
• Session charter (includes a mission statement, and
areas to be tested)
• Tester name(s)
• Date and time started
• Task breakdown (the TBS metrics)
• Data files
• Test notes
• Issues
• Bugs
Session Report
More info on SBTM: http://www.satisfice.com/articles/sbtm.pdf
38. UAT WITH EXPLORATORY TESTING
UAT Challenge ET Benefit
UATer’s are unfamiliar test case syntax
and need continual clarification
Allow UATer’s to perform the business
flows they know well without test
scripts
UATer’s are not trained on test case
management, automation tools, etc.
Focus UATer’s time on learning how to
document proper defects, reduce time
to ramp
UATer’s have a shorter attention span
– they are not used to testing 6-8 hrs.
per day
Allow UATer’s to veer off the rails from
time to time and investigate areas of
interest
UATer’s have a short period of time in
which to provide feedback
Ensure that as much of the UATer’s
time as possible is dedicated to ET
39. 39
Keep your Best Testing Talent
IMPROVE JOB SATISFACTION MOVE FROM TESTER TO SME
WHY DO EXPLORATORY TESTING?
40. QASymphony’s qTest Explorer tool
• Applicable on any application;
Including mobile, desktop, and web
• Automatically generate TCs from
your sessions
• Capture environment details
• All sessions stored in central
repository
qTest Explorer
40