This document discusses automated acceptance testing and continuous delivery. It describes how consistent, predictable, high-quality software can be delivered through every iteration using automated testing and continuous delivery practices. These practices include writing specifications and scenarios in a Given/When/Then format to define requirements and tests, automating the testing process, and setting up continuous delivery pipelines to deploy changes quickly and reliably.
1. Automated Acceptance Testing and Continuous Delivery
Larry Apke
Agile Expert
www.agile-doctor.com
larry@agile-doctor.com
2. • The ability to consistently and predictably deliver high-
•
•
•
•
quality software that the end user can easily understand
and use into a production-like environment every
iteration.
Consistent – every iteration
Predictable – over 90% story completion
High Quality – fully tested with no end user
compromising defects
Usable– the end user knows how to, and can, efficiently
use the functionality
4. • Sometimes described with different terms:
• TDD – Test Driven Development
• ATDD – Automated Test Driven Development
• BDD – Behavioral Driven Development
• Specification by Example
• AMDD – Agile Model Driven Development
• Executable specifications
• EDD - Example Driven Development
• AAT - Agile Acceptance Testing
5. • A collaborative approach to defining requirements
and business-oriented functional tests for software
products based on capturing and illustrating
requirements using realistic examples instead of
abstract statements. It is applied in the context of
Agile software development methods, in particular
Behavior driven development. This approach is
particularly successful for managing requirements
and functional tests on large-scale projects of
significant domain and organisational complexity.
Wikipedia.
6. • Specifications are written in terms of behavior or by
using examples
Given…When…Then (Initial Context…Event…Result)
• Referred to as scenarios
• Focuses the conversation at the right level and
becomes something that business and developers can
understand
•
• Scenarios become the framework for automated tests
• The tests that are generated/written from scenarios
fail when run
• Code is written / refactored until tests pass and it
meets code quality standards
7. • Write the Story
Feature: Calculator
In order to avoid silly mistakes
As a math idiot
I want to be able to add, subtract, multiply and divide
numbers
8. • Write the Scenarios
Scenario: 1. Add two numbers
Given I have entered +50 into the calculator
And I have entered +70 into the calculator
When I press calculate
Then the result should be 120 on the screen
9. • Code and Passing the Tests
• Code Coverage
• Feature Files
10. • How long does it take for one line of code that is
changed to flow through the delivery chain?
• How long should it take? Some companies release to
production multiple times per day – Flickr, Facebook,
Twitter, LinkedIn – are held up as examples
11. • Automated Testing
• Every code check in is a release candidate
• Broken builds are fixed immediately
• There is a deployment pipeline that takes release
candidates from one level to the next ie Dev, QA,
UAT, Production
• There are different levels of testing along the pipeline
based on increasing confidence
• Highly automated – one button deploy
12. • Always use version control
• Check in everything – “have everything that can
possibly change at any point in the life of the project
stored in a controlled manner” – Jez Humble –
Continuous Delivery
• Check in to trunk
• Manage dependencies and configuration through
automated scripts
• Build your environments automatically from scratch
13. • Specification By Example - How successful teams
•
•
•
•
deliver the right software
Continuous Delivery: Reliable Software Releases
through Build, Test, and Deployment Automation
The Cucumber Book: Behaviour-Driven Development
for Testers and Developers
Dan North – Introducing BDD http://dannorth.net/introducing-bdd/
Fitnesse - http://fitnesse.org/