Software Engineering Process using Agile Methodology

Software Engineering Process using Agile Methodology

Author: Rakesh Wagh

Keywords
"Agile software development",
"Continuous Integration",
"Test driven development"

Abstract
Agile software development practices offer compelling advantages like
streamlined processes, less “paper work”, high quality, creativity in work,
shorter delivery cycles, and above all customer satisfaction. Proper adoption and
implementation of agile processes is a challenge in itself. Automated tools and
practices like Continuous Integration, Test Driven Development, etc can largely
simplify agile adoption.

http://itnow.blogspot.com 1
Software Engineering Process using Agile Methodology

Agile software development................................................................................................3

Advantages of Agile .......................................................................................................3
Agile in a nutshell ...........................................................................................................3
Ever changing scope ...................................................................................................4
Quality and Testing .....................................................................................................5
Deliverables.................................................................................................................7
Documentation ............................................................................................................7
Communication............................................................................................................8
Challenges with Agile methodology................................................................................9
Project Management ...................................................................................................9
People ..........................................................................................................................9
Other Challenges..............................................................................................................9
Agile Methods ...............................................................................................................10
Extreme Programming (XP)......................................................................................10
Feature Driven Development (FDD).........................................................................11
Dynamic Systems Development Method (DSDM)...................................................12
Rational Unified Process or Agile Unified process(RUP or AUP)...........................13
SCRUM......................................................................................................................14
Tools for Agile process......................................................................................................14
Continuous Integration (CI) Tools ................................................................................14
Commercial CI servers .............................................................................................15
Open Source or Free CI Servers ...............................................................................15
Test Driven Development Tools (TDD) .......................................................................15
Issue/Bug/Defect Tracker .............................................................................................16
Team Collaboration and Wiki Tools .............................................................................16
Java based IDE ..............................................................................................................16
Build system ..................................................................................................................16
Source Code Management (SCM) ................................................................................16
Code quality and optimization ......................................................................................17
Final words: ......................................................................................................................17
External References...........................................................................................................17

http://itnow.blogspot.com 2
Software Engineering Process using Agile Methodology

Agile software development

Agile is an evolutionary software engineering methodology that addresses the drawbacks
found in traditional software development practices. Agile methodology emphasizes on:
individuals and interactions over processes and tools, working software over
comprehensive documentation, customer collaboration over contract negotiation,
responding to change over following a plan.

Below are some notable quotes about Agile.

"Agile software development is a conceptual framework for undertaking software

engineering projects that embraces and promotes evolutionary change
throughout the entire life-cycle of the project."
- Wikipedia

"Characterized by quickness, lightness, and ease of movement"

- Dictionary Definition of the word ‘Agile’

"Agile is an iterative and incremental (evolutionary) approach to software

development which is performed in a highly collaborative manner with ‘just
enough’ ceremony that produces high quality software in a cost effective and
timely manner which meets the changing needs of its stakeholders.”
- AgileModeling.com

Advantages of Agile
- Shortened development cycle
- Higher stability of work-loads
- Higher utilization of work-load, that is, developing large-scale, software
systems with a fixed number of developers,
- Higher flexibility to change of Management & development plans
- Higher quality by earlier feedback from the customers.
- Customer satisfaction by rapid, continuous delivery of useful software
- Working software is delivered frequently (weeks rather than months)
- Even late changes in requirements are welcomed
- Simplicity

Agile in a nutshell
- Develop project in small iterations. Iteration length should be fixed early on
(based on the practice, example XP, RUP, etc).
- Each iteration (often called as time-box) is a mini project in itself. Every
iteration will have its own planning, requirements analysis, design, coding,
testing, and documentation phase
- Do not stick to a pre defined process. Process should be tailored based on
project need and various other parameters like people skills, requirements, etc.

http://itnow.blogspot.com 3
Software Engineering Process using Agile Methodology

- Aim for fluid long term plans and concrete short term plan.
- Reevaluate project priorities and process at the end of each iteration.
- A usable though minimal system can go in production at the end of every
iteration.
- Questions to ask in the meeting at the end of every iteration
o What did we do well?
o What have we learned?
o What can we do better?
o What puzzles us?
- Good team communication and collaboration is very important. Prefer face to
face, oral communication over written documentation. Use collaboration tool
to share ideas and easily disseminate information.
- If possible, physically locate the team together. Modern communication
methods like frequent telephone/video conferencing should be used if the
project team is geographically distributed (e.g. onsite-offshore model).
- Prefer developed software (or the features completed) over documents or task
list as a primary measure to track progress.
- Developers should have access to good quality business expertise.
- Welcome changes; be adaptive rather then predictive; people oriented, rather
than process oriented.
- Control an unpredictable process through iterations.
- Define a good feedback system that will tell us where we are at the end of
every iteration.
o Example: Customer himself can provide excellent feedback.
- Primary concern should be "business value", did the customer get what he
asked for?

Ever changing scope

- It is important to understand that business user (customer) does not always
have a very good picture of what the production software should actually look
like. Add it with the complexity of communicating these requirements back to
the development team and finally blending it into a finished product.
- “Customer requirements” does not necessarily always translate to a “business
requirement”.
o Example: Customer might require a “link” on the form page. However
it is important to know the business case behind having that link and
providing a solution to address the business case and not the link!
- Scope change is the reality of software development. As the system is
developed, both business and development learn new things about the
problem they are trying to solve. It is not uncommon to have a huge delta
between the understanding of requirements at the beginning and at the end of
a project. With this delta not delivered in the software, we often end up with
unhappy customer.
- Most valuable features are not obvious until customer have had a chance to
play with the software.

http://itnow.blogspot.com 4
Software Engineering Process using Agile Methodology

- It is not uncommon to see an unhappy customer even when the deliverables

strictly adhere to the original project charter.

- Scope changes in traditional software development

o Processes are based on scenarios that are envisioned in the planning
phase. New scenarios, or changes to existing ones are often considered
"bad" for the project plan.
o Change in scope lead to slippage of schedule, increase in cost and lot
of frequent changes to the project plan baseline.
o This happens because scope, cost and schedule baselines are predicted
early on in the planning phase. Project is “expected” to stick on this
baseline there after. Changing the baseline plan is an option but it is
not very desirable neither is it always possible (especially late in the
project).
o Over all, high resistance towards change in scope.

- Scope changes in Agile approach

o Agile model welcome change.
o Change is looked at as the driving force towards delivering a better
product/service.
o Change and new requirement help to define the deliverable for the
next iteration.
o Changes fuel incremental growth of the application in a direction
desired by the customer.
o Changes drive the design and development.
o Customer is always happy with the end product because he has had
active participation in the development process.
o Cost and time baselines are fixed in the beginning of the project.
Scope is never base lined and is added incrementally.
o Agile process encourages the business to learn about their need as the
software is build.

Quality and Testing

- Software quality is not only about delivering bug free code, but delivering a
system that is adaptive to future changes.
- Quality is often considered as an integral part of scope.
- A good quality code (and design) will be much more adaptive to changes than
its reverse. The vice versa is also true and the driving force for agile the
development practice: “A piece of software that can easily adapt to new
changes is inherently superior in quality”.
- Quality requirement for IT projects are difficult to baseline and quantify.
- Quality is provided by performing rigorous testing.
o Testing is broadly divided into "functional testing" and "unit testing".

http://itnow.blogspot.com 5
Software Engineering Process using Agile Methodology

o Functional testing look at the bigger picture and concentrates more on

functionality then on programming and design errors. Functional
testing is performed by team of testers who have nothing to do with
real code.
o Unit testing looks into the actual code and attempt to capture bugs
induced during construction and design. Development team is
responsible for unit testing the application.
- A rigorous unit and functional testing is key to delivering a good quality
software.
- Traditional approach towards Testing/QA
o Construction and Testing (QA) are two distinct (often non-
overlapping) phases performed by separate teams.
o Unit testing is often ignored or not planned properly because of this
phase separation.
o Creating "unit test case" is not the primary objective of construction
phase; reason why unit testing is largely ignored in projects.
o While the system is in QA, development resources are expected to fix
the bugs found by quality team. There is no real time allotted for unit
testing.
o Pushing QA(testing) towards the end of project can lead to a "fat" bug
fixing cycle which often result in code patches that might inject further
"non unit tested" errors.
o If design bugs are discovered when in QA, there is no choice but to
hide them.
o "Work around"(which should be avoided) is the quickest and often
most adopted way of fixing bugs in this stage.
o Software created by such practice can be "bug free" but "low in
quality". Low quality of software is typically discovered when the
system is in production.
- Agile approach towards Testing/QA
o Testing is integral part of construction.
o Functional team will do the functional testing and development team
will do unit testing. This is done for every iteration.
o Incremental testing and bug fixing will avoid a major bug to be
discovered late in the project.
o Developers get the chance to provide the best solution instead of a
quick fix or work around.
o No notion of final testing and bug fixing.
o Quality of the final deliverable is greater or equal to the quality of
deliverables at the end of each iteration.
o Testing time is accounted in the iteration plan.
o Code development will go hand in hand with test development.
o As unwanted features are eliminated in each iteration, major testing
effort is concentrated only on the real system.

http://itnow.blogspot.com 6
Software Engineering Process using Agile Methodology

Deliverables
- Traditional approach for project deliverables
o Requirement collection is often the first (and only) phase in project life
cycle. Requirements provided by the user often form the scope
baseline which is verified against the final deliverable.
o Lot of customer interaction is done early on to properly "capture the
requirement" and to deliver the same.
o This approach attempts to deliver the entire software at once, based on
the requirements captured in early stage of the project.
o Projects often have milestone or end of phase deliverables. But these
deliverables are in form of documents. Documents can differ from the
actual development and fail to give a clear picture of the developed
software.
- Agile approach for project deliverables
o Final deliverables are not "completely" planned early on.
o Deliverables are planned only for the next iteration.
o Entire application is delivered in "production ready" quality at the end
of each iteration (with more features in each delivery).
o Customer can get early access to a striped down version of the
software.
o Customer expectations are satisfied and re evaluated at the completion
of every iteration.
o Difference in understanding between the customer and development
team reduces with every iteration.
o No surprises in the final delivery!
o Delivery in form of working system instead of documents.

Documentation
- Documentation is a form of communication.
- Documentation can also be used for information archival or historic purposes.
- Traditional approach
o Traditional process will typically have lot of documentation in every
phase of software dev. Note: “Code-and-fix” style software
development might have no documentation but it often leads to chaos.
o Documentation create in such projects will generally hide the system
flaws and concentrate on what works.
o While documentation might make a system look "complete", in reality
it might not capture or correctly communicate the real essence of its
existence.
o Documentation is done either in early phase or later in the
development process, leading to documents that are either not

http://itnow.blogspot.com 7
Software Engineering Process using Agile Methodology

synchronized with the real application or documents that poorly

represent the real software.
- Agile Approach
o Documents do not contribute directly to the working of software and
should be avoided if possible.
o Prefer executable work packages such as customer tests and developer
tests over static work packages such as plain old documentation
(POD).
o Efforts should be directed to create high quality source code backed by
test cases instead of creating good documents.
o Over all system should be "lean" and simple enough so that new
developers can kick start development by looking at the source code!
o Document should be created only if the benefit of having
documentation is greater than the cost of creating and maintaining it.
o Prefer other forms of communication over documentation.
o Require people to justify documentation requests. “I need to
show/deliver something to my boss” is not a legitimate request!
o Documentation should be concise: overviews/roadmaps are generally
preferred over detailed documentation.
o Detailed code commenting should be encouraged.
o Some of the circumstances where documentation might be
unavoidable: project stakeholder’s communication; B2B like
communication where you have a contract model (e.g. web services);
communication with an “external” group; idea collection for brain
storming, audit purposes, requirement/support/user documents.
o In most cases, documentation efforts should be recognized and
“funded” by the business. Overall documents should be understood as
cost to the project and not primary project deliverable. Primary project
deliverable should be the application software/service.

Communication
- Good communication is one of the keys to successful agile projects. Bad
communication can very easily drive agile projects to failure.
- Every team member should be aware of the advantages of communicating and
the disadvantages of not doing so.
- Project management should provide an environment where good
communication can take place.
- People availability (especially business analyst or domain expert) is an
important factor.
- Face to face communication should be done where ever possible.
- Oral communication should be preferred over written.
- Special measures should be taken if the team is distributed (example offshore)
or working remotely (work from home).

http://itnow.blogspot.com 8
Software Engineering Process using Agile Methodology

o (daily/weekly) reports and status update meetings, which allow the

team members to synchronize, discuss achievements, and reveal
problems should be conducted.
o In offshore outsourcing projects, development managers should be
aware of language, cultural, and time-zone barriers, and must find
ways to surmount these obstacles.

Challenges with Agile methodology

Project Management
o Agile demand that people be recognized for their skills and be treated
as an important/integral part of the application development cycle.
o On the contrary modern project management looks at people as
"resources" and their capabilities as "roles".
o From a project management perspective, final product of the project is
delivered only when the user signs it off.
o Agile requires "production ready" product to be delivered at the end of
each iteration.
o In a typical agile scenario, management shares its responsibility with
development and recognizes there importance and skills.
o Many such differences should be sorted out and agreed upon between
management and rest of the team.

People
o Requires very efficient team of developers.
o People should work together and communicate frequently and openly.
o Agile recognize people for there skills, expertise, creativity and
uniqueness. Every team member is important and responsible for
his/her part of the project. Loss of such team members often create a
void space which is not easily filled by a new replacement.
o Agile projects succeed fast, but if not handled properly, they can also
fail fast!
o "Self-Organizing Teams" often require micro management which is
not always welcomed.
o People must be trusted
o Organizations must live with the decisions developers make

Other Challenges
- Getting started and coming up to speed might be the most difficult part.
- Flawed implementation of agile could be very dangerous for the project
health.
- Loss of privacy. It takes time for people to get used to working together.
- Communication and management can be challenging with offshore teams.
- The culture of the organization must be supportive of negotiation

http://itnow.blogspot.com 9
Software Engineering Process using Agile Methodology

Agile Methods
Numerous agile methods have been proposed and used over the last decade. Listed below
are some of the most popular methods, often adopted in agile projects. Explaining each
method is not in scope of this document. It is recommended to select a method that fits
individual project and team requirement. Once the approach is finalized, it should be
properly understood by all the team members (project manager, developers, testing team,
business analysts, etc).

Extreme Programming (XP)

- The main aim of XP is to reduce the cost of change. XP sets out to reduce the
cost of change by introducing basic values, principles and practices. By
applying XP, a system should be more flexible with respect to changes.
- Extreme Programming is recognized for four values: Communication,
Simplicity, Feedback, Courage and Respect
- Best Practices:
- Fine scale feedback
o Pair programming
o Planning Game
o Test Driven Development
o Whole team
- Continuous process
o Continuous Integration
o Design Improvement
o Small Releases
- Shared understanding
o Coding Standards
o Collective Code Ownership
o Simple Design
o System Metaphor
- Programmer welfare
o Sustainable Pace

http://itnow.blogspot.com 10
Software Engineering Process using Agile Methodology

Feature Driven Development (FDD)

- FDD blends a number of industry-recognized best practices into a cohesive
whole. These practices are all driven from a client-valued functionality
(feature) perspective. Its main purpose is to deliver tangible, working software
repeatedly in a timely manner.
- FDD is a model-driven short-iteration process that consists of five basic
activities. For accurate state reporting and keeping track of the software
development project, milestones that mark the progress made on each feature
are defined.
- Ffive basic activities:
o Develop overall model
o Build feature list
o Plan by feature
o Design by feature
o Build by feature.
- Best Practices:
o Domain Object Modeling: consists of exploring and explaining the
domain of the problem to be solved.
o Developing by Feature: If a function requires more than 2 weeks of
development time, decompose it into smaller features.
o Individual Class (Code) Ownership.
o Feature Teams: is a small, dynamically formed team that develops a
small activity.
o Configuration Management: identifying the source code for all
features that have been completed
o Regular Builds.
o Visibility of progress and results
o Inspections

http://itnow.blogspot.com 11
Software Engineering Process using Agile Methodology

Dynamic Systems Development Method (DSDM)

DSDM recognizes that projects are limited by time and resources, and plans
accordingly to meet the business needs. In order to achieve these goals,
DSDM encourages the use of RAD with the consequent danger that too many
corners are cut. DSDM applies some principles, roles, and techniques.

DSDM consists of 3 phases: pre-project phase, project life-cycle phase, and

post project phase. The project life-cycle phase is subdivided into 5 stages:
feasibility study, business study, functional model iteration, design and build
iteration, and implementation.

http://itnow.blogspot.com 12
Software Engineering Process using Agile Methodology

Rational Unified Process or Agile Unified process(RUP or AUP)

The RUP is not a single concrete prescriptive process, but rather an adaptable process
framework, intended to be tailored by the development organizations and software
project teams that will select the elements of the process that are appropriate for their
needs.

- Characteristics:
- Iterative and Incremental
- Use Case Driven(use cases are used to capture the functional
requirements and to define the contents of the iterations)
- Architecture Centric
- Risk Focused
- Project Lifecycle: Inception Phase, Elaboration Phase, Construction Phase,
Transition Phase.
- AUP is a simplified version of RUP

http://itnow.blogspot.com 13
Software Engineering Process using Agile Methodology

SCRUM
- SCRUM is a management, enhancement and maintenance methodology
for an existing system. SCRUM will address new or re-engineered
systems development efforts at a later date.
- Software product releases are planned based on the following variables :
o Customer requirements - how the current system needs enhancing.
o Time pressure - what time frame is required to gain a competitive
advantage.
o Competition - what is the competition up to, and what is required to
best them.
o Quality - What is the required quality, given the above variables.
o Vision - what changes are required at this stage to fulfill the system
vision.
o Resource - what staff and funding are available.

These variables form the initial plan for a software enhancement project.
However, these variables also change during the project. A successful
development methodology must take these variables and their evolutionary nature
into account.

Tools for Agile process

In this section we will list the most popular tools for various software engineering
categories.

Continuous Integration (CI) Tools

- Continuous Integration is a software engineering process that executes the
build-test-deploy cycle at regular intervals.
- This process guarantee the latest code in all (but production) environments.
- Entire process is automatically handled by the continuous integration server.
- Integration problems are detected and fixed continuously - no last minute
hiatus before release dates;
- Early warning of broken/incompatible code;
- Immediate unit testing of all changes;
- Constant availability of a "current" build for testing, demo, or for release
purposes;
- The immediate impact of checking-in incomplete or broken code acts as an
incentive to developers to learn to work more incrementally with shorter
feedback cycles.
- Modern CI tools allow advanced features like report generation, alerts via IM,
email, etc.

http://itnow.blogspot.com 14
Software Engineering Process using Agile Methodology

- CI tools can interact with other software development life cycle tools like bug
tracking system and eclipse IDE to provide valuable development time
information.

Commercial CI servers
- AnthillPro - a commercial tool from Urbancode. Along with
CruiseControl, Anthill is one of the original tools in this category; it
remains a market and technology leader constantly adding new
features such as dependency mangement, workflow automation, etc.
- Bamboo - Commercial continuous integration server from Atlassian
- Build Forge - IBM Rational's adaptive framework to standardize and
automate build and release processes.
- Electric Commander - a continuous integration server by Electric
Cloud, John Ousterhout of Tcl fame's company.
- TeamCity - a commercial tool by JetBrains. Web-based with IDE
(IntelliJ IDEA, Eclipse, Microsoft Visual Studio) support. Includes
many innovative features.
- QuickBuildPro - by PMEase

Open Source or Free CI Servers

- CruiseControl - Java-based framework for a continuous build process.
- Hudson - MIT licensed, written in Java, runs in servlet container,
supports CVS, Subversion, Ant, Maven, and shell scripts.
- LuntBuild - a powerful open source build automation and management
tool
- Tinderbox - a Mozilla based product
- Bitten - A continuous integration plugin for Trac.
- BuildBot - a Python/Twisted-based continuous build system
- CABIE - Continuous Automated Build and Integration Environment.
Open source, written in Perl, works with CVS, Subversion and
Perforce.
- DamageControl - a continuous integration server for Ruby.
- Apache Gump - Apache's continuous integration tool.
- Apache Continuum - a continuous integration server supporting
Apache Maven and Apache Ant.

Test Driven Development Tools (TDD)

TDD is the key for agile practices. Strict adoption of TDD results in high quality code.
- junit4 - for unit testing
- StrutsTestCase - for struts testing
- TestNG - junit plus some new features like annotation, data-driven
testing,

http://itnow.blogspot.com 15
Software Engineering Process using Agile Methodology

- DbUnit- to test database components

- XMLUnit- to test XML components
- JUnitPerf - for performance testing
- JMeter- load and performance testing
- Cobertura - to Maximizing test coverage.
- JSTester - Testing javascript
- Selenium - Automated functional testing
- Jester - makes some changes to the code and try to find out if the test
cases catch it!

Issue/Bug/Defect Tracker
- JIRA(commercial) - Java based issue tracking system
- Trac – A Wiki-based issue tracking system
- Mylin(old name Mylar) - to manage tasks in Eclipse

Team Collaboration and Wiki Tools

- Openfire - cross-platform real-time collaboration server based on the
XMPP (Jabber) protocol from Ignite.
- Macromedia Breeze - open source
- Spark - Cross-platform real-time collaboration client optimized for
business and organizations.
- Maven 2 - can do build, reporting and documentation from a center
place.
- Confluence - By Atlassian is an enterprise wiki that makes it easy for
your team to collaborate and share knowledge.
- twiki - Enterprise Wiki & Collaboration Platform

Java based IDE

- Eclipse
- IDea (commercial)
- NetBeans
- MyEclipse - Commercial Eclipse plugin that simplify eclipse
development.
- Weblogic Workshop - Commercial Eclipse plugin that simplify eclipse
development.

Build system
- Ant
- Maven2
- Krysalis Centipede

Source Code Management (SCM)

- Generation 0: VSS, PVCS, …
- Generation 1: SVN, CVS, etc..

http://itnow.blogspot.com 16
Software Engineering Process using Agile Methodology

- Generation 2: Git, Mercurial, Bazaar

Code quality and optimization

- Checkstyle - Defining and enforcing coding standards
- PMD - Detecting defects and non-optimal code
- FindBugs - Detecting potential bugs
- QaLab - Build Statistics
- SourceMonitor (for cyclometer complexity, etc).
- AppPerfect

Final words:
Agile methodology has been criticized in some areas that parallel or cross management
practices. However, if implemented properly and thoughtfully, agile practices can make
the best use of people to deliver very high quality software that meets or exceeds
customer expectations. TDD, CI and collaboration tools can help simplify the agile
process.

External References
Agile Documentation
http://www.agilemodeling.com/essays/agileDocumentation.htm
http://www.ddj.com/184415786

Agile and offshore

http://msdn2.microsoft.com/en-us/library/bb245671.aspx
http://www.agileadvice.com/archives/2006/09/offshoredistrib.html

SCM Tool comparison

http://www.relisoft.com/co_op/vcs_breakdown.html
http://en.wikipedia.org/wiki/Comparison_of_revision_control_software

Defining Promotion Levels

http://www.cmcrossroads.com/content/view/6729/202/

Streamed Lines: Branching Patterns for Parallel Software Development

http://www.cmcrossroads.com/bradapp/acme/branching/

Agile and "Traditional PMI" Methods

http://leadinganswers.typepad.com/leading_answers/2006/09/agile_and_tradi.html

Using Agile alongside PMBOK

http://leadinganswers.typepad.com/leading_answers/files/using_agile_alongside_the_pmbok.pdf

DSDM with Prince2

http://agilealliance.org/system/article/file/904/file.pdf

Agile Challenges
http://www.agileadvice.com/archives/2006/09/agile_challenge.html

Agile Testing What is it? Can it work?

http://www.io.com/~wazmo/papers/agile_testing_challenges.pdf

Functional test tools

http://www.opensourcetesting.org/functional.php

Wiki tools Comparison

http://itnow.blogspot.com 17
Software Engineering Process using Agile Methodology

http://en.wikipedia.org/wiki/Comparison_of_wiki_software

http://itnow.blogspot.com 18