Agile Java Testing With Open Source Frameworks
- 1. Agile Java Testing With Open Source Frameworks Presented by Mr. Viraf Karai ********** on Wed. Feb. 25, 2009 1
- 2. Summary of this talk Why test Why test frameworks Why automate tests Unit test frameworks Kinds of tests Integ test frameworks When do you test Load/perf test frameworks Unit testing Other tools (CI) Integration testing Resources Acceptance testing 2
- 3. Why test ? To ensure that the product satisfies all use cases/userstories To gain confidence in your product's abilities To learn about the behavior of your product To deliver a quality product to your customer To fulfil a binding contract with the govt/military 3
- 4. A quote on testing Perhaps one day, writing code without tests will be considered as professionally irresponsible as constructing a bridge without performing a structural analysis. Brian Goetz (JVM and concurrency guru) author of Java Concurrency In Practice. 4
- 5. Why automate tests ? Humans are errorprone by nature Automated tests Save time ● Consume minimal resources on beefy hardware ● Remove the drudgery from repetitive tasks ● Can be run frequently ● Are highly effective with continuous integration ● 5
- 6. Kinds of tests Typically the following tests are done Unit testing – development ● Integration testing – development ● Functional testing (includes load/perf testing) – QA ● Acceptance testing – customers / product manager ● 6
- 7. When do you test ? Agile shops are increasingly embracing TDD Testing is a 1st class activity and should never be done as an afterthought Unit testing while developing code Integration testing – while and after developing code Functional testing – by QA after code complete Load/perf testing – by QA after code compelete Acceptance testing – by customers after QA is done 7
- 8. Unit testing Always done by developers In a TDD environment, tests are written before writing code TDD forces you to make your code testable and can result in very clean design if done right Code shouldn't be checked in without passing tests. Run by developers before code checkin and by CI system periodically (Daily / after code checkins) 8
- 9. Integration testing Ideally done by both dvlp and QA (separately) Involves more bigpicture tests Focus is on collaboration of a number of objects Fixtures to set up are more complex than unit tests Typically touches aspects such as network (http, ftp, smtp) and database interactions 9
- 10. Acceptance testing Tests used to help define when a user story is done Done every iteration and derived from user stories A story isn't complete until all acceptance tests pass Big bang acceptance testing occurs after code complete and testcomplete Onus is on customer/product manager to verify correctness of acceptance tests Automation of acceptance tests is strongly desired 10
- 11. Why use test frameworks ? Absolutely mainstream in agile environments Used by developers, QA and customers Used to test all tiers of an enterprise app Usually involves a learning curve – nothing is free ROI can be substantial once mastered Excellent way to achieve automation Tests almost always experience a faster turnaround 11
- 12. Unit test frameworks Junit 4.x* – unit test Java classes and interfaces TestNG* – unit test Java classes and interfaces EasyMock provides mock objects for interfaces in JUnit/TestNG tests by generating objects on the fly Both* heavily leverage Java5 annotations Both* are supported by Java IDEs: IntelliJ, Eclipse Both* are supported by build tools: Maven, Ant Both* generate reports about failed tests & coverage 12
- 13. Integration test frameworks DbUnit – used to test your persistence tier XmlUnit – used to compare 2 XML documents HttpUnit – used to interact with your webapp and test forms, tables, links, etc. Junit/TestNG with MockHttpServletRequest and MockHttpServletResponse (Spring Framework) to test your servlets 13
- 14. Integration test frameworks (cont'd) Cactus – tests server components e.g. Servlets, EJBs, servlet filters, JSP taglibs, etc. SoapUI inspect, invoke and test web services over HTTP (supported cleanly in Eclipse and IntelliJ) 14
- 15. Functional test frameworks Abbot, JFCUnit, Marathon – Swing func test tools Selenium – advanced tool to test your web app – record, playback and run tests on a large grid. Supports Java, Ruby, Python, Perl, PHP JWebUnit – works with HtmlUnit and Selenium to navigate your web app and verify correctness Canoo WebTest – similar to JWebUnit – write rules (tests) as Ant tasks or Groovy unit tests 15
- 16. Load / perf test frameworks LoadRunner ($) generates load for backend servers by emulating user requests The Grinder – load test HTTP web servers, SOAP and REST web services. Uses Jython for test scripts. JUnitPerf – load and perf testing with Junit TestNG – has annotations to give you control over # concurrent requests and # threads e.g. @Test(invocationCount = 100, threadPoolSize = 10 16
- 17. Acceptance test frameworks Fit, FitNesse Concordion Exactor 17
- 18. Other tools (continuous integration) Consider using a separate build and continuous integration (CI) server Will quickly generate failed test reports with each run. Developers should respond quickly Hook up CI server with CM system (CVS / SVN) and set up build times (daily / after each checkin) Popular CI tools are Hudson, Continuum, Luntbuild, CruiseControl, TeamCity ($) 18
- 19. Resources (web) http://www.ambysoft.com http://www.martinfowler.com http://www.opensourcetesting.org/unit_java.php http://www.onjava.com http://www.ibm.com/developerworks http://www.javaworld.com http://www.parleys.com (MM) http://developers.sun.com/learning/javaoneonline (MM) 19
- 20. Resources (books) JUnit Recipes by J B Rainsberger Test Driven (Practical TDD.....) by Lasse Koskela Next Generation Java Testing by Cedric Beust xUnit Test Patterns by Gerard Meszaros The Art Of Agile Development by James Shore et al 20