This project was originally called 'CL-TEST-MORE'.
'prove' is yet another unit testing framework for Common Lisp.
The advantages of 'prove' are:
- Various simple functions for testing and informative error messages
- ASDF integration
- Extensible test reporters
- Colorizes the report if it's available (note for SLIME)
- Reports test durations
(in-package :cl-user)
(defpackage my-test
(:use :cl
:prove))
(in-package :my-test)
(plan 3)
(ok (not (find 4 '(1 2 3))))
(is 4 4)
(isnt 1 #\1)
(finalize)(prove:run #P"myapp/tests/my-test.lisp")
(prove:run #P"myapp/tests/my-test.lisp" :reporter :list)See also: ASDF integration, Reporters
You can install 'prove' via Quicklisp.
(ql:quickload :prove)Checks if test is true (non-NIL).
(ok 1)
;-> ✓ 1 is expected to be TChecks if got is equivalent to expected.
(is 1 1)
;-> ✓ 1 is expected to be 1
(is #(1 2 3) #(1 2 3))
;-> × #(1 2 3) is expected to be #(1 2 3)
(is #(1 2 3) #(1 2 3) :test #'equalp)
;-> ✓ #(1 2 3) is expected to be #(1 2 3)
;; with description
(is 1 #\1 "Integer = Character ?")
;-> × Integer = Character ?Checks if got is not equivalent to expected.
(isnt 1 1)
;-> × 1 is not expected to be 1
(isnt #(1 2 3) #(1 2 3))
;-> ✓ #(1 2 3) is not expected to be #(1 2 3)Checks if the multiple values of got is equivalent to expected. This is same to (is (multiple-value-list got) expected).
(defvar *person* (make-hash-table))
(is-values (gethash :name *person*) '("Eitaro" T))
;-> × (NIL NIL) is expected to be ("Eitaro" T)
(setf (gethash :name *person*) "Eitaro")
(is-values (gethash :name *person*) '("Eitaro" T))
;-> ✓ ("Eitaro" T) is expected to be ("Eitaro" T)Checks if got is a type of expected-type.
(is-type #(1 2 3) 'simple-vector)
;-> ✓ #(1 2 3) is expected to be a type of SIMPLE-VECTOR (got (SIMPLE-VECTOR 3))
(is-type (make-array 0 :adjustable t) 'simple-vector)
;-> × #() is expected to be a type of SIMPLE-VECTOR (got (VECTOR T 0))Checks if got matches a regular expression regex.
(like "Hatsune 39" "\\d")
;-> ✓ "Hatsune 39" is expected to be like "\\d"
(like "初音ミク" "\\d")
;-> × "初音ミク" is expected to be like "\\d"Checks if got outputs expected to *standard-output*
(is-print (princ "Hi, there") "Hi, there")
;-> ✓ (PRINC "Hi, there") is expected to output "Hi, there" (got "Hi, there")Checks if form raises a condition and that is a subtype of condition.
(is-error (error "Something wrong") 'simple-error)
;-> ✓ (ERROR "Something wrong") is expected to raise a condition SIMPLE-ERROR (got #<SIMPLE-ERROR "Something wrong" {100628FE53}>)
(define-condition my-error (simple-error) ())
(is-error (error "Something wrong") 'my-error)
;-> × (ERROR "Something wrong") is expected to raise a condition MY-ERROR (got #<SIMPLE-ERROR "Something wrong" {100648E553}>)Checks if got will be macroexpanded to expected.
(is-expand (when T (princ "Hi")) (if T (progn (princ "Hi"))))
;-> ✓ (WHEN T (PRINC "Hi")) is expected to be expanded to (IF T
; (PROGN (PRINC "Hi"))) (got (IF T
; (PROGN
; (PRINC
; "Hi"))
; NIL))If a symbol that starts with "$" is contained, it will be treated as a gensym.
This will always be passed. This is convenient if the test case is complicated and hard to test with ok.
(pass "Looks good")
;-> ✓ Looks goodThis will always be failed. This is convenient if the test case is complicated and hard to test with ok.
(fail "Hopeless")
;-> × HopelessSkip a number of how-many tests and mark them passed.
(skip 3 "No need to test these on Mac OS X")
;-> ✓ No need to test these on Mac OS X (Skipped)
; ✓ No need to test these on Mac OS X (Skipped)
; ✓ No need to test these on Mac OS X (Skipped)Run tests of body in a new sub test suite.
(subtest "Testing integers"
(is 1 1)
(is-type 1 'bit)
(is-type 10 'fixnum))
;-> ✓ 1 is expected to be 1
; ✓ 1 is expected to be a type of BIT (got BIT)
; ✓ 10 is expected to be a type of FIXNUM (got (INTEGER 0 4611686018427387903))
;-> ✓ Testing integersOutputs desc to a *test-result-output*.
(diag "Gonna run tests")
;-> # Gonna run testsDeclares a number of num tests are going to run. If finalize is called with no plan, a warning message will be output. num is allows to be NIL if you have no plan yet.
Finalizes the current test suite and outputs the test reports.
Set the threshold of slow test durations for the current test suite. The default threshold value is prove:*default-slow-threshold*.
(slow-threshold 150)You can change the test report formats by setting prove:*default-reporter* to :list, :dot, :tap or :fiveam. The default value is :list.
prove:run also takes a keyword argument :reporter.
The :list repoter outputs test results list as test cases pass or fail.
The :dot reporter outputs a series of dots that represent test cases, failures highlight in red, skipping in cyan.
The :fiveam reporter outputs test results like FiveAM does.
The :tap reporter outputs in Test Anything Protocol format.
Set prove:*debug-on-error* T for invoking CL debugger whenever getting an error during running tests.
SLIME doesn't support to color with ANSI colors in the REPL buffer officially.
You can add the feature by using slime-repl-ansi-color.el.
After installing it, set prove:*enable-colors* to T before running tests.
;; A part of my ~/.sbclrc
(ql:quickload :prove)
(setf prove:*enable-colors* t)The following snippet is a little bit complicated, however it would be better if you don't like to load prove in all sessions.
(defmethod asdf:perform :after ((op asdf:load-op) (c (eql (asdf:find-system :prove))))
(setf (symbol-value (intern (string :*enable-colors*) :prove)) t))Add :defsystem-depends-on (:prove-asdf) to your testing ASDF system to enable :test-file in the :components.
:test-file is same as :file except it will be loaded only when asdf:test-system.
;; Main ASDF system
(defsystem my-app
;; ...
:in-order-to ((test-op (test-op my-app-test))))
;; Testing ASDF system
(defsystem my-app-test
:depends-on (:my-app
:prove)
:defsystem-depends-on (:prove-asdf)
:components
((:test-file "my-app"))
:perform (test-op :after (op c)
(funcall (intern #.(string :run) :prove) c)))To run tests, execute asdf:test-system or prove:run in your REPL.
(asdf:test-system :my-app)
(asdf:test-system :my-app-test)
;; Same as 'asdf:test-system' except it returns T or NIL as the result of tests.
(prove:run :my-app-test)Test functions like is uses prove:*default-test-function* for testing if no :test argument is specified. The default value is #'equal.
Test reports will be output to prove:*test-result-output*. The default value is T, which means *standard-output*.
Although Common Lisp isn't supported by Travis CI officially, you can run tests by using cl-travis.
Here's a list of .travis.yml from projects using prove on Travis CI:
Please report any bugs to e.arrows@gmail.com, or post an issue to GitHub.
Copyright (c) 2010-2014 Eitaro Fukamachi <e.arrows@gmail.com>
'prove' and CL-TEST-MORE is freely distributable under the MIT License (http://www.opensource.org/licenses/mit-license).