There are three main directories of tests right now:

• pass/ which contain Python files that must pass mypy checking with no type errors
• fail/ which contain Python files that must fail mypy checking with the annotated errors
• reveal/ which contain Python files that must output the correct types with reveal_type

fail and reveal are annotated with comments that specify what error mypy threw and what type should be revealed respectively. The format looks like:

bad_function   # E: <error message>
reveal_type(x)   # E: <type name>

Right now, the error messages and types are must be contained within corresponding mypy message.

To setup your test environment, cd into the root of the repo and run:

pip install -r test-requirements.txt

To run the tests, do

python runtests.py

from the repo root. To run mypy on a specific file (which can be useful for debugging), you can also run:

pip install .  # Make sure stubs are installed; runtests does this for you
mypy <file_path>

Note that it is assumed that all of these commands target the same underlying Python interpreter. To ensure you're using the intended version of Python you can use python -m versions of these commands instead:

python -m pip install -r test-requirements.txt
python -m pip install .
python -m mypy <file_path>