To start, we need to download some Haskell packages to fuzz. This is most easily done with the ./prepare.sh script.

See the section "Obtaining Packages, Continued" at the bottom for how to do this manually

You can automatically generate and run test suites for modules we now how to fuzz by running the script ./test-all-working.sh

To run the synth tool (for testing), use the command

stack run -- --searchterm [DEFS_FILE] TYPE_NAME

where DEFS_FILE is an optional file that includes package-specific definitions. Note that the prelude in searchterm/prelude.txt is loaded by default.

The rulecheck project is responsible for generating a test suite for the rewrite rules in a given package. Before using it, you should tell it the appropriate values for $PACKAGEDIR and $RULESFILE. This can be done by modifying appropriate values in the Config.hs file.

To generate a suite for a single package $PACKAGENAME, run the command:

stack run -- $PACKAGENAME

If you want to generate a suite for every package in $RULESFILE, this can be done by running without any arguments:

stack run

After generating a test suite for $PACKAGENAME, the appropriate tests will be created in package-tests/$PACKAGENAME-test.

You can then easily run the fuzz test by running stack test from inside the generated directory.

The above two steps can be done at once with the command ./test.sh $PACKAGENAME

Just edit ./test-all-working.sh, it should be straightforward

This can be done by adding a file in test-template/extra-arbitrary-instances/$PACKAGENAME.hs.

You may want to do this to expose additional modules, functions, etc. This can be done by putting modified versions of the file in the vendored directory. The script ./apply-vendored-patches.sh will apply the patches.

You can use scripts in the auto directory to download packages containing rewrite rules. Here's how to do it in three steps (from the auto directory):

./run.sh main find --listing $LISTINGFILE
./run.sh main download --listing $LISTINGFILE --scratch $PACKAGEDIR
./run.sh main extract --listing $LISTINGFILE --scratch $PACKAGEDIR --output $RULESFILE

where $LISTINGFILE is a temporary file to store the package information from hackage. $PACKAGEDIR is where the packages will be downloaded, and $RULESFILE contains information on packages containing rewrite rules.

IMPORTANT Current we expect $PACKAGEDIR to be named haskell-packages and placed adjacent to this directory (i.e. ../haskell-packages). AND you need to update the path of haskellPackagesDir in Config.hs.

The first thing you need to do is to download the modified version of haddock that can generate the definition files. This can be done by downloading the zip file:

https://github.com/zgrannan/haddock/archive/refs/heads/rulecheck-9.0.2.zip

Then run cabal install in that directory.

To generate the defs for a package, first switch to the directory for that package. Then, FROM THAT DIRECTORY run the generate-defs-file.sh script, i.e. $RULECHECK_DIR/generate-defs-file.sh. The result should be a file defs.txt in the current directory.

Various issues can arise during both test suite generation and running the generated test suites. Below are some common issues, and how they can be addressed.

The test generation logic runs separately for each file in a project that contains rewrite rules. However, some files actually shouldn't be analyzed for the following reasons:

• The file is actually a test file (for programs that analyze Haskell code, for example)
• The rewrite rule only applies to other version of GHC besides 9.0.2
• The rule isn't actually a rule at all (for example, it could been commented out). This is possible because rule identification uses a regular expression and could have false positives.

These files can be skipped by adding an entry in filesToSkip in the Config.hs. Note that the filesToSkip entry matching is performed via an infix match, so facilitating skipping multiple files at once.

If a whole package seems to have a problem, you can also skip it via the packagesToSkip in Config.hs.

It's possible that required imports are not in scope. The importsForPackage function in Config.hs defines what imports should be added for each test file for a package. You can modify that function to add necessary imports and then re-generate the test suite.

Basically hlint should be happy and the test suites should pass:

make lint test

You can format (if you have stylish-haskell installed) with:

make format

• ghc-lib
• ghc-lib-parser