Hypertext grabber...
Snapshot and sanitize webpages just-as-they-are in a headless Firefox.
Using the Freeze-dry javascript library for self-contained HTML file snapshots.
- Debug mode
- HTTP API
- Make use of freeze-dry's customisation abilities
- Firefox (in your path)
- Go (to compile)
- Earthly or Nix (for building)
Build & Run Manually
$ mkdir -p js/dist
$ wget -O js/dist/freeze-dry.umd.js https://git.callpipe.com/dvn/hyperwalker/-/jobs/16551/artifacts/raw/build/freeze-dry/freeze-dry.umd.js
$ go build
$ ./HyperWalker -url https://en.wikipedia.org/wiki/Special:RandomUsing Earthly
$ earthly config global.disable_analytics true
$ earthly config global.disable_log_sharing true
$ earthly +build
$ ./build/hyperwalker -url https://en.wikipedia.org/wiki/Special:RandomUsing Nix Flake
Nix super-quickstart:
$ nix run sourcehut:~dvn/HyperWalker -- -url https://en.wikipedia.org/wiki/Special:RandomMore involved:
$ nix build
$ mkdir -p $HOME/.hyperwalker/logs
$ ./result/HyperWalker -url https://en.wikipedia.org/wiki/Special:Random
$ # or to build and run directly:
$ mkdir -p $HOME/.hyperwalker/logs
$ nix run . -- -url https://en.wikipedia.org/wiki/Special:RandomRun Using Docker
$ docker run --rm registry.git.callpipe.com/dvn/hyperwalker:latest /hyperwalker/hyperwalker -url https://en.wikipedia.org/wiki/Special:RandomWarning: This runs Firefox in a docker container and takes several minutes the first time around.
$ earthly +firefox-image
$ earthly +application-test