Video - Building a web crawler

Underjord is a tiny, wholesome team doing Elixir consulting and contract work. If you like the writing you should really try the code. See our services for more information.

Had some great fun building on this webcrawler on the stream. Plan to continue with this in coming livestreams, so watch this to get up to speed on what we are doing.

We grab an HTTP client, and HTML parser and then we go hog wild spawning Task processes and fetching web pages.

The project repo I built up during the stream is on Github as lawik/caterpillar and the video is below.

Also available on the YouTube channel.

Enjoy the things I do? Sign up for the newsletter, I write more about Elixir, my projects and developer experiences there.

Want more voice? I talk about Elixir on Beam Radio and more about programming in general on Regular Programming.

If you want to discuss it with me, you can find me via email as lars@underjord.io or on Twitter where I’m @lawik.

Underjord is a 4 people team doing Elixir consulting and contract work. If you like the writing you should really try the code. See our services for more information.

Note: Or try the videos on the YouTube channel.