I spent quite a bit of time last week creating a custom Heroku buildpack for a Python library I wanted to use on Heroku. It's a library for converting HTML to PDF (Weasyprint). It has quite a bit of dependencies on other packages, such as a few -dev packages. Unfortunately, Heroku doesn't have a lot of these packages installed by default.

The solution is to create a custom Heroku buildpack. This was my first attempt in creating one and I ran into a lot of issues. I hope the following tips will help others who are new to buildpacks.

Don't start from scratch

Someone said “Good Artists Borrow, Great Artists Steal." So go to github, search if someone already created a buildpack for a library that you need or something similar to it and fork it. Take a quick look at their script to see how it works, then modify it to suit your needs.

One of the libraries I needed was cairo and someone actually already created a buildpack for it. So I forked it to make some modifications and added the additional libraries I needed.

Use 'Vulcan' to compile your libraries

Libraries you compiled on your local machine or compiled by someone else might not run properly on your Heroku environment. So it's recommended that you use vulcan, a build server designed for Heroku, to compile all the additional libraries that you will need to run your app on Heroku.

You basically just run a command to send your source files to a remote server, compile them, and send back the binary file. You can then host the binary somewhere such as Amazon's S3 service, which your custom buildpack can then use to download the file from during deployment.

Here's an example command using vulcan:

vulcan build -v -s libffi-3.0.9/ -p /tmp -c "./configure --prefix=/tmp && make install"

Make sure that your Heroku account has been verified

One of my first issues was actually trying to use vulcan and received this unknown error:

Uploading source package… !! Unknown error, no build output given

It turned out I needed the Cloudant add-on, which vulcan actually tries to add automatically but since I didn't provide my credit card information, it failed. Heroku requires this information to verify your account before they'll let you add any add-ons.

Check your paths/environment variables

This gave me the most trouble. I would spend hours trying to figure out what was wrong and a lot of times it was just an issue with my path variables. I would have typos, not setting them properly, or missing environment variables.

Look out especially for LIBRARY_PATH, LD_LIBRARY_PATH, and PKG_CONFIG_PATH variables.

Run Heroku's bash shell to troubleshoot issues

This is really useful when troubleshooting. I didn't know you could do this when I was starting out and could've saved me a lot of time.

To access the bash shell of the remote server, type this command:

heroku run bash

You can do quite a bit here: check the directory structure, logs, environment variables, etc. Note that the shell is read-only, so any changes you make here gets wiped out when you exit.

One at a time

It was tempting to just add all the extra libraries I needed to the script in one shot. But from previous experiences, I know this will bite me in the ass later and will make things much harder to troubleshoot.

So I recommend to start with one library first, test it, add the second one, test again, and so on.

We actually ended up not using Weasyprint as we found the dependencies to be a little too much and decided to look into ReportLab instead, but it was still a good experience as this gave us a better idea on how Heroku works.

If you have additional tips, please let me know by commenting below. I'd love to hear them!