Share the Knowledge
RSS icon Home icon
  • How to deploy encrypted copies of your SSL keys and other files with Ansible and OpenSSL

    Posted on April 5th, 2014 webmaster 2 comments         

    I’ve been working on fully automating server provisioning and deployment of my Django app, GlucoseTracker, the last couple of weeks with Ansible. Since I made this project open-source, I needed to make sure that passwords, secret keys, and other sensitive information are encrypted when I push my code to my repository.

    Of course, I have the option to not commit them to the repo, but I want to be able to build my entire infrastructure from code and maintain all configuration in one place, the git repo.

    Fortunately, Ansible has a command line tool called Ansible Vault (comes with the core package) that will allow you to encrypt your configuration files and decrypt them during deployment by passing in the password or password file in the command.  This is mainly useful for encrypting your environment variables files that contain the passwords/keys for your application.

    For example, in my Django settings, instead of assigning the values directly in the settings file, I do something like this:

    SECRET_KEY = os.environ['DJANGO_SECRET_KEY']

    Django would then read the value from the server’s environment variables.  In my case, since I use virtualenv, I have a postactivate script that sets the environment variables for that virtualenv.

    Since I want Ansible to fully automate my server configuration and store all the information that I need to do so in a git repo, I have to encrypt the variables that my app use in production.  For example, I have a file called production in the env_vars folder of my repo that the playbook will use.  I encrypt this file with a password using Ansible Vault and when running the playbook I decrypt it by passing in the password in the command argument.  If you use CI tools like Jenkins you can then just set this password in Jenkins (perhaps as an environment variable) so you won’t have to type it in manually.

    If I need to make a change in the vars file, I can simply just type in:

    ansible-vault edit env_vars/production

    This will prompt me for the password and display the decrypted values in vim for me where I can make my changes and save it back in the encrypted version.  This is a nice option if you just need to make small changes as you won’t forget to encrypt the file again after decrypting it.

    Here’s what an Ansible Vault encrypted vars file will look like:


    What about files that get copied to the server such as private keys for SSL certificates?

    This is where I had to do something extra as the Ansible copy module will copy these files as they’re stored in the repository (i.e. the encrypted version).  To get around this, I simply used OpenSSL to encrypt these files using symmetric encryption and set the password to decrypt it in my vars file (which Ansible Vault will encrypt).

    To encrypt a file with OpenSSL using AES 256 encryption:

    openssl aes-256-cbc -salt -a -e -in ssl_signed/unencrypted.key -out ssl_signed/encrypted.key -k MysupasecuresecretPasswordZ.x!!

    To decrypt an AES 256 encrypted file with OpenSSL:

    openssl aes-256-cbc -salt -a -d -in ssl_signed/encrypted.key -out ssl_signed/unencrypted.key -k MysupasecuresecretPasswordZ.x!!

    Example vars file:

    # Nginx settings.
    ssl_src_dir: ssl_signed
    ssl_dest_dir: /etc/ssl
    ssl_key_password: MysupasecuresecretPasswordZ.x!!

    I have a task in my playbook that would copy my SSL keys to the remote server and run the OpenSSL command to decrypt it (using the password from the ssl_key_password variable in my vars file):

    - name: Copy the SSL cert and key to the remote server
      copy: src={{ ssl_src_dir }}/ dest={{ ssl_dest_dir }}
    - name: Decrypt the SSL key
      command: openssl aes-256-cbc -salt -a -d -in {{ ssl_dest_dir }}/nginx.key
               -out {{ ssl_dest_dir }}/decrypted.key -k {{ ssl_key_password }}
               creates={{ ssl_dest_dir }}/decrypted.key
    - name: Rename the decrypted SSL key
      command: mv {{ ssl_dest_dir }}/decrypted.key {{ ssl_dest_dir }}/nginx.key
               removes={{ ssl_dest_dir }}/decrypted.key

    Now let’s run the production playbook:

    ansible-playbook -i inventory/production –private-key=/aws-keys/ec2-glucosetracker.pem –vault-password-file=~/ansible/decryption_password -vvvv production.yml

    This is just one example and this same simple concept can be applied to different scenarios.  Just to summarize the steps:

    1. Encrypt your files with OpenSSL using symmetric encryption.
    2. Assign the decryption password to a variable in your Ansible vars file.
    3. Encrypt your vars file using Ansible Vault.
    4. Create a task in your playbook to decrypt the encrypted files using OpenSSL and the password in the encrypted vars file.
    5. Run your Ansible playbook, passing in the Ansible Vault password in the command or specifying the file where the password is stored.

    View my entire playbook here:

  • Django Tip: How to configure Gunicorn to auto-reload your code during development

    Posted on March 30th, 2014 webmaster No comments         

    I just finished fully automating my entire server stack for my Django app with Ansible and Vagrant (using VirtualBox).  One of the reasons I did this is to make my development environment as close to production as possible to hopefully eliminate any surprises when deploying to production.  It also allows me to setup a development environment very quickly as I won’t have to deal with manual installation and configuration of different packages.  In a team environment, the benefit of doing this multiplies.

    This is basically my process:

    1. Type in ‘vagrant up’ to create or start the VirtualBox virtual machine.

    2. I have my virtual machine configured via Vagrant to share my local code (which is located in my Dropbox folder) with the virtual machine.

    3. I make a change to my code, then I open my web browser and enter my virtual machine’s IP which is statically set to  The browser shows my changes.

    What I want Gunicorn to do is similar to what the Django runserver does: automatically reload the application server when the code changes.

    There are different ways to approach this, such as using a package called watchdog to watch the file changes and then restart Gunicorn.  But it turned out there’s an even simpler way to do this with Gunicorn by setting the max_requests setting to 1 (full list of settings).  When calling Gunicorn simply add this option (note that it’s 2 dashes in the beginning):

    –max-requests 1

    What this will basically do is tell Gunicorn to restart the process for every request, which would reload your code.  It won’t know whether your code changed or not, it will always reload it.  For production, this is probably not a good idea, but during development it’s a nice simple trick and you won’t really see a difference in performance as you’d be the only user.

    Here’s the shell script that I use to start my Gunicorn process (note that I use Nginx to communicate with Gunicorn via a socket file, also note that I have variables set here which Ansible replaces with the actual values):

    NAME="{{ application_name }}"
    DJANGODIR={{ application_path }}
    SOCKFILE={{ virtualenv_path }}/run/gunicorn.sock
    USER={{ gunicorn_user }}
    GROUP={{ gunicorn_group }}
    # Set this to 0 for unlimited requests. During development, you might want
    # to set this to 1 to automatically restart the process on each request
    # (i.e. your code will be reloaded on every request).
    MAX_REQUESTS={{ gunicorn_max_requests }}
    echo "Starting $NAME as `whoami`"
    # Activate the virtual environment.
    source ../../bin/activate
    source ../../bin/postactivate
    # Create the run directory if it doesn't exist.
    RUNDIR=$(dirname $SOCKFILE)
    test -d $RUNDIR || mkdir -p $RUNDIR
    # Programs meant to be run under supervisor should not daemonize themselves
    # (do not use --daemon).
    exec python run_gunicorn \
    		--settings={{ django_settings_file }} \
     		--name $NAME \
      		--workers $NUM_WORKERS \
      		--max-requests $MAX_REQUESTS \
      		--user=$USER --group=$GROUP \
      		--log-level=debug \
  • Deploying your Django app with Fabric

    Posted on January 25th, 2014 webmaster No comments         

    I’ve been making quite a bit of improvements and changes to my Django app, GlucoseTracker, lately that the small amount of time I spent creating a deployment script using Fabric had already paid off.

    Fabric is basically a library written in Python that lets you run commands on remote servers (works locally as well) via SSH. It’s very easy to use and can save you a lot of time. It eliminates the need to connect to a remote server manually, and if you have a lot of servers to update, then the time savings really add up.

    Since I only have one server for my production environment, my setup and deployment process are very simple.  I use GitHub to store my code, a virtualenv environment where my code is deployed to, Gunicorn for the WSGI server managed by Supervisor, PostgreSQL for the database, and Nginx for the web server.

    My deployment process goes something like this:

    1. Run unit tests.
    2. Pull latest code from the master branch hosted on GitHub.
    3. Activate virtualenv.
    4. Run pip install against the requirements file (in case a new library was added or updated).
    5. Run South migrations for all apps (in case the there were changes in the database/table schemas).
    6. Restart Gunicorn with Supervisor.

    Here’s what my file looks like:

    from fabric.api import local, env, cd, sudo
    env.hosts = ['']
    # The user account that owns the application files and folders.
    owner = 'glucosetracker'
    app_name = 'glucosetracker'
    app_directory = '/webapps/glucosetracker/glucose-tracker'
    settings_file = 'settings.production'
    def run_tests():
        local('coverage run test -v 2 --settings=settings.test')
    def deploy():
        Deploy the app to the remote host.
            1. Change to the app's directory.
            2. Pull changes from master branch in git.
            3. Activate virtualenv.
            4. Run pip install using the requirements.txt file.
            5. Run South migrations.
            6. Restart gunicorn WSGI server using supervisor.
        with cd(app_directory):
            sudo('git pull', user=owner)
            venv_command = 'source ../bin/activate'
            pip_command = 'pip install -r requirements.txt'
            sudo('%s && %s' % (venv_command, pip_command), user=owner)
            south_command = 'python glucosetracker/ migrate --all ' \
                            '--settings=%s' % settings_file
            sudo('%s && %s' % (venv_command, south_command), user=owner)
            sudo('supervisorctl restart glucosetracker')

    To run this script, you first need to install the Fabric library:

    pip install fabric

    Then call the run_tests method by typing:

    fab run_tests

    Deploy with:

    fab deploy

    Make sure to run the fab command in the directory where the file is located.

    For the run_tests() method, you’ll notice that I use the function local(). Since I don’t have a staging environment I just run my tests locally. I then deploy directly to my production server.

    Also note that the SSH session is not persistent, which is why you can see in my script that I combine the virtualenv activation command with the commands dependent on it being active.  I also run my app using a user account named glucosetracker which has limited access to the server to minimize damage in case someone figures out a way to run malicious code through my app.

    That’s pretty much it.  This is just a very simple example and you can do a lot more with it.  It takes very little time to get started, so even for small projects it’s definitely worth checking out.  It’s really nice to be able to make even just tiny changes to your app and have it deployed in seconds by running one simple command.

  • Display messages to your users with django-sticky-messages

    Posted on January 8th, 2014 webmaster No comments         

    I launched my Django app, GlucoseTracker, the beginning of this year and I’ve already added a few new things to it.  To notify users about these new features, I used this nice, simple app called django-sticky-messages.  It was written by a friend of mine for his Django app, Pool Manager.

    The app lets you set the message to display to the users in the Django admin.  You can set a start and end time when the message will be displayed.  If you use Twitter Bootstrap 3, you can have a dismissible message similar to the one shown below using the CSS classes alert alert-dismissable.


    The code in my dashboard template looks something like this:

    {% if sticky_message %}
    <div class="alert alert-info alert-dismissable">
      <button type="button" class="close" data-dismiss="alert"
      {{ sticky_message.message|safe }}
    {% endif %}

    You can change the alert-info class to change the color of the message. For example, alert-success will display a message with a green font and background.

    This is also great for notifying your users if you need to take down the server for a scheduled maintenance.

  • Launching a new Django project: GlucoseTracker

    Posted on January 1st, 2014 webmaster No comments         


    I finally launched a project I’ve been working on the last couple of months just in time for the new year.  It’s a web application for tracking blood glucose levels using all open source software: Python, Django, Twitter Bootstrap 3, PostgreSQL, Nginx, Gunicorn, and a bunch of others.

    I originally started this project over 2 years ago to teach myself how to use the Django web framework, but I lost interest at some point and abandoned it. Then I left my corporate job a few months later and hit the road to travel for the next 2 years.  Near the end of my trip, I decided to pick it up again and start from scratch, this time with more knowledge about Python and Django thanks to some work I’ve done for a friend of mine with his Django app.  I was also able to take advantage of other tools that became available during this time, such as the Bootstrap framework, which made designing a nice-looking website so much easier.

    The reason I chose this type of project is because I’m a diabetic myself, diagnosed with Type 1 when I was 13. There are a few web apps out there for this purpose, but most of them do a lot more than just tracking blood sugar.  They do things such as carb counting, food logging, recording blood pressure, and tracking exercise.  I’m too lazy to log the other stuff and just wanted something much simpler to track just blood glucose levels.  So I built my own, focusing on simplicity.

    One thing that I’ve always wanted in an app like this is to save me time entering data by having most fields already populated.  For example, most of the time, I just want to enter the number right after I check my blood sugar.  So on the app’s dashboard, I put a “Quick Add” form where the cursor autofocuses on the value field and the category automatically set to what makes sense based on the time of day.  All I have to do when I visit the site is type in the value and hit the enter key.  The date and time will automatically be set to the current date and time.

    You can really learn a lot working on your own project.  The best part is you get to decide everything: from your operating system of choice, programming language, database system, frameworks, IDE to your hosting provider.  And seeing your progress is very motivating.

    Before launching, I had just a simple page up and running where people can enter their email address so they can be notified when I finally launch GlucoseTracker.  I also have a link on that page to a live demo site that gets automatically updated as I push code to my GitHub repository.  I got about 9 people enter their email addresses there, which is not a lot but made me happy knowing that there are people interested in the app.

    I decided to make the project open-source so anyone can make improvements to it.  I’m a big fan of open source software, and while I don’t believe that all software should be free, everyone should at least contribute to the open source community if they can.  I also have a more selfish reason for making my project open source, of course, such as being able to show it off on my resume.  But it’s selfishness that benefits others.

    For the server, I decided to go with Rackspace as they offer a free 6 month trial and found the price-performance ratio very good.  I originally deployed my app to an Amazon AWS EC2 micro instance and found the performance quite low.

    Since it will pretty much cost me the same amount to host the app for 1 user (me) or 100 users, I made it completely free.  No ads, no catch.  Just knowing other people are finding value from what I built makes me feel good and motivated to keep improving it.

    Below are the main features of GlucoseTracker:

    • Simple, easy to use. Entering glucose data should be faster than finding a pen and paper and writing down the number. Form fields have pre-set values where it makes sense (such as the date, time, and category based on time of day).
    • Send glucose data via email. Email it to your doctor before your visit, no more carrying log books (and you’re saving trees)! You can choose whether to send the data as a CSV or PDF attachment.
    • Charts and graphs. Simple charts and graphs to see trends on how your diabetes is doing.
    • Data filtering. Filter your data by glucose range, category, tag name, and notes.
    • Tagging. An optional tag field to help further organize and make sense of your data. For example, it might be useful to add tags to a record such as the following: exercise, sick, insulin, fasting, etc.
    • A1C estimation. Estimate A1C based on data from the last 3 months.
    • Mobile friendly. The page layout automatically changes based on the size of your screen.  The site is still easy to use and view whether you’re on your desktop, tablet, or phone.

    Features I plan to add in the future:

    • CSV upload.  Most people who are already using other systems and thinking of switching would probably want their old data imported into the system.  I’m planning to add a simple form to upload a CSV (comma-separated) file to handle this.
    • Tools page. It might be helpful to provide a page for calculating A1C or converting units.
    • Tweet button.  I noticed a lot of people on Twitter like to tweet their blood glucose using the #bgnow hashtag.
    • Mobile app. I understand that it’s still not possible to find Internet connection everywhere, especially when you’re traveling.  I don’t have a data plan for my phone myself, as I still find it quite expensive and I’m not doing anything that requires me to be connected 24/7.  I want to create a simple mobile app which allows you to log your glucose offline and automatically syncs to the server when an Internet connection is available.
    • Fitbit integration.  I got a Fitbit Zip last month and absolutely love this little device and I wear it all the time.  It really motivates me to walk a lot more.  Fitbit has an API to retrieve the data and I’m planning to integrate it with GlucoseTracker so I could easily see how much exercise is affecting my blood sugar levels.

    There’s still a ton of work to do, but so far I’m very happy with how this project turned out. Smile