Share the Knowledge
RSS icon Home icon
  • Deploying your Django app with Fabric

    Posted on January 25th, 2014 webmaster No comments         

    I’ve been making quite a bit of improvements and changes to my Django app, GlucoseTracker, lately that the small amount of time I spent creating a deployment script using Fabric had already paid off.

    Fabric is basically a library written in Python that lets you run commands on remote servers (works locally as well) via SSH. It’s very easy to use and can save you a lot of time. It eliminates the need to connect to a remote server manually, and if you have a lot of servers to update, then the time savings really add up.

    Since I only have one server for my production environment, my setup and deployment process are very simple.  I use GitHub to store my code, a virtualenv environment where my code is deployed to, Gunicorn for the WSGI server managed by Supervisor, PostgreSQL for the database, and Nginx for the web server.

    My deployment process goes something like this:

    1. Run unit tests.
    2. Pull latest code from the master branch hosted on GitHub.
    3. Activate virtualenv.
    4. Run pip install against the requirements file (in case a new library was added or updated).
    5. Run South migrations for all apps (in case the there were changes in the database/table schemas).
    6. Restart Gunicorn with Supervisor.

    Here’s what my fabric.py file looks like:

    from fabric.api import local, env, cd, sudo
    
    env.hosts = ['www.glucosetracker.net']
    
    # The user account that owns the application files and folders.
    owner = 'glucosetracker'
    
    app_name = 'glucosetracker'
    app_directory = '/webapps/glucosetracker/glucose-tracker'
    settings_file = 'settings.production'
    
    
    def run_tests():
        local('coverage run manage.py test -v 2 --settings=settings.test')
    
    
    def deploy():
        """
        Deploy the app to the remote host.
    
        Steps:
            1. Change to the app's directory.
            2. Pull changes from master branch in git.
            3. Activate virtualenv.
            4. Run pip install using the requirements.txt file.
            5. Run South migrations.
            6. Restart gunicorn WSGI server using supervisor.
        """
        with cd(app_directory):
    
            sudo('git pull', user=owner)
    
            venv_command = 'source ../bin/activate'
    
            pip_command = 'pip install -r requirements.txt'
            sudo('%s && %s' % (venv_command, pip_command), user=owner)
    
            south_command = 'python glucosetracker/manage.py migrate --all ' \
                            '--settings=%s' % settings_file
            sudo('%s && %s' % (venv_command, south_command), user=owner)
    
            sudo('supervisorctl restart glucosetracker')
    

    To run this script, you first need to install the Fabric library:

    pip install fabric

    Then call the run_tests method by typing:

    fab run_tests

    Deploy with:

    fab deploy

    Make sure to run the fab command in the directory where the fabric.py file is located.

    For the run_tests() method, you’ll notice that I use the function local(). Since I don’t have a staging environment I just run my tests locally. I then deploy directly to my production server.

    Also note that the SSH session is not persistent, which is why you can see in my script that I combine the virtualenv activation command with the commands dependent on it being active.  I also run my app using a user account named glucosetracker which has limited access to the server to minimize damage in case someone figures out a way to run malicious code through my app.

    That’s pretty much it.  This is just a very simple example and you can do a lot more with it.  It takes very little time to get started, so even for small projects it’s definitely worth checking out.  It’s really nice to be able to make even just tiny changes to your app and have it deployed in seconds by running one simple command.

  • Display messages to your users with django-sticky-messages

    Posted on January 8th, 2014 webmaster No comments         

    I launched my Django app, GlucoseTracker, the beginning of this year and I’ve already added a few new things to it.  To notify users about these new features, I used this nice, simple app called django-sticky-messages.  It was written by a friend of mine for his Django app, Pool Manager.

    The app lets you set the message to display to the users in the Django admin.  You can set a start and end time when the message will be displayed.  If you use Twitter Bootstrap 3, you can have a dismissible message similar to the one shown below using the CSS classes alert alert-dismissable.

    django_sticky_messages_01

    The code in my dashboard template looks something like this:

    
    {% if sticky_message %}
    <div class="alert alert-info alert-dismissable">
      <button type="button" class="close" data-dismiss="alert"
        aria-hidden="true">&times;</button>
      {{ sticky_message.message|safe }}
    </div>
    {% endif %}
    
    

    You can change the alert-info class to change the color of the message. For example, alert-success will display a message with a green font and background.

    This is also great for notifying your users if you need to take down the server for a scheduled maintenance.

  • Launching a new Django project: GlucoseTracker

    Posted on January 1st, 2014 webmaster No comments         

    glucosetracker_home_01

    I finally launched a project I’ve been working on the last couple of months just in time for the new year.  It’s a web application for tracking blood glucose levels using all open source software: Python, Django, Twitter Bootstrap 3, PostgreSQL, Nginx, Gunicorn, and a bunch of others.

    I originally started this project over 2 years ago to teach myself how to use the Django web framework, but I lost interest at some point and abandoned it. Then I left my corporate job a few months later and hit the road to travel for the next 2 years.  Near the end of my trip, I decided to pick it up again and start from scratch, this time with more knowledge about Python and Django thanks to some work I’ve done for a friend of mine with his Django app.  I was also able to take advantage of other tools that became available during this time, such as the Bootstrap framework, which made designing a nice-looking website so much easier.

    The reason I chose this type of project is because I’m a diabetic myself, diagnosed with Type 1 when I was 13. There are a few web apps out there for this purpose, but most of them do a lot more than just tracking blood sugar.  They do things such as carb counting, food logging, recording blood pressure, and tracking exercise.  I’m too lazy to log the other stuff and just wanted something much simpler to track just blood glucose levels.  So I built my own, focusing on simplicity.

    One thing that I’ve always wanted in an app like this is to save me time entering data by having most fields already populated.  For example, most of the time, I just want to enter the number right after I check my blood sugar.  So on the app’s dashboard, I put a “Quick Add” form where the cursor autofocuses on the value field and the category automatically set to what makes sense based on the time of day.  All I have to do when I visit the site is type in the value and hit the enter key.  The date and time will automatically be set to the current date and time.

    You can really learn a lot working on your own project.  The best part is you get to decide everything: from your operating system of choice, programming language, database system, frameworks, IDE to your hosting provider.  And seeing your progress is very motivating.

    Before launching, I had just a simple page up and running where people can enter their email address so they can be notified when I finally launch GlucoseTracker.  I also have a link on that page to a live demo site that gets automatically updated as I push code to my GitHub repository.  I got about 9 people enter their email addresses there, which is not a lot but made me happy knowing that there are people interested in the app.

    I decided to make the project open-source so anyone can make improvements to it.  I’m a big fan of open source software, and while I don’t believe that all software should be free, everyone should at least contribute to the open source community if they can.  I also have a more selfish reason for making my project open source, of course, such as being able to show it off on my resume.  But it’s selfishness that benefits others.

    For the server, I decided to go with Rackspace as they offer a free 6 month trial and found the price-performance ratio very good.  I originally deployed my app to an Amazon AWS EC2 micro instance and found the performance quite low.

    Since it will pretty much cost me the same amount to host the app for 1 user (me) or 100 users, I made it completely free.  No ads, no catch.  Just knowing other people are finding value from what I built makes me feel good and motivated to keep improving it.

    Below are the main features of GlucoseTracker:

    • Simple, easy to use. Entering glucose data should be faster than finding a pen and paper and writing down the number. Form fields have pre-set values where it makes sense (such as the date, time, and category based on time of day).
    • Send glucose data via email. Email it to your doctor before your visit, no more carrying log books (and you’re saving trees)! You can choose whether to send the data as a CSV or PDF attachment.
    • Charts and graphs. Simple charts and graphs to see trends on how your diabetes is doing.
    • Data filtering. Filter your data by glucose range, category, tag name, and notes.
    • Tagging. An optional tag field to help further organize and make sense of your data. For example, it might be useful to add tags to a record such as the following: exercise, sick, insulin, fasting, etc.
    • A1C estimation. Estimate A1C based on data from the last 3 months.
    • Mobile friendly. The page layout automatically changes based on the size of your screen.  The site is still easy to use and view whether you’re on your desktop, tablet, or phone.

    Features I plan to add in the future:

    • CSV upload.  Most people who are already using other systems and thinking of switching would probably want their old data imported into the system.  I’m planning to add a simple form to upload a CSV (comma-separated) file to handle this.
    • Tools page. It might be helpful to provide a page for calculating A1C or converting units.
    • Tweet button.  I noticed a lot of people on Twitter like to tweet their blood glucose using the #bgnow hashtag.
    • Mobile app. I understand that it’s still not possible to find Internet connection everywhere, especially when you’re traveling.  I don’t have a data plan for my phone myself, as I still find it quite expensive and I’m not doing anything that requires me to be connected 24/7.  I want to create a simple mobile app which allows you to log your glucose offline and automatically syncs to the server when an Internet connection is available.
    • Fitbit integration.  I got a Fitbit Zip last month and absolutely love this little device and I wear it all the time.  It really motivates me to walk a lot more.  Fitbit has an API to retrieve the data and I’m planning to integrate it with GlucoseTracker so I could easily see how much exercise is affecting my blood sugar levels.

    There’s still a ton of work to do, but so far I’m very happy with how this project turned out. Smile

  • Using Highcharts with Django 1.6 and Twitter Bootstrap 3

    Posted on December 13th, 2013 webmaster No comments         

    GlucoseTracker.net-charts-001GlucoseTracker.net-charts-002

    I was looking for a library to create charts to use for my Django app about a week ago and a friend suggested to try Highcharts as he had heard good things about it from fellow Python/Django developers.

    Highcharts is a charting library purely written in JavaScript.  The charts look really pretty and interactive.  Using the library is quite easy, too, thanks to the well written documentation with lots of examples.  You can view a demo here and start playing with it right away.

    Below is an example of how I set it up for my project, using JSON for the data.  I use Twitter Bootstrap 3 also and the charts blend with it very nicely.

    Let’s try to recreate this line chart shown on my app’s dashboard:

    GlucoseTracker.net-charts-003

    Step 1: Make sure you have jQuery and the Highcharts JavaScript files referenced in your template.

    This is how I have mine setup:

    templates\base.html

    ...
            <!-- JavaScript -->
            <!-- Placed at the end of the document so the pages load faster -->
            {% compress js %}
            <script src="{{ STATIC_URL }}jquery/jquery.min.js"></script>
            {% block extrajs %}{% endblock %}
            {% endcompress %}
    ...
    

    templates\core\dashboard.html

    {% extends 'base.html' %}
    ...
    {% block extrajs %}
    <script src="{{ STATIC_URL }}highcharts/js/highcharts.js"></script>
    <script src="{{ STATIC_URL }}highcharts/js/modules/exporting.js"></script>
    {% endblock %}
    

    It’s important here to make sure that the jQuery js file is loaded first before Highcharts’ js.  The exporting.js file is optional.  Including this file will automatically put the extra option (the 3 small horizontal lines on the top right corner) on your charts to export the graphics to different image formats.

    Step 2: Prepare and serve your data from a view in JSON format.

    # glucoses/reports.py
    
    from datetime import datetime, timedelta
    
    import core
    
    from .models import Glucose
    
    class ChartData(object):
    ...
        @classmethod
        def get_avg_by_day(cls, user, days):
            now = datetime.now(tz=user.settings.time_zone).date()
    
            glucose_averages = Glucose.objects.avg_by_day(
                (now - timedelta(days=days)), now, user)
    
            data = {'dates': [], 'values': []}
            for avg in glucose_averages:
                data['dates'].append(avg['record_date'].strftime('%m/%d'))
                data['values'].append(core.utils.round_value(avg['avg_value']))
    
            return data
    
    # glucoses/views.py
    
    import json
    
    @login_required
    def chart_data_json(request):
        data = {}
        params = request.GET
    
        days = params.get('days', 0)
        name = params.get('name', '')
        if name == 'avg_by_day':
            data['chart_data'] = ChartData.get_avg_by_day(
                user=request.user, days=int(days))
    
        return HttpResponse(json.dumps(data), content_type='application/json')
    

    In this step, we have a reports.py module that generates the data in a format that Highcharts would accept.  We created a dictionary with 2 objects to hold the list of dates and values.

    In views.py, we read the parameters in a GET request, and based on those parameters we generate the response.

    Step 3: Create the HTML element where the chart will be loaded and the JavaScript to create the chart.

    ...
        <div class="panel panel-default">
            <div id="chart_panel" class="panel-body" 
                style="width:100%;height:314px"></div>
        </div>
    ...
    
    <script type="text/javascript">
    $(document).ready(function() {
    
        // Glucose Average by Day chart
         var avgByDayOptions = {
            chart: {
                renderTo: 'chart_panel',
                type: 'line',
            },
            legend: {enabled: false},
            title: {text: 'Average Glucose by Day'},
            subtitle: {text: 'Last 14 Days'},
            xAxis: {title: {text: null}, labels: {rotation: -45}},
            yAxis: {title: {text: null}},
            series: [{}],
        };
    
        var chartDataUrl = "{% url 'chart_data_json' %}?name=avg_by_day"
            + "&days=14";
        $.getJSON(chartDataUrl,
            function(data) {
                avgByDayOptions.xAxis.categories = data['chart_data']['dates'];
                avgByDayOptions.series[0].name = 'Avg Glucose (mg/dL)';
                avgByDayOptions.series[0].data = data['chart_data']['values'];
                var chart = new Highcharts.Chart(avgByDayOptions);
        });
    
    } );
    </script>
    

    As you can see here, we first created a panel with an id ‘chart_panel’ to reference where the chart will be loaded to.  The class names came from the Bootstrap CSS.  The width and height are optional, but if specified, Highcharts will inherit those attributes.

    Now on the JavaScript side, you’ll see that the options are pretty simple.  Highcharts has a lot more options as you can see here, but I just needed the basic ones.  For example, you can specify whether to enable the legend, labels for x and y axes, etc.

    When the template is loaded, the chart will load the data via a JSON call from the specified URL and creates the chart.  Notice how the URL parameters match the ones in the view.  The chart will be rendered to the panel that has the id ‘chart_panel’ as specified in the renderTo option.

    That’s pretty much it for the basics.  If you’d like to learn more you can check out the General Documentation page which gives a more detailed explanation.  You can also check out my project on GitHub for the complete implementation.

  • Grouping data by field with the Django ORM

    Posted on December 7th, 2013 webmaster No comments         

    I just finished creating some charts for my Django app using Highcharts and had to group some data for them. I thought it would be just as simple as calling a group_by() method but turned out it works a little differently but still pretty straightforward and well documented.

    Just to show an example (I’m using Django 1.6):

    # glucoses/models.py
    
    class Category(models.Model):
        name = models.CharField(unique=True, max_length=255)
    
    class Glucose(TimeStampedModel):
        user = models.ForeignKey(User)
        value = models.PositiveIntegerField() # in mg/dL
        category = models.ForeignKey('Category')
        ...
    

    If I want to group the data by category, take the average of the values , and count the number of records for that group, I would write something like this:

    from django.db.models import Avg, Count
    
    from glucoses.models import Glucose
    
    data = Glucose.objects.values('category__name')\
        .annotate(num_values=Count('value'), average=Avg('value'))\
        .order_by('-average')
    
    

    The output is a ValuesQuerySet object that looks something like this:

    [
        {'category__name': u'Bedtime', 'average': 154.62, 'num_values': 51},
        {'category__name': u'Dinner', 'average': 151.2, 'num_values': 60},
        {'category__name': u'Lunch', 'average': 144.73, 'num_values': 73},
        {'category__name': u'Breakfast', 'average': 142.17, 'num_values': 57}
    ]
    

    The key here for grouping by field is the values() clause. You can then use annotate() to do calculations for the grouped values.