Share the Knowledge
RSS icon Home icon
  • Using Highcharts with Django 1.6 and Twitter Bootstrap 3

    Posted on December 13th, 2013 webmaster No comments         

    GlucoseTracker.net-charts-001GlucoseTracker.net-charts-002

    I was looking for a library to create charts to use for my Django app about a week ago and a friend suggested to try Highcharts as he had heard good things about it from fellow Python/Django developers.

    Highcharts is a charting library purely written in JavaScript.  The charts look really pretty and interactive.  Using the library is quite easy, too, thanks to the well written documentation with lots of examples.  You can view a demo here and start playing with it right away.

    Below is an example of how I set it up for my project, using JSON for the data.  I use Twitter Bootstrap 3 also and the charts blend with it very nicely.

    Let’s try to recreate this line chart shown on my app’s dashboard:

    GlucoseTracker.net-charts-003

    Step 1: Make sure you have jQuery and the Highcharts JavaScript files referenced in your template.

    This is how I have mine setup:

    templates\base.html

    ...
            <!-- JavaScript -->
            <!-- Placed at the end of the document so the pages load faster -->
            {% compress js %}
            <script src="{{ STATIC_URL }}jquery/jquery.min.js"></script>
            {% block extrajs %}{% endblock %}
            {% endcompress %}
    ...
    

    templates\core\dashboard.html

    {% extends 'base.html' %}
    ...
    {% block extrajs %}
    <script src="{{ STATIC_URL }}highcharts/js/highcharts.js"></script>
    <script src="{{ STATIC_URL }}highcharts/js/modules/exporting.js"></script>
    {% endblock %}
    

    It’s important here to make sure that the jQuery js file is loaded first before Highcharts’ js.  The exporting.js file is optional.  Including this file will automatically put the extra option (the 3 small horizontal lines on the top right corner) on your charts to export the graphics to different image formats.

    Step 2: Prepare and serve your data from a view in JSON format.

    # glucoses/reports.py
    
    from datetime import datetime, timedelta
    
    import core
    
    from .models import Glucose
    
    class ChartData(object):
    ...
        @classmethod
        def get_avg_by_day(cls, user, days):
            now = datetime.now(tz=user.settings.time_zone).date()
    
            glucose_averages = Glucose.objects.avg_by_day(
                (now - timedelta(days=days)), now, user)
    
            data = {'dates': [], 'values': []}
            for avg in glucose_averages:
                data['dates'].append(avg['record_date'].strftime('%m/%d'))
                data['values'].append(core.utils.round_value(avg['avg_value']))
    
            return data
    
    # glucoses/views.py
    
    import json
    
    @login_required
    def chart_data_json(request):
        data = {}
        params = request.GET
    
        days = params.get('days', 0)
        name = params.get('name', '')
        if name == 'avg_by_day':
            data['chart_data'] = ChartData.get_avg_by_day(
                user=request.user, days=int(days))
    
        return HttpResponse(json.dumps(data), content_type='application/json')
    

    In this step, we have a reports.py module that generates the data in a format that Highcharts would accept.  We created a dictionary with 2 objects to hold the list of dates and values.

    In views.py, we read the parameters in a GET request, and based on those parameters we generate the response.

    Step 3: Create the HTML element where the chart will be loaded and the JavaScript to create the chart.

    ...
        <div class="panel panel-default">
            <div id="chart_panel" class="panel-body" 
                style="width:100%;height:314px"></div>
        </div>
    ...
    
    <script type="text/javascript">
    $(document).ready(function() {
    
        // Glucose Average by Day chart
         var avgByDayOptions = {
            chart: {
                renderTo: 'chart_panel',
                type: 'line',
            },
            legend: {enabled: false},
            title: {text: 'Average Glucose by Day'},
            subtitle: {text: 'Last 14 Days'},
            xAxis: {title: {text: null}, labels: {rotation: -45}},
            yAxis: {title: {text: null}},
            series: [{}],
        };
    
        var chartDataUrl = "{% url 'chart_data_json' %}?name=avg_by_day"
            + "&days=14";
        $.getJSON(chartDataUrl,
            function(data) {
                avgByDayOptions.xAxis.categories = data['chart_data']['dates'];
                avgByDayOptions.series[0].name = 'Avg Glucose (mg/dL)';
                avgByDayOptions.series[0].data = data['chart_data']['values'];
                var chart = new Highcharts.Chart(avgByDayOptions);
        });
    
    } );
    </script>
    

    As you can see here, we first created a panel with an id ‘chart_panel’ to reference where the chart will be loaded to.  The class names came from the Bootstrap CSS.  The width and height are optional, but if specified, Highcharts will inherit those attributes.

    Now on the JavaScript side, you’ll see that the options are pretty simple.  Highcharts has a lot more options as you can see here, but I just needed the basic ones.  For example, you can specify whether to enable the legend, labels for x and y axes, etc.

    When the template is loaded, the chart will load the data via a JSON call from the specified URL and creates the chart.  Notice how the URL parameters match the ones in the view.  The chart will be rendered to the panel that has the id ‘chart_panel’ as specified in the renderTo option.

    That’s pretty much it for the basics.  If you’d like to learn more you can check out the General Documentation page which gives a more detailed explanation.  You can also check out my project on GitHub for the complete implementation.

  • Grouping data by field with the Django ORM

    Posted on December 7th, 2013 webmaster No comments         

    I just finished creating some charts for my Django app using Highcharts and had to group some data for them. I thought it would be just as simple as calling a group_by() method but turned out it works a little differently but still pretty straightforward and well documented.

    Just to show an example (I’m using Django 1.6):

    # glucoses/models.py
    
    class Category(models.Model):
        name = models.CharField(unique=True, max_length=255)
    
    class Glucose(TimeStampedModel):
        user = models.ForeignKey(User)
        value = models.PositiveIntegerField() # in mg/dL
        category = models.ForeignKey('Category')
        ...
    

    If I want to group the data by category, take the average of the values , and count the number of records for that group, I would write something like this:

    from django.db.models import Avg, Count
    
    from glucoses.models import Glucose
    
    data = Glucose.objects.values('category__name')\
        .annotate(num_values=Count('value'), average=Avg('value'))\
        .order_by('-average')
    
    

    The output is a ValuesQuerySet object that looks something like this:

    [
        {'category__name': u'Bedtime', 'average': 154.62, 'num_values': 51},
        {'category__name': u'Dinner', 'average': 151.2, 'num_values': 60},
        {'category__name': u'Lunch', 'average': 144.73, 'num_values': 73},
        {'category__name': u'Breakfast', 'average': 142.17, 'num_values': 57}
    ]
    

    The key here for grouping by field is the values() clause. You can then use annotate() to do calculations for the grouped values.

  • Auto-deploy your Django app to Heroku with Travis CI on git push

    Posted on December 3rd, 2013 webmaster No comments         

    I was setting up a demo site for my latest open-source project, GlucoseTracker, a few weeks back and decided to run it on Heroku.  The free tier (called ‘Hobby Dev’) allows 10,000 rows in a PostgreSQL database and 20 concurrent connections, which is more than good enough for me at this point.

    I thought about using Jenkins CI to do the deployment to Heroku initially, but as I was installing it on my Rackspace VPS, I kept running into issues where the service would stop responding.  It turned out that I didn’t have enough memory on my VPS as it only has 512MB of RAM and I’m running 2 WordPress sites on it with MySQL, a PHP app, a Django app with Gunicorn and PostgreSQL, and Apache2.  Jenkins alone would probably need most of that 512MB of RAM.

    So I decided to use Travis CI instead as it’s integrated with GitHub where I have my source code hosted. It’s free and setting it up was very easy.  Below are the steps.

    1. Log-in to your GitHub account, then go to https://travis-ci.org and click the “Sign in with GitHub” link on the top right.

    2.  Select the repository you want to use use with Travis and enable the Travis “service hook” for that repository.

    3.  Create a .travis.yml in the root of your repository and insert this line:

    
    deploy:
    
    

    4.  Install the Travis command line tools (requires Ruby, more info here):

    gem install travis -v 1.6.3 –no-rdoc –no-ri

    5.  Assuming that you have the Heroku toolbelt CLI already installed, run this command to automatically insert other parts of the configuration for you including the encrypted Heroku auth token:

    travis setup heroku

    6.  Your config should now look something like this:

    deploy:
      provider: heroku
      api_key:
        secure: very_long_encrypted_token
      app: glucosetracker-demo
      on:
        repo: jcalazan/glucose-tracker
    

    7.  Add additional settings.  For example, mine looks something like this:

    language: python
    deploy:
      provider: heroku
      buildpack: python
      api_key:
        secure: very_long_encrypted_token
      strategy: git
      app: glucosetracker-demo
      on:
        repo: jcalazan/glucose-tracker
      run:
        - "python glucosetracker/manage.py syncdb --noinput --settings=settings.heroku"
        - "python glucosetracker/manage.py migrate --all --settings=settings.heroku"
        - "python glucosetracker/manage.py load_random_glucose_data jsmith --settings=settings.heroku"
        - restart
    script: "coverage run --source=. glucosetracker/manage.py test -v 2"
    

    There are 2 important settings here that the Travis documentation for Heroku deployment didn’t mention: language and script.

    The language: setting should be set to python so the system knows it’s deploying a Python application and use the proper commands to prepare the environment.

    The script: can be just a simple command or something that will execute a script file.  The main thing is the command/script must return an exit code of 0 to tell the system that it ran successfully.  A good script to run here is your test or build script as it’s executed before the deployment process starts.  If you don’t have anything to run, simply enter any command here like “pwd” as it’s required to have something here.

    The run: portion is optional and it’s used for executing commands after deployment.  In my case, I want to run syncdb again, run South migrations for all apps in case there are changes in the schema, and run a custom management command to populate my database with dummy data.

    That’s pretty much it for a basic setup.  Travis CI will automatically be notified by GitHub whenever you push new code to the master branch and deployment will automatically start.  You will also get email notifications from Travis if a build fails.

    References

  • Loading dummy data for your Django app using management commands

    Posted on November 23rd, 2013 webmaster No comments         

    I’m at a point in my project now, GlucoseTracker, a web application for keeping tracking of blood sugar levels, where I’m starting to work on reporting and statistics.  I need some dummy data automatically loaded in the database to work on this part and I need to refresh the data every few days or so.

    It would be too much work to do this manually so I wrote a simple script that would be ran as a management command in Django.  Using management commands is a good way to run the scripts that depend on the Django ORM and other Django features as the settings and paths will already be set for you.

    To do this, all you have to do is a create a management.commands package in your application package and place your Python script there.  For example, I have a script called load_random_glucose_data.py, so I’d put it here in my project:

    glucosetracker\

    —-glucoses\

    ——–management\

    ————commands\

    —————-load_random_glucose_data.py

    And here’s the content of my script:

    import sys
    import random
    from datetime import date, timedelta
    
    from django.core.management.base import BaseCommand
    from django.db.models.base import ObjectDoesNotExist
    from django.contrib.auth.models import User
    
    from core.models import UserSettings
    
    from ...models import Glucose, Category
    from ...tests.factories import GlucoseFactory
    
    class Command(BaseCommand):
        args = 'username...'
        help = 'Populate glucose table with random dummy data.'
    
        def handle(self, *args, **options):
            if len(args) == 0:
                sys.stdout.write('You must specify a username.\n')
                sys.exit(1)
    
            try:
                user = User.objects.get(username=args[0])
            except ObjectDoesNotExist:
                user = User.objects.create(username=args[0])
                user.first_name = 'John'
                user.last_name = 'Smith'
                user.email = 'test@glucosetracker.net'
                user.set_password('demo')
                user.save()
    
                # Create an entry for the User Settings.
                UserSettings.objects.create(user=user).save()
    
            # Delete existing data.
            Glucose.objects.filter(user=user).delete()
    
            end_date = date.today()
            start_date = end_date - timedelta(days=90)
            for i in self.get_date_list(start_date, end_date):
                for _ in range(4):
                    GlucoseFactory(
                        user=user,
                        category=random.choice(Category.objects.all()),
                        record_date=i
                    )
    
        @classmethod
        def get_date_list(cls, start, end):
            delta = end - start
            return [(start + timedelta(days=i)) for i in range(delta.days+1)]
    

    To run this script, I would type:

    python manage.py load_random_glucose_data jsmith –settings=settings.local

    The script basically takes a username as an argument and creates that user if it doesn’t exist.  It will then delete all glucose data records for that user and re-populate it with dummy data for the past 90 days, 4 records per day, with random values and category.

    I created a factory class called GlucoseFactory using Factory Boy inside my test package which takes care of the creation of glucose objects.

    Now in my app, I can view these randomly generated data and it gives me an idea of what a user might see after he had entered enough data.  For example:

    glucosetracker_01

    The app displays low and high blood glucose levels in red, the target range in green, and the rest in blue.  Later on, I will have a page or dashboard that will give the user a summary of his/her stats, like the last average glucose level the past 7 days, how may highs and lows were there, how many are within his/her target range, average value for each category, etc.  I need to refresh the data from time to time because they’re date-sensitive and being able to just run one simple command to do this is a big time saver.

    I also have my project set-up on GitHub to use Travis for continuous integration.  So every time I push new code to the master branch it would auto-deploy the new code to my Heroku instance which I use to demo my app and the command to load the data also runs as part of the deployment.

    If you have an app that relies on having fresh data loaded regularly, then having a command to auto-generate them would really help and save a lot of time during development.

  • Loading initial data for your Django app using fixtures and South

    Posted on November 13th, 2013 webmaster No comments         

    I’ve been thinking about deployment for my app these last few days and one of the things I needed to be able to do is automatically load initial data to one of the tables.  If you’re already using South (which you should be unless you have a very, very simple project), you can easily include this in a data migration by loading in a fixture.

    You can accomplish this in a few simple steps:

    1.  Export the data from the table to a JSON file, you can use a Django dumpdata command to do this.  For example:

    python manage.py dumpdata glucoses.category –settings=settings.local > fixtures/initial_glucoses_category_data.json

    2.  Create a migration. For example:

    python manage.py datamigration glucoses load_categoryfixture –settings=settings.local

    3.  Edit the migration file so the forwards method looks something like this:

    class Migration(DataMigration):
    
        def forwards(self, orm):
            from django.core.management import call_command
            call_command("loaddata",
                         "fixtures/initial_glucoses_category_data.json")
    

    That’s it, you’re done!  Now when you deploy your app to a new environment with a new database and run the South migration, the table will be populated with your fixture automatically at the correct time.

    More information here: http://south.readthedocs.org/en/latest/fixtures.html