Wagtail and Azure Storage Blob Containers

So recently I’ve been working on a project to move old legacy sites into Wagtail and we’ve set this Wagtail site up on the Azure Cloud using Azure Web Apps for Linux with a custom Docker Container. Ideally we wanted images uploaded by the user and our static files stored separately so we used Azure Storage and setup two containers. One container for static files, and one for images and docs uploaded by the users through Wagtail.

It seems to be working quite well, so here’s the way I eventually got it to work after a lot of searching for examples or tutorials, most of which seemed to be out date or incorrect. We’ll be using Django-Storages but we’ll need the Azure library too both can be installed via pip.

Note that for development static files and images are just as normal, it’s only in the production settings we’re using cloud storage.

So first in your requirements file add the following (Other combinations of version may or may not work)


Then in your settings file:

Add storages to installed apps.

Then add these specific settings for Azure Storage, note they reference two separate storage scripts for static and media uploads.
DEFAULT_FILE_STORAGE = 'yourproject.azure_storage.AzureStorage'

AZURE_ACCOUNT_NAME = "yourstorageaccount"
AZURE_ACCOUNT_KEY = "yourkeyhere"
AZURE_CONTAINER = "uploadscontainer"
AZURE_STATIC_CONTAINER = "staticcontainer"

STATICFILES_STORAGE = "yourproject.azure_storage_static.AzureStorage"
STATIC_URL = "https://yourapp.blob.core.windows.net/appstatic/"

With Azure Portal, it’s very easy and quick to setup a Storage app and create containers, you can also use Azure CDN for caching and serving images from nodes near the image’s geographical request origin. There is also an Azure Storage Explorer client you can install even for Mac Os x, all that is discussed in this video https://www.youtube.com/watch?v=lP0KobMaXzM

Then you’ll need to add your storage scripts referenced in your settings, basically they’re the same but reference different values, probably a better way to do this but here’s my one taking care of static storage.

I got this from a fork of the original Django-storages by Josh Schneier at https://github.com/jschneier/django-storages/blob/master/storages/backends/azure_storage.py

import mimetypes
import os.path
import time
from datetime import datetime
from time import mktime

from django.core.exceptions import ImproperlyConfigured
from django.core.files.base import ContentFile
from django.core.files.storage import Storage
from django.utils.deconstruct import deconstructible

from yourapp.utils import setting

    import azure  # noqa
except ImportError:
    raise ImproperlyConfigured(
        "Could not load Azure bindings. "
        "See https://github.com/WindowsAzure/azure-sdk-for-python")

    # azure-storage 0.20.0
    from azure.storage.blob.blobservice import BlobService
    from azure.common import AzureMissingResourceHttpError
except ImportError:
    from azure.storage import BlobService
    from azure import WindowsAzureMissingResourceError as AzureMissingResourceHttpError

def clean_name(name):
    return os.path.normpath(name).replace("\\", "/")

class AzureStorage(Storage):
    account_name = setting("AZURE_ACCOUNT_NAME")
    account_key = setting("AZURE_ACCOUNT_KEY")
    azure_container = setting("AZURE_STATIC_CONTAINER")
    azure_ssl = setting("AZURE_SSL")

    def __init__(self, *args, **kwargs):
        super(AzureStorage, self).__init__(*args, **kwargs)
        self._connection = None

    def connection(self):
        if self._connection is None:
            self._connection = BlobService(
                self.account_name, self.account_key)
        return self._connection

    def azure_protocol(self):
        if self.azure_ssl:
            return 'https'
        return 'http' if self.azure_ssl is not None else None

    def __get_blob_properties(self, name):
            return self.connection.get_blob_properties(
        except AzureMissingResourceHttpError:
            return None

    def _open(self, name, mode="rb"):
        contents = self.connection.get_blob(self.azure_container, name)
        return ContentFile(contents)

    def exists(self, name):
        return self.__get_blob_properties(name) is not None

    def delete(self, name):
            self.connection.delete_blob(self.azure_container, name)
        except AzureMissingResourceHttpError:

    def size(self, name):
        properties = self.connection.get_blob_properties(
            self.azure_container, name)
        return properties["content-length"]

    def _save(self, name, content):
        if hasattr(content.file, 'content_type'):
            content_type = content.file.content_type
            content_type = mimetypes.guess_type(name)[0]

        if hasattr(content, 'chunks'):
            content_data = b''.join(chunk for chunk in content.chunks())
            content_data = content.read()

        self.connection.put_blob(self.azure_container, name,
                                 content_data, "BlockBlob",
        return name

    def url(self, name):
        if hasattr(self.connection, 'make_blob_url'):
            return self.connection.make_blob_url(
            return "{}{}/{}".format(setting('MEDIA_URL'), self.azure_container, name)

    def modified_time(self, name):
            modified = self.__get_blob_properties(name)['last-modified']
        except (TypeError, KeyError):
            return super(AzureStorage, self).modified_time(name)

        modified = time.strptime(modified, '%a, %d %b %Y %H:%M:%S %Z')
        modified = datetime.fromtimestamp(mktime(modified))

        return modified

You’ll need the second one of these for media uploads that references the settings for the other storage container you’ve setup.

Also you will need to add the code in the link below to your utils.py or create a new file and import it.

Hopefully this might save someone some time, it took me a while to get it working. The documentation for Django-storages is very meagre and I found it hard to get a sense of how it all hangs together and how to configure everything. This maybe completely wrong but it’s working, I welcome any corrections or feedback!

Note that the document uploads has an issue where it doesn’t deliver them directly from storage, this is an issue with Django/Wagtail I believe and is discussed here on the google group for Wagtail https://groups.google.com/forum/#!topic/wagtail/g90DyrxAX9w


Heist nomination for a recent project!

I’m pleased to say a recent massive project I worked on to launch a new Courses website for the University has been nominated for a Heist Award.

The courses website is built within a Wagtail site, using some Django Views and some Wagtail features, and talks to a GoLang based API.

I was only one of many in the project team which consisted of IT and Marketing people working together. It was a tough project at times, doing things we’d never done before, but at the same time a lot of fun, and a bonus to get even just a nomination!

Some Wagtail V1 Streamfield examples

Ok, firstly just to say StreamField is amazing, it adds so much flexibility for adding content onto a page, the way its been developed makes it so versatile.

I’ve started upgrading one of our existing article pages within our Wagtail CMS to use StreamField. Streamfields can get long so I’ve put it into its own custom tab (another V1 feature).

I’m going to show some example of blocks and how I’ve used them below because at the moment there aren’t too many examples out there. Once people realise how good this CMS and its StreamField feature is, there will be loads more trust me.

First of all I added a SF to my page model ‘Article’ but also don’t forget to add (below) at the top of your models.py :

from wagtail.wagtailcore.fields import StreamField
from wagtail.wagtailcore import blocks
from wagtail.wagtailimages.blocks import ImageChooserBlock
from wagtail.wagtailembeds.blocks import EmbedBlock
from wagtail.wagtailadmin.edit_handlers import FieldPanel, FieldRowPanel,MultiFieldPanel, \
    InlinePanel, PageChooserPanel, StreamFieldPanel


class ArticlePage(Page):
    page_content = StreamField([
        ('heading', blocks.CharBlock(classname="full title",icon="title")),
        ('paragraph', blocks.RichTextBlock()),
        ('image', ImageChooserBlock(icon="image")),
        ('two_columns', TwoColumnBlock()),
        ('three_columns', ThreeColumnBlock()),
        ('embedded_video', EmbedBlock(icon="media")),
        ('google_map', GoogleMapBlock()),
        ('image_carousel', blocks.ListBlock(ImageCarouselBlock(),template='yourapp/blocks/carousel.html',icon="image")),


You can see there are some of the standard built in block types there, like heading,paragraph and image, but also some custom ones. I’ll go through some of these next.

Google Map Block

We often need to place a map or two on a page, up until now that was in fixed position within a template, but now they can be added in any order and also in columns (see next bit). So the code to define the block referred to above is :

class GoogleMapBlock(blocks.StructBlock):
    map_long = blocks.CharBlock(required=True,max_length=255)
    map_lat = blocks.CharBlock(required=True,max_length=255)
    map_zoom_level = blocks.CharBlock(default=14,required=True,max_length=3)

    class Meta:
        template = 'yourapp/blocks/google_map.html'
        icon = 'cogs'
        label = 'Google Map'

This is based on a StructBlock and just has three fields for the google map template. It’s very simple like the Twitter one from a previous post.

The template is something like this: (google_map.html)

<script src="http://maps.googleapis.com/maps/api/js"></script>
function initialize() {
  var mapProp = {
    center:new google.maps.LatLng({{ self.map_lat }},{{ self.map_long }}),
    zoom:{{ self.map_zoom_level }},
  var map=new google.maps.Map(document.getElementById("googleMap-{{ self.map_lat }}{{ self.map_long }}"),mapProp);
google.maps.event.addDomListener(window, 'load', initialize);

<div id="googleMap-{{ self.map_lat }}{{ self.map_long }}" style="width:100%;height:380px;"></div>

Because there maybe any number of Maps added through streamfield, you need to make sure the mapID is unique-ish, so I’ve used the long/lat values to form the ID, assuming the same map isn’t added twice, a timestamp would be another way to prefix the ID (as I’ve done in the image gallery later on)

Also this loads the Google Map API JS each time a map is used, so best to add that just once somewhere else, ideally conditionally, I’ve put it within the block template just for example.

Two Column Block

class TwoColumnBlock(blocks.StructBlock):

    background = blocks.ChoiceBlock(choices=COLOUR_CHOICES,default="white")
    left_column = blocks.StreamBlock([
            ('heading', blocks.CharBlock(classname="full title")),
            ('paragraph', blocks.RichTextBlock()),
            ('image', ImageChooserBlock()),
            ('embedded_video', EmbedBlock()),
            ('google_map', GoogleMapBlock()),
        ], icon='arrow-left', label='Left column content')

    right_column = blocks.StreamBlock([
            ('heading', blocks.CharBlock(classname="full title")),
            ('paragraph', blocks.RichTextBlock()),
            ('image', ImageChooserBlock()),
            ('embedded_video', EmbedBlock()),
            ('google_map', GoogleMapBlock()),
        ], icon='arrow-right', label='Right column content')

    class Meta:
        template = 'yourapp/blocks/two_column_block.html'
        icon = 'placeholder'
        label = 'Two Columns'

Again this is based on a StructBlock, but now we have two fields that use StreamBlock type, which allows nesting more block types within. You can define which blocks are then allowed in each column. This is really useful as in some implementations you wouldn’t probably want users to add say a wide image gallery into a narrow column. This block also has a background ChoiceBlock field to define a background colour class which gets applied to the entire row DIV that holds the two columns of StreamBlocks.

Here is the template:

<div class="row {{ self.background }}">

      <div class="col-md-6">
           {% include "yourapp/includes/sf_blocks.html" with blocks=self.left_column only %}
      <div class="col-md-6">
           {% include "yourapp/includes/sf_blocks.html" with blocks=self.right_column only %}


and the include that renders the blocks:

{% load wagtailcore_tags wagtailimages_tags %}

{% if blocks %}

						{% for block in blocks %}
						    {% if block.block_type == 'heading' %}
						        <h1>{{ block.value }}</h1>
						    {% elif block.block_type == 'image' %}
						        {% image block.value width-900 class="img-responsive" %}
						    {% else %}
						       <section class="block-{{ block.block_type }}">
						           {{ block }}
						    {% endif %}
						{% endfor %}

{% endif %}

This template code gets used in a few places so I made it as an include.

Image Gallery/Carousel

This is based on a StructBlock but gets used as part of a list block (remembering back to our SF in the model)

        ('image_carousel', blocks.ListBlock(ImageCarouselBlock(),template='yourapp/blocks/carousel.html',icon="image")),

It’s a simple image carousel with just an image and a caption.

class ImageCarouselBlock(blocks.StructBlock):
    image = ImageChooserBlock()
    caption = blocks.TextBlock(required=False)

    class Meta:
        icon = 'image'

The template for this uses Royal Slider to render the images in a nice interface. It refers to specific Royal Slider CSS and JS, you’ll need to replace this with your specific slider code etc

{% load wagtailimages_tags static block_tags %}

	{% timestamp as id_prefix %}

	  <link rel="stylesheet" href="{% static "yourapp/css/sw_carousel.min.css" %}" />

	  <div class="container-carousel">

	      <div id="{{ id_prefix }}-gallery" class="royalSlider rsDefault visibleNearby">

		 {% for item in self %}

	            {% image item.image height-400 as carouselimagedata %}
	            <a class="rsImg" data-rsw="{{ carouselimagedata.width }}" data-rsh="{{ carouselimagedata.height }}"  href="{{ carouselimagedata.url }}">
	  				    {{ item.caption }}

		 {% endfor %}



	  // Important note! If you're adding CSS3 transition to slides, fadeInLoadedSlide should be disabled to avoid fade-conflicts.
	  jQuery(document).ready(function($) {
	    var si = $('#{{ id_prefix }}-gallery').royalSlider({
	      addActiveClass: true,
	      arrowsNav: true,
	      controlNavigation: 'none',
	      autoScaleSlider: true,
	      autoScaleSliderWidth: 900,
	      autoScaleSliderHeight: 250,
	      loop: true,
	      fadeinLoadedSlide: false,
	      globalCaption: true,
	      keyboardNavEnabled: true,
	      globalCaptionInside: false,
	      visibleNearby: {
	        enabled: true,
	        centerArea: 0.4,
	        center: true,
	        breakpoint: 650,
	        breakpointCenterArea: 0.64,
	        navigateByCenterClick: true


This template uses an assignment tag from block_tags.py which creates a timestamp number which is used to prefix the gallery ID. This is in case there is more than one gallery on a page. The ID is used in the HTML and then the JS for the slider. The assignment tag looks like this:

    def timestamp():
        dt = datetime.now()
        ts = dt.microsecond
        return str(ts)

Please let me know by commenting if I’ve done anything crazy or there is a better way, hopefully these example may assist someone else as a starting point to using StreamField in Wagtail.

Official StreamField docs : http://docs.wagtail.io/en/v1.0b1/pages/streamfield.html

Hopefully I will add another post soon on using StreamField for a Parallax image block, Instagram Feed and a pull out quote maybe.

My first DjangoCon

Screen Shot 2015-06-03 at 20.36.58DjangoCon Europe 2015 was my first DjangoCon, held in Cardiff City Hall and Cardiff University, luckily Cardiff is where I live so the traveling was easy!

There were some great talks and I learnt a lot, it’s given me a whole array of things I need to look into now.

Highlights for me were Baptiste Mispelon’s adventures in Djangoland, the tales of someone living a much bolder and exciting life than me ūüė¶ Also Peter Finch the poet gave a very entertaining talk on Cardiff, which was a nice way to end the opening day’s morning talks.

After lunch we got to experience the 24-hour free #django emergency hotline, with the legendary doismellburning, apollo13 and MarkusH, who enacted it live in the room, I haven’t seen anything like this before and was more interesting than just a normal talk, plus it made me realise there is a lot more Django support and help out there that I haven’t known about.

Erik Romijn’s talk on Django security was very good, and I need to try his Online Django Security Checker on a few websites, it’s at https://www.ponycheckup.com/ check it out!

Ola Sendecka, the key note speaker on Tuesday gave a well presented talk on¬†the dangers that rabbit holes present to the programmer,¬†Stefan Foulis’ talk on Docker was interesting for someone who has so far only used Vagrant, Stefan talked about the advantages of Docker, and gave a thorough talk with many examples.

In the afternoon on Tuesday¬†Kat Stevens presented her talk ¬†‘The Full Stack Octopus’, all about being the only developer in a company, this really made me appreciate getting to work in a team, I really admire Kat she has to cover a lot of areas in her job.

Loek Van Gent gave one of my favourite talks entitled ¬†‘True beauty is on the inside, but users are shallow’, all about front end development. There was a lot of talk at the conference generally about separating out the front end of app or sites, using things like Ember, the slides of his talk can be found here¬†http://www.slideshare.net/LoekvanGent/shallow

Wednesday’s highlights for me were¬†Ludvig Wadenstein’s talk ‘Better web applications through user testing’, this talk made me realise how we could be doing more regular user testing and the benefits, also he outlined a relatively straight forward way to do this on a monthly basis. Although the ‘Testing Chamber’ sounds a little frightening, check out his presentation slides here¬†http://ludw.se/~ludw/user_testing.pdf

The after lunch CMS panel talked about the state of the CMS in Django, with Iacopo Spalletti and Tom Dyson from the django CMS and Wagtail teams, this was too short for my liking I would have liked more CMS talk, but that’s just because CMSes are my thing, and obviously I’m already a big Wagtail fan!

Ana Balica’s ‘Demystifying mixins with Django’ was really good, even during the talk It made me think of something I’d recently worked on that could benefit from mixins, and though I have used them, maybe I could use them a bit more actually.

DjangoCon 2015 was a great event and these are just some of my personal highlights!


Adding a Twitter Widget for Wagtail’s new StreamField

Wagtail‘s latest feature is called the ‘StreamField’, currently only available in the 1.0 Beta 1 version, but clearly a bit of a Django CMS game changer.

It allows the editor to assemble a flow of page content of varying types (‘Blocks’ defined by the developer) in any order / combination they wish, kind of a Sir Trevor / Tumblr style affair. ¬†It comes with a good selection of default block types, but it’s easy to repurpose these to create things like a Twitter Widget block.

In this guide I’m going to add a StreamField to an existing page class, which allows the user to add headers, paragraphs, images and a Twitter Widget to the page in any iteration they choose. The Twitter Widget I’m talking about is the one you can generate from your twitter settings by going to¬†https://twitter.com/settings/widgets when logged in.

The user will be able to specify their twitter username, widget ID and a limit to how many latest tweets to display. The same principle could be applied for also adding Soundcloud, Instagram or whatever widgets, especially useful if you want a social media rich page, which has your content mixed in with feed content from the social media sites you use.


I like things laid out in easy clear steps, I’m a bit slow I guess ūüė¶ so here goes… (Bare in mind the code may change, but this works for the current incarnation)

First off, add this class to your models.py

At the top I had to add (modify the edit_handlers Panels as you need, but you will obviously need the StreamFieldPanel for this)

from wagtail.wagtailcore.fields import StreamField
from wagtail.wagtailcore import blocks
from wagtail.wagtailimages.blocks import ImageChooserBlock
from wagtail.wagtailadmin.edit_handlers import FieldPanel, FieldRowPanel,MultiFieldPanel, \
 InlinePanel, PageChooserPanel, StreamFieldPanel


class TwitterBlock(blocks.StructBlock):
    twitter_box_username = blocks.CharBlock(required=True)
    twitter_box_widget_id = blocks.CharBlock(required=True)
    twitter_box_tweet_limit = blocks.CharBlock(required=True,max_length=2)

    class Meta:
        template = 'yourapp/blocks/twitter.html'
        icon = 'cogs'
        label = 'Twitter Widget'

We’re just making our own class based on the¬†StructBlock (one of the built in types) and setting up the fields we need for our widget. The other thing I like is you can specify a template for your widget at this point, and choose an Icon, I’ve gone with a Cog, but there are plenty of others. The Label is useful to tell the users what it is once it’s part of the interface.

Next go to your page class where you want the StreamField to appear, and add the code below with other existing fields.

class ArticlePage(Page):

content = StreamField([
        ('heading', blocks.CharBlock(classname="full title")),
        ('paragraph', blocks.RichTextBlock()),
        ('image', ImageChooserBlock()),
        ('twitter', TwitterBlock()),


This sets up our content field which can have a Header, Paragraph,Image or twitter widget.

 Then, so it appears in the admin, add it to the correct panel

ArticlePage.content_panels = [
    FieldPanel('title', classname="full title"),


At this point you’ll need to create and apply migrations for the new field. (If you’re replacing a body field with a Streamfield, you might want to think about adding a new field as we’ve done here rather than changing the existing body field. This will allow your editors to migrate the main page content from the old rich text field into the new StreamField one)

./manage.py makemigrations
python manage.py migrate

The final step is to obviously take care of the templating part!

You’ll need to add a new folder called ‘blocks’ in the template folder of your app, in the location we referenced earlier when defining the block, then create twitter.html as below.

&lt;div class="twitter-widget"&gt;

{% if self.twitter_box_username %}

    &lt;a class="twitter-timeline" href="https://twitter.com/{{ self.twitter_box_username }}" data-widget-id="{{ self.twitter_box_widget_id }}" data-tweet-limit="{{ self.twitter_box_tweet_limit }}"&gt;Tweets by {{ self.twitter_box_username}}&lt;/a&gt;
    &lt;script&gt;!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0],p=/^http:/.test(d.location)?'http':'https';if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src=p+"://platform.twitter.com/widgets.js";fjs.parentNode.insertBefore(js,fjs);}}(document,"script","twitter-wjs");&lt;/script&gt;

{% endif %}


Then of course the main template for your page that has the new StreamField content will need to be changed. Add this code below whereever you want the content StreamField to appear. There a few ways of outputting the blocks, I’ve chosen one from the Wagtail examples where I get a bit more control, for more examples go to http://docs.wagtail.io/en/v1.0b1/pages/streamfield.html#template-rendering

 {% for block in self.content %}
     {% if block.block_type == 'heading' %}
         &lt;h1&gt;{{ block.value }}&lt;/h1&gt;
     {% elif block.block_type == 'image' %}
         {% image block.value width-400 %}
     {% else %}
        &lt;section class="block-{{ block.block_type }}"&gt;
            {{ block }}
     {% endif %}
 {% endfor %}

Hopefully this might help someone, let me know of any mistakes.

Upgrading Wagtail to use Django 1.7 locally using vagrant

These are the steps I followed to upgrade an existing Wagtail 0.8.3 which was using Django 1.6 to use 1.7. It wasn’t that obvious to me so I’ve detailed the steps here.

This assumes you’re using Vagrant locally, to test things, I cloned a new local version of our site in case it went wrong, which it did a few times before I got it right.

The steps may differ depending on how you set things up, and if you’re using additional/different Python modules etc

Firslty make these changes in the wagtail project :

Edit base.txt in requirements, and remove south, set django to 1.7, set taggit to use 0.12.0

Edit base.py in settings (and any other settings files) and remove south from installed apps.

Delete all the old south numbered migrations but leave the migration folder and the __init__.py file (Migrations are now handled with Django,and will be remade from scratch, so you don’t need the old south ones)

Also it’s worth checking through the Wagtail release notes to make sure you’ve checked the upgrade considerations related to Django 1.7, they can be found here :¬†http://docs.wagtail.io/en/latest/releases/index.html

then to get it running locally using vagrant:

Go to your vagrant folder for the project, ¬†do vagrant destroy if you’re not working from scratch.

Ignore this warning when it appears at various points :
RemovedInDjango18Warning: `MP_NodeManager.get_query_set` method should be renamed `get_queryset`.
class MP_NodeManager(models.Manager):

vagrant up
vagrant ssh
cd /vagrant

(next 3 steps assume you have existing data for your local vagrant in a sql file, skip otherwise)

dropdb -Upostgres <dbname>

createdb -Upostgres <dbname>

psql -Upostgres foxtail -f<dbname>.sql

./manage.py makemigrations
./manage.py migrate –fake

That’s it, these steps worked for me, hopefully they’ll be some help.

For more info on Django 1.7 and migrations – https://docs.djangoproject.com/en/1.7/topics/migrations/#upgrading-from-south

Outputing JSON for a model with properties and db fields in Wagtail/Django

I thought I’d share this example because it took me ages to figure out how to output JSON for a model which included a model property as well as the actual fields of the model. For our footer links we had a property called ‘link’ in the snippet which would return either a full URL or a relative link to a Wagtail CMS page. If we just tried to return link_page directly without the link property we’d just get the ID of the page, this wouldn’t be any use to us in the JSON.

This example uses a comprehension list to get the model property and model fields together.
In the models.py for the footer snippet:

class MainFooterLink(models.Model):

 link_title = models.CharField(max_length=255)
 link_url = models.URLField("Link URL", blank=True)
 link_page = models.ForeignKey(
 order_rank = models.IntegerField(default=1)

 panels = [

 def __unicode__(self):
 return self.link_title

 def link(self):
 if self.link_page:
 return "<your base URL here>" + self.link_page.url
 return self.link_url


Define your view function (make sure to import json and HttpResponse):

def footer_json(request):

footerlinks = MainFooterLink.objects.all().order_by('order_rank')

the_links = []
the_links = [{'title':s.link_title, 'link':s.link , 'order':s.order_rank} for s in footerlinks]

return HttpResponse(json.dumps(the_links, sort_keys=True, indent=4, separators=(',', ': ')), mimetype='application/json')