Wagtail and Azure Storage Blob Containers

So recently I’ve been working on a project to move old legacy sites into Wagtail and we’ve set this Wagtail site up on the Azure Cloud using Azure Web Apps for Linux with a custom Docker Container. Ideally we wanted images uploaded by the user and our static files stored separately so we used Azure Storage and setup two containers. One container for static files, and one for images and docs uploaded by the users through Wagtail.

It seems to be working quite well, so here’s the way I eventually got it to work after a lot of searching for examples or tutorials, most of which seemed to be out date or incorrect. We’ll be using Django-Storages but we’ll need the Azure library too both can be installed via pip.

Note that for development static files and images are just as normal, it’s only in the production settings we’re using cloud storage.

So first in your requirements file add the following (Other combinations of version may or may not work)

django-storages==1.6.5
azure==1.0

Then in your settings file:

Add storages to installed apps.

'storages'
Then add these specific settings for Azure Storage, note they reference two separate storage scripts for static and media uploads.
DEFAULT_FILE_STORAGE = 'yourproject.azure_storage.AzureStorage'

AZURE_ACCOUNT_NAME = "yourstorageaccount"
AZURE_ACCOUNT_KEY = "yourkeyhere"
AZURE_CONTAINER = "uploadscontainer"
AZURE_STATIC_CONTAINER = "staticcontainer"
AZURE_SSL = True

STATICFILES_STORAGE = "yourproject.azure_storage_static.AzureStorage"
STATIC_URL = "https://yourapp.blob.core.windows.net/appstatic/"
COMPRESS_STORAGE = STATICFILES_STORAGE
COMPRESS_ROOT = ''

With Azure Portal, it’s very easy and quick to setup a Storage app and create containers, you can also use Azure CDN for caching and serving images from nodes near the image’s geographical request origin. There is also an Azure Storage Explorer client you can install even for Mac Os x, all that is discussed in this video https://www.youtube.com/watch?v=lP0KobMaXzM

Then you’ll need to add your storage scripts referenced in your settings, basically they’re the same but reference different values, probably a better way to do this but here’s my one taking care of static storage.

I got this from a fork of the original Django-storages by Josh Schneier at https://github.com/jschneier/django-storages/blob/master/storages/backends/azure_storage.py


import mimetypes
import os.path
import time
from datetime import datetime
from time import mktime

from django.core.exceptions import ImproperlyConfigured
from django.core.files.base import ContentFile
from django.core.files.storage import Storage
from django.utils.deconstruct import deconstructible

from yourapp.utils import setting

try:
    import azure  # noqa
except ImportError:
    raise ImproperlyConfigured(
        "Could not load Azure bindings. "
        "See https://github.com/WindowsAzure/azure-sdk-for-python")

try:
    # azure-storage 0.20.0
    from azure.storage.blob.blobservice import BlobService
    from azure.common import AzureMissingResourceHttpError
except ImportError:
    from azure.storage import BlobService
    from azure import WindowsAzureMissingResourceError as AzureMissingResourceHttpError


def clean_name(name):
    return os.path.normpath(name).replace("\\", "/")


@deconstructible
class AzureStorage(Storage):
    account_name = setting("AZURE_ACCOUNT_NAME")
    account_key = setting("AZURE_ACCOUNT_KEY")
    azure_container = setting("AZURE_STATIC_CONTAINER")
    azure_ssl = setting("AZURE_SSL")

    def __init__(self, *args, **kwargs):
        super(AzureStorage, self).__init__(*args, **kwargs)
        self._connection = None

    @property
    def connection(self):
        if self._connection is None:
            self._connection = BlobService(
                self.account_name, self.account_key)
        return self._connection

    @property
    def azure_protocol(self):
        if self.azure_ssl:
            return 'https'
        return 'http' if self.azure_ssl is not None else None

    def __get_blob_properties(self, name):
        try:
            return self.connection.get_blob_properties(
                self.azure_container,
                name
            )
        except AzureMissingResourceHttpError:
            return None

    def _open(self, name, mode="rb"):
        contents = self.connection.get_blob(self.azure_container, name)
        return ContentFile(contents)

    def exists(self, name):
        return self.__get_blob_properties(name) is not None

    def delete(self, name):
        try:
            self.connection.delete_blob(self.azure_container, name)
        except AzureMissingResourceHttpError:
            pass

    def size(self, name):
        properties = self.connection.get_blob_properties(
            self.azure_container, name)
        return properties["content-length"]

    def _save(self, name, content):
        if hasattr(content.file, 'content_type'):
            content_type = content.file.content_type
        else:
            content_type = mimetypes.guess_type(name)[0]

        if hasattr(content, 'chunks'):
            content_data = b''.join(chunk for chunk in content.chunks())
        else:
            content_data = content.read()

        self.connection.put_blob(self.azure_container, name,
                                 content_data, "BlockBlob",
                                 x_ms_blob_content_type=content_type)
        return name

    def url(self, name):
        if hasattr(self.connection, 'make_blob_url'):
            return self.connection.make_blob_url(
                container_name=self.azure_container,
                blob_name=name,
                protocol=self.azure_protocol,
            )
        else:
            return "{}{}/{}".format(setting('MEDIA_URL'), self.azure_container, name)

    def modified_time(self, name):
        try:
            modified = self.__get_blob_properties(name)['last-modified']
        except (TypeError, KeyError):
            return super(AzureStorage, self).modified_time(name)

        modified = time.strptime(modified, '%a, %d %b %Y %H:%M:%S %Z')
        modified = datetime.fromtimestamp(mktime(modified))

        return modified

You’ll need the second one of these for media uploads that references the settings for the other storage container you’ve setup.

Also you will need to add the code in the link below to your utils.py or create a new file and import it.
https://github.com/jschneier/django-storages/blob/master/storages/utils.py

Hopefully this might save someone some time, it took me a while to get it working. The documentation for Django-storages is very meagre and I found it hard to get a sense of how it all hangs together and how to configure everything. This maybe completely wrong but it’s working, I welcome any corrections or feedback!

Note that the document uploads has an issue where it doesn’t deliver them directly from storage, this is an issue with Django/Wagtail I believe and is discussed here on the google group for Wagtail https://groups.google.com/forum/#!topic/wagtail/g90DyrxAX9w

Advertisements

One thought on “Wagtail and Azure Storage Blob Containers

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.