Wagtail and Azure Storage Blob Containers

So recently I’ve been working on a project to move old legacy sites into Wagtail and we’ve set this Wagtail site up on the Azure Cloud using Azure Web Apps for Linux with a custom Docker Container. Ideally we wanted images uploaded by the user and our static files stored separately so we used Azure Storage and setup two containers. One container for static files, and one for images and docs uploaded by the users through Wagtail.

It seems to be working quite well, so here’s the way I eventually got it to work after a lot of searching for examples or tutorials, most of which seemed to be out date or incorrect. We’ll be using Django-Storages but we’ll need the Azure library too both can be installed via pip.

Note that for development static files and images are just as normal, it’s only in the production settings we’re using cloud storage.

So first in your requirements file add the following (Other combinations of version may or may not work)

django-storages==1.6.5
azure==1.0

Then in your settings file:

Add storages to installed apps.

'storages'
Then add these specific settings for Azure Storage, note they reference two separate storage scripts for static and media uploads.
DEFAULT_FILE_STORAGE = 'yourproject.azure_storage.AzureStorage'

AZURE_ACCOUNT_NAME = "yourstorageaccount"
AZURE_ACCOUNT_KEY = "yourkeyhere"
AZURE_CONTAINER = "uploadscontainer"
AZURE_STATIC_CONTAINER = "staticcontainer"
AZURE_SSL = True

STATICFILES_STORAGE = "yourproject.azure_storage_static.AzureStorage"
STATIC_URL = "https://yourapp.blob.core.windows.net/appstatic/"
COMPRESS_STORAGE = STATICFILES_STORAGE
COMPRESS_ROOT = ''

With Azure Portal, it’s very easy and quick to setup a Storage app and create containers, you can also use Azure CDN for caching and serving images from nodes near the image’s geographical request origin. There is also an Azure Storage Explorer client you can install even for Mac Os x, all that is discussed in this video https://www.youtube.com/watch?v=lP0KobMaXzM

Then you’ll need to add your storage scripts referenced in your settings, basically they’re the same but reference different values, probably a better way to do this but here’s my one taking care of static storage.

I got this from a fork of the original Django-storages by Josh Schneier at https://github.com/jschneier/django-storages/blob/master/storages/backends/azure_storage.py


import mimetypes
import os.path
import time
from datetime import datetime
from time import mktime

from django.core.exceptions import ImproperlyConfigured
from django.core.files.base import ContentFile
from django.core.files.storage import Storage
from django.utils.deconstruct import deconstructible

from yourapp.utils import setting

try:
    import azure  # noqa
except ImportError:
    raise ImproperlyConfigured(
        "Could not load Azure bindings. "
        "See https://github.com/WindowsAzure/azure-sdk-for-python")

try:
    # azure-storage 0.20.0
    from azure.storage.blob.blobservice import BlobService
    from azure.common import AzureMissingResourceHttpError
except ImportError:
    from azure.storage import BlobService
    from azure import WindowsAzureMissingResourceError as AzureMissingResourceHttpError


def clean_name(name):
    return os.path.normpath(name).replace("\\", "/")


@deconstructible
class AzureStorage(Storage):
    account_name = setting("AZURE_ACCOUNT_NAME")
    account_key = setting("AZURE_ACCOUNT_KEY")
    azure_container = setting("AZURE_STATIC_CONTAINER")
    azure_ssl = setting("AZURE_SSL")

    def __init__(self, *args, **kwargs):
        super(AzureStorage, self).__init__(*args, **kwargs)
        self._connection = None

    @property
    def connection(self):
        if self._connection is None:
            self._connection = BlobService(
                self.account_name, self.account_key)
        return self._connection

    @property
    def azure_protocol(self):
        if self.azure_ssl:
            return 'https'
        return 'http' if self.azure_ssl is not None else None

    def __get_blob_properties(self, name):
        try:
            return self.connection.get_blob_properties(
                self.azure_container,
                name
            )
        except AzureMissingResourceHttpError:
            return None

    def _open(self, name, mode="rb"):
        contents = self.connection.get_blob(self.azure_container, name)
        return ContentFile(contents)

    def exists(self, name):
        return self.__get_blob_properties(name) is not None

    def delete(self, name):
        try:
            self.connection.delete_blob(self.azure_container, name)
        except AzureMissingResourceHttpError:
            pass

    def size(self, name):
        properties = self.connection.get_blob_properties(
            self.azure_container, name)
        return properties["content-length"]

    def _save(self, name, content):
        if hasattr(content.file, 'content_type'):
            content_type = content.file.content_type
        else:
            content_type = mimetypes.guess_type(name)[0]

        if hasattr(content, 'chunks'):
            content_data = b''.join(chunk for chunk in content.chunks())
        else:
            content_data = content.read()

        self.connection.put_blob(self.azure_container, name,
                                 content_data, "BlockBlob",
                                 x_ms_blob_content_type=content_type)
        return name

    def url(self, name):
        if hasattr(self.connection, 'make_blob_url'):
            return self.connection.make_blob_url(
                container_name=self.azure_container,
                blob_name=name,
                protocol=self.azure_protocol,
            )
        else:
            return "{}{}/{}".format(setting('MEDIA_URL'), self.azure_container, name)

    def modified_time(self, name):
        try:
            modified = self.__get_blob_properties(name)['last-modified']
        except (TypeError, KeyError):
            return super(AzureStorage, self).modified_time(name)

        modified = time.strptime(modified, '%a, %d %b %Y %H:%M:%S %Z')
        modified = datetime.fromtimestamp(mktime(modified))

        return modified

You’ll need the second one of these for media uploads that references the settings for the other storage container you’ve setup.

Also you will need to add the code in the link below to your utils.py or create a new file and import it.
https://github.com/jschneier/django-storages/blob/master/storages/utils.py

Hopefully this might save someone some time, it took me a while to get it working. The documentation for Django-storages is very meagre and I found it hard to get a sense of how it all hangs together and how to configure everything. This maybe completely wrong but it’s working, I welcome any corrections or feedback!

Note that the document uploads has an issue where it doesn’t deliver them directly from storage, this is an issue with Django/Wagtail I believe and is discussed here on the google group for Wagtail https://groups.google.com/forum/#!topic/wagtail/g90DyrxAX9w

Advertisements

Heist nomination for a recent project!

I’m pleased to say a recent massive project I worked on to launch a new Courses website for the University has been nominated for a Heist Award.

The courses website is built within a Wagtail site, using some Django Views and some Wagtail features, and talks to a GoLang based API.

I was only one of many in the project team which consisted of IT and Marketing people working together. It was a tough project at times, doing things we’d never done before, but at the same time a lot of fun, and a bonus to get even just a nomination!

Blog Update

I haven’t posted much on Wagtail or anything really this year, though we’re still using Wagtail, I’ve been working on a large project for most of the year which involves Rails, GoLang as well as ‘some’ Django/Wagtail for the front end. But I’m planning in the new year to do some Wagtail based blog posts and hopefully more blog posts generally 🙂

Merry Xmas

Joss

 

Using CAS authentication with Wagtail

Hi, this is quick setup guide for using django-cas-client (1.2.0) with Wagtail, gave me a few headaches so maybe this can help someone else.

I’m assuming you have a CAS authentication server setup already, this guide doesn’t cover that.

First off you’ll need to install the django-cas-client, I did this by adding a line in my requirements.txt of :

django-cas-client==1.2.0

Then in your settings file, make sure you add a backend like :


AUTHENTICATION_BACKENDS = (
	'django.contrib.auth.backends.ModelBackend',
	'cas.backends.CASBackend',
)

then :

CAS_SERVER_URL = 'https://login.yourserveraddresstocas.com/cas/'

CAS_REDIRECT_URL = '/admin/' 

CAS_AUTO_CREATE_USER = False

(The auto create setting is really important otherwise the CAS client will attempt to make a user in Wagtail with no details, this will cause a redirect loop on login for authenticated users)

next :

Add

'cas.middleware.CASMiddleware', 

to the MIDDLEWARE_CLASSES section

then :

Add

'cas',

to the list of installed apps.

That’s all for settings, next open up your urls.py and add these lines :


url(r'^admin/login/$', 'cas.views.login', name='login'),
url(r'^admin/logout/$', 'cas.views.logout', name='logout'),

That should be it, any problems comment below!

How to stop users accidentally leaving a Wagtail edit page with JQuery

We had a request recently to implement a feature to stop users accidentally losing work while editing a page then forgetting to save. Especially now in Wagtail with features like StreamField, they can get pretty engrossed in their page layout design then forget to save, hit the ‘Back’ button my mistake etc.

Although I believe this functionality will be introduced into Wagtail itself soon, here is how to do it in a ‘quick and easy’ way before that time.

First off create a Javascript file in your app where ever you keep those, let’s call it leave_page.js (You can think of a better title name I’m sure)

Then add this code:


(function() {
(function($) {

window.onload = function() {

var submitted = false;

$(":submit").click(function(event){
submitted = true;
});

var del_button = $( "a:contains('Delete')" );
del_button.click(function(event){
submitted = true;
});

var unpublish_button = $( "a:contains('Unpublish')" );
unpublish_button.click(function(event){
submitted = true;
});

window.addEventListener("beforeunload", function (e) {

if (submitted == false){
var confirmationMessage = 'It looks like you have been editing something. ';
confirmationMessage += 'If you leave before saving, your changes will be lost.';

(e || window.event).returnValue = confirmationMessage; //Gecko + IE
return confirmationMessage; //Gecko + Webkit, Safari, Chrome etc.
}
else {
return undefined;
}

});
};

})(jQuery);

}).call(this);


The first part of the code sets a flag so we can tell if any of the Wagtail main actions have been pressed like Save,Publish, Delete etc, we don’t want to bother the user if the page is being unloaded by those actions. If someone clicks Publish then we assume that’s what they want.

The code finds the Wagtail specific buttons and gives them a click handler which sets the flag to true.

The last part adds an event listener for the page exiting ‘beforeunload’, so just before we can remind them with an alert that they might lose their work, as long as submitted isn’t true.

You’ll need to get this JS picked up by Wagtail so use the Wagtail Hook ‘insert_editor_js’, create a file called ‘wagtail_hooks.py’ if you don’t already have one, then register the new JS Code from that. For more info on how to do that visit http://docs.wagtail.io/en/latest/reference/hooks.html

Hope this helps, let me know if there are any mistakes or if there is a cleaner, nicer way to do it.

More StreamField Examples – Top news stories block by tag

We thought it would be useful to be able to display say the top 5 news stories tagged with a certain word anywhere within a page using StreamField, so for example the latest sport related news stories could be added into a column next to something else on a page in Wagtail. You could adapt this example to use any page type in your Models.py, doesn’t have to be news. Also I really like the way you can set a default template for a latest news block, but then override that and set another one if you need it rendered within a thinner column. Anyway here’s what I did :

Firstly make a new block class bases on StructBlock,

The filter below on the query for news assumes your news model is hooked up using taggit so each news story can be tagged.



class NewsStoriesBlock(blocks.StructBlock):
    tagged_by_keyword = blocks.CharBlock(required=True)
    stories_limit = blocks.CharBlock(required=True,max_length=2)
    
    def render(self, value): 
        news = NewsPage.objects.filter(live=True).filter(tags__name=value['tagged_by_keyword']).order_by('-date')
        news = news[:value['stories_limit']]
        
        return render_to_string(self.meta.template, { 
            'self': value, 
            'news_stories': news, 
            
        })
        
    class Meta:
        template = 'yourapp/blocks/news_stories.html'
        icon = 'cogs'
        label = 'News Stories Widget'

tagged_by_keyword is the field used to hold the tag name ‘sport’ etc

for stories_limit I’ve set max_length to 2, meaning 99 is the most a user could request, but you could more practically change this to a choiceBlock of restricted values, or even add some code into the render method, so it checks the value is no more than say 30, and if it is just sets it 30 as a cap.

Then add the new block to your StreamField in your page model, like in the example one below. For how to do a two column block see ‘Some Wagtail v1 StreamField Examples‘ post


class ArticlePage(Page):
    ...
    page_content = StreamField([
            ('heading', blocks.CharBlock(classname="full title",icon="title")),
            ('paragraph', blocks.RichTextBlock()),
            ('image', ImageChooserBlock(icon="image")),
            ('two_columns', TwoColumnBlock()),
            ('news_stories', NewsStoriesBlock()),
        ],null=True,blank=True)
     ...

When you add the news block to a two column StreamField, set another block template if you need it rendered in a different format better suited to a thinner half column, like


...
left_column = blocks.StreamBlock([
            ('heading', blocks.CharBlock(classname="full title")),
            ('paragraph', blocks.RichTextBlock()),
            ('image', ImageChooserBlock()),
            ('news_stories', NewsStoriesBlock(icon="cogs",template='yourapp/blocks/news_stories_cols.html')),
        ], icon='arrow-left', label='Left column content')            
...

And a block template for rendering news into three bootstrap columns (note you need to use a filter from a template tag to make news page URL workable, code below. You can’t use {% pageurl news %} there is no request object at this level, if someone knows how to do that, or if I’m missing something let me know)

Also this assumes your news model has a title, date and news_image field, change to reflect your page setup.


{% load wagtailimages_tags static yourapp_tags %}

{% if news_stories %}
		
<div class="container-newsbytag">

  <div class="row">
		
              <div class="col-md-12">
                 <h2>News tagged with {{ self.tagged_by_keyword }}</h2>
              </div>
			
	      {% for news in news_stories %}
	      
              {% if forloop.first %}<div class='row'>{% endif %}	

              <div class="col-md-4">

                   <a href="{{ news.url_path|clean_root_url }}" class="news_itembytag">
      
                      {% image news.news_image fill-300x200 class="img-responsive" %}
		
                      <h4>{{ news.title }}</h4>
                      <p class="date">{{ news.date|date:"j F Y" }}</p>

                   </a>


              </div>
	      
              {% if forloop.counter|divisibleby:3 %}</div><div class='row'>{% endif %}
			
              {% if forloop.last %}</div>{% endif %}	
		  	
	      {% empty %} No News Stories found
							
              {% endfor %}

 </div>

</div>
	

{% endif %}


in your yourapp_tags.py (what ever ‘templatetags’ code you have setup. This is used to remove the root /home/ part from the url_path of the news page.


@register.filter
def clean_root_url(url):
    url = url.split('/')
    new_url = ""
    for item in url[2:]:
        new_url = new_url + "/" + item
          
    return new_url

Parallax Background Image Block for Wagtail’s StreamField

Another custom block idea for Streamfield, to go with the ones described in a previous post

I will also refer to some code in the previous post too, to save repeating code here.

This block definition and block template allow you to add a column with a background image in the parallax style and allows you to set some adjustments like text align, image alignment etc.

Firstly here is the block class, based on a StructBlock. Basically some choice block fields and a StreamBlock so you can nest headers and text within it.


ALIGN_CHOICES = (
    ('left', "Left"),
    ('right', "Right"),
    ('center', "Centre"),
)

SIZE_CHOICES = (
    ('auto', "Auto"),
    ('cover', "Cover"),
    ('50%', "Small"),
    ('200%', "Large"),
)


PERCENT_CHOICES = (
    ('10%', "10%"),
    ('20%', "20%"),
    ('30%', "30%"),
    ('40%', "40%"),
    ('50%', "50%"),
    ('60%', "60%"),
    ('70%', "70%"),
    ('80%', "80%"),
    ('90%', "90%"),
    ('100%', "100%"),
)

class OneColumnBlock(blocks.StructBlock):

    back_image = ImageChooserBlock()
    background_size = blocks.ChoiceBlock(choices=SIZE_CHOICES,default="auto")
    background_x_position = blocks.ChoiceBlock(choices=PERCENT_CHOICES,default="50%")
    background_y_position = blocks.ChoiceBlock(choices=PERCENT_CHOICES,default="50%")
    text_align = blocks.ChoiceBlock(choices=ALIGN_CHOICES,default="center")
    one_column = blocks.StreamBlock([
           ('heading', blocks.CharBlock(classname="full title")),
           ('paragraph', blocks.RichTextBlock()),
        ], icon='arrow-left', label='Parallax content')

    class Meta:
        template = 'yourapp/blocks/one_column_block.html'
        icon = 'placeholder'
        label = 'Parallax Column'

Obviously you’ll need to add this as an allowed block to your StreamField in your model, see the previous post (link above)

Then the block template code to render it:


{% load wagtailimages_tags block_tags %}

{% image self.back_image width-2000 as back_photo %}

{% timestamp as id_prefix %}

<style>

.parallax{{ id_prefix }} {
 height: 50vh;

 background-attachment: fixed;
 background-size:{{ self.background_size }};
 background-repeat:no-repeat;
 background-position: {{ self.background_x_position }} {{ self.background_y_position }};
}

</style>
<div class="parallax{{ id_prefix }}" style="background-image: url({{ back_photo.url }});">

 <div style="text-align:{{ self.text_align }};padding:15px;">

 {% include "yourapp/includes/sf_blocks.html" with blocks=self.one_column only %}

 </div>

</div>

The block_tags and include for sf_blocks (above) are explained here in ‘Some Wagtail V1 StreamField Examples

The CSS could be applied via an ID rather than class which would make sense, but I’ve done it as a class with an added millisecond timestamp, to allow more than one of these block per page.