Django Fieldsets

HTML forms contain a construct called a fieldset. These are generally used to segment a form: splitting a form into groups of fields that are logically grouped. Each fieldset may also have a legend.

Django’s forms have no concept of a fieldset natively, but with a bit of patching, we can make every django form capable of rendering itself using fieldsets, yet still be backwards compatible with non-fieldset-aware templates.

Ideally, we would like to be able to render a form in a way similar to:

<form>
  {% for fieldset in form.fieldsets %}
  <fieldset>
    <legend>{{ fieldset.title }}</legend>
    
    <ul>
      {% for field in fieldset %}
        <li>
          {{ field.label_tag }}
          {{ field }}
          {{ field.help_text }}
          {{ field.errors }}
        </li>
      {% endfor %}
    </ul>
  </fieldset>
  {% endfor %}
  
  <!-- submit button -->
</form>

And, it would make sense to be able to declare a form’s fieldsets in a manner such as:

class MyForm(forms.Form):
  field1 = forms.BooleanField(required=False)
  field2 = forms.CharField()
  
  class Meta:
    fieldsets = (
      ('Fieldset title', {
        'fields': ('field1', 'field2')
      }),
    )

This is similar to how fieldsets are declared in the django admin.

We can’t just simply create a subclass of forms.Form, and do everything there, as the metaclass stuff doesn’t work correctly. Instead, we need to duck-punch.

First, we want to redefine the metaclass __init__ method, so it will accept the fieldsets attribute.

from django import forms
from django.forms.models import ModelFormOptions

_old_init = ModelFormOptions.__init__

def _new_init(self, options=None):
  _old_init(self, options)
  self.fieldsets = getattr(options, 'fieldsets', None)

ModelFormOptions.__init__ = _new_init

Next, we will need a Fieldset class:

class Fieldset(object):
  def __init__(self, form, title, fields, classes):
    self.form = form
    self.title = title
    self.fields = fields
    self.classes = classes
  
  def __iter__(self):
    # Similar to how a form can iterate through it's fields...
    for field in self.fields:
      yield field

And finally, we need to give every form a fieldsets method, which will yield each fieldset, as a Fieldset defined above:

def fieldsets(self):
  meta = getattr(self, '_meta', None)
  if not meta:
    meta = getattr(self, 'Meta', None)
  
  if not meta or not meta.fieldsets:
    return
  
  for name, data in meta.fieldsets:
    yield Fieldset(
      form=self,
      title=name,
      fields=(self[f] for f in data.get('fields',(,))),
      classes=data.get('classes', '')
    )

forms.BaseForm.fieldsets = fieldsets

I am using this code (or something very similar to it), in projects. It works for me, but your mileage may vary…

Django Proxy Model State Machine

Finite State Machines (fsm) are a great way to model something that has, well, a finite number of known states. You can easily specify the different states, and the transitions between them.

Some time ago, I came across a great way of doing this in python: Dynamic State Machines. This maps well onto an idea I have been toying with lately, replacing a series of linked models representing different phases in a process with one model type. Initially, I had thought to just use a type flag, but actually changing the class seems like a better idea.

One aspect of django’s models that makes it easy to do this is the concept of a Proxy Model. These are models that share the database table, but have different class definitions. However, usually a model instance will be of the type that was used to fetch it:

class ModelOne(models.Model):
  field = models.CharField()
  
class ModelOneProxy(ModelOne):
  class Meta:
    proxy = True

ModelOneProxy.objects.get(pk=1) # Returns a ModelOneProxy object.
ModelOne.objects.all() # Returns all ModelOne objects.

However, by using a type field, we can, at the time it is fetched from the database, turn it into the correct type.

class StateMachineModel(models.Model):
  status = models.CharField(max_length=64)
  
  def __init__(self, *args, **kwargs):
    super(StateMachineModel, self).__init__(*args, **kwargs)
    self.__class__ = class_mapping[self.status]

However, having to store a registry of status : <ProxyModelClass> objects is not much fun.

Enter __subclasses__.

  @property
  def _get_states(self):
    """
    Get a mapping of {status: SubClass, ...}
    
    The status key will be the name of the SubClass, with the
    name of the superclass stripped out.
    
    It is intended that you prefix your subclasses with a meaningful
    name, that will be used as the status value.
    """
    return dict([
      (
        sub.__name__.lower().replace(self.__class__.__name__, ''),
        sub
      ) for sub in self.__class__.__subclasses__()
    ])
  
  # in __init__, above, replace the last line with:
    self.__class__ = self._get_states[self.status]

Now, we need to change the underlying class when the type gets changed

  def __setattr__(self, attr, value):
    if attr == 'status':
      states = self._get_states
      if value in states:
        self.__class__ = states[value]
    return super(StateMachineModel, self).__setattr__(attr, value)

As the docstring on _get_states indicates, it looks at the subclass name, and compares it to the superclass name to work out the values that will be stored as the status (and used to dynamically change the class).

This has a fairly large implication: you cannot fetch database objects of any of the subclass types directly: you would need to:

SuperClass.objects.filter(status="prefix")

Of course, you could use queryset methods to do this: that’s what I have been doing.

This is still a bit of a work in progress: it’s not well tested, but is an interesting idea.

The full version of this model class, which is slightly different to above:

from django.db import models

class StateMachineModel(models.Model):
    status = models.CharField(max_length=64)
    
    class Meta:
        abstract = True
    
    def __init__(self, *args, **kwargs):
        self._states = dict([
            (sub.__name__.replace(self.__class__.__name__, '').lower(), sub)
            for sub in self.__class__.__subclasses__()
        ])
        super(StateMachineModel, self).__init__(*args, **kwargs)
        self._meta.get_field_by_name('status')[0]._choices = [(x, x) for x in self._states.keys()]
        self._set_state()
            
    def _set_state(self):
        if getattr(self, 'status', None) in self._states:
            self.__class__ = self._states[self.status]
    
    def __setattr__(self, attr, value):
        if attr == 'status':
            self._set_state()
        return super(StateMachineModel, self).__setattr__(attr, value)

Neat and tidy read-only fields

I have a recurring pattern I’m seeing, where I have a field in a model that needs to be read-only. It usually is a Company to which an object belongs, but it also occurs in the case where an object belongs to some collection, and isn’t permitted to be moved to a different collection.

Whilst there are some workarounds that apply the field’s value to the instance after creating, it’s nicer to be able to apply the read-only nature declaratively, and not have to remember to do something in the form itself.

Unfortunately, in django, normal field subclasses don’t have access to the initial argument that was used to construct it. But forms.FileField objects do. So we can abuse that a little.

We also need a widget, that will always return False for questions about if the value has been changed, and re-render with the initial value at all times.

from django import forms

class ReadOnlyWidget(forms.HiddenInput):
    def render(self, name, value, attrs):
      value = getattr(self, 'initial', value)
      return super(ReadOnlyWidget, self).render(name, value, attrs)
    
    def _has_changed(self, initial, data):
      return False

class ReadOnlyField(forms.FileField):
  widget = forms.HiddenInput
  
  def __init__(self, *args, **kwargs):
    forms.Field.__init__(self, *args, **kwargs)
  
  def clean(self, value, initial):
    self.widget.initial = initial
    return initial

So, that’s all well and good. But a common use for me was for this field to be a related field: a Company as described above, or a User.

Enter ReadOnlyModelField, and ReadOnlyUserField.

Now, ReadOnlyModelField is a bit tricky: it’s not actually a class, but a factory function, so we will look at ReadOnlyUserField first:

class ReadOnlyUserField(ReadOnlyField):
  def clean(self, value, initial):
    initial = super(ReadOnlyUserField, self).clean(value, initial)
    return User.objects.get(pk=initial)

Note, it will have a database query.

Now, we are ready to look at a more general case:

def ReadOnlyModelField(ModelClass, *args, **kwargs):
  class ReadOnlyModelField(ReadOnlyField):
    def clean(self, value, initial):
      initial = super(ReadOnlyModelField, self).clean(value, initial)
      return ModelClass.objects.get(pk=initial)
  return ReadOnlyModelField(*args, **kwargs)

This is a bit tricky. We create a function that looks like a class, but actually creates a new class when it is called. This is so we can use it in a form definition:

class MyForm(forms.ModelForm):
  company = ReadOnlyModelField(Company)
  
  class Meta:
    model = MyModel

Django AJAX Forms

I think the more Django code I write, the more I like one particular feature.

Forms.

Simple as that. Forms are the reason I keep coming back to django, and discard other web frameworks in other languages, even though I really want to try them.

One pattern I have been using a fair bit, which was touched on in another post, is using AJAX to handle form submission, and displaying the response.

Before we continue, a quick recap on what Django’s forms offer us.

  • A declarative approach to defining the fields a form has, including validation functions.
  • Will render themselves to HTML input elements, as appropriate.
  • Handle validation of incoming form-encoded (or otherwise provided) data.
  • Fields can validate themselves, and can include validation error messages as part of the HTML output.
  • (Model forms) handle instantiation of and updating of model instances.

A normal form-submission cycle contains a POST or GET request to the server, which responds with a fresh HTML page, which the browser renders. The normal pattern for successful POST requests is to redirect to a GET afterwards, to prevent duplicate submission of forms.

By doing an ajax request instead of a full-page request means we can:

  • reduce the amount of data that is sent back from the server
  • improve apparent performance by only re-rendering the relevant data
  • reduce the amount of time spent rendering parts of the page that have not changed, such as menu, etc.

The way I have been doing it, in broad terms, is to have a template just for the form. If the request is an ajax request, then this will be rendered and returned. If it’s not an ajax request, then the full page will be returned.

Some example code, for one way to do this:

def view(request, pk):
  instance = MyModel.objects.get(pk=pk)
  
  if request.is_ajax():
    template = 'form.html'
  else:
    template = 'page.html'
  
  
  if request.method == 'POST':
    form = MyForm(request.POST, instance=instance)
    if form.is_valid():
      form.save()
      if not request.is_ajax():
        return redirect('redirect-name-here')
  else:
    form = MyForm(instance=instance)
  
  return render(request, template, {'form': form})

Our template files. page.html:

{% extends 'base.html' %}

{% block main %}
  {% include 'form.html' %}
{% endblock %}

{% block script %}
{# Assumes jQuery is loaded... #}
{# This should be in a seperate script file #}
<script>
$(document).on('submit', 'form.dynamic-form', function(form) {
  var $form = $(form);
  $.ajax({
    type: form.method,
    url: form.action,
    data: $form.serialize(),
    success: function(data) {
      $form.replace(data);
    }
  });
});
</script>
{% endblock %}

And form.html:

<form action="/path/to/url/" method="POST" class="dynamic-form">
  {% csrf_token %}
  {{ form }}
  <button type="input">Submit</button>
</form>

Obviously, this is a fairly cut-down example, but it gets the message across.

One thing I dislike in general about django is that failed form submissions are returned with a status code of 200: personally I think a 409 is more appropriate in most cases, but returning a 200 actually means this code is simpler.