Schinckel.net

Leveraging HTML and Django Forms: Pagination of Filtered Results

Django’s forms are fantastic for parsing user input, but I’ve come up with a nice way to use them, in conjunction with HTML forms, for pagination, using the inbuilt Django pagination features.

It all stems from the fact that I’ve begun using forms quite heavily for GET purposes, rather than just for POST. Basically, anytime you have a URL that may have some parts of the query string that may need to be built, it’s simpler to use a form element, than to manually build up the url in your template.

Thus, where you may have something like:

<a href="{% url 'foo' %}?page={{ page }}">

It may be better do do something more like:

<form action="{% url 'foo' %}">
  <input type="hidden" name="page" value="{{ page }}">
</form>

Indeed, you can even use named buttons for submission, which will refer to the page. That is the key to the process outlined below.

Django comes with lots of “batteries”, including form handling and pagination. The Class Based Views (CBV) that deal with collections of objects will include pagination, although it is possible to use this pagination in your own views. For simplicity, we’ll stick with a simple ListView.

Let’s begin with that simple view: in our views.py:

from django.views.generic import ListView, DetailView

from .models import Person

person_list = ListView.as_view(
    queryset=Person.objects.all(),
    template_name='person/list.html',
    paginate_by=10,
)

person_detail = DetailView.as_view(
    queryset=Person.objects.all(),
    template_name='person/detail.html',
)

Essentially, that’s all you need to do. You could use implied template names, but I almost never do this. The takeaway from this block is that we are stating the queryset that our ListView will use as the base, the template it should render, and the number of items per page.

I’ve stubbed out the person_detail view, just so we can refer to it in our urlconf, and then in turn in the template. Because of the simplicity of it, we could have just done all of this in our urls.py.

Speaking of our urls.py, we have something like:

from django.conf.urls import url

import views

urlpatterns = [
    url(r'^people/$', views.person_list, name='person_list'),
    url(r'^people/(?P<pk>\d+)/', views.person_detail, name='person_detail'),
]

Then, in our template, we can render it as (ignoring the majority of the page):

<ul class="people">
  {% for object in object_list %}
    <li>
      <a href="{% url 'person_detail' pk=object.pk %}">
        {{ object }}
      </a>
    </li>
  {% endfor %}
</ul>

But this doesn’t give us our pagination. It will only show the first ten results, with no way to access the others. All we need to do to access the others is to append ?page=X, but, as we will see, there is another way.

Typically, your pagination block might look something like:

<ul class="pagination">
  <li>
    {% if page_obj.has_previous %}
      <a href="?page={{ page_obj.previous_page_number }}">
        prev
      </a>
    {% else %}
      <span>prev</span>
    {% endif %}
  </li>

  {% for page_number in paginator.page_range %}
    {% if page_number = page_obj.number %}
      <li class="active">
        <span>{{ page_number }}</span>
      </li>
    {% else %}
      <li>
        <a href="?page={{ page_number }}">
          {{ page_number }}
        </a>
      </li>
    {% endif %}
  {% endfor %}


  <li>
    {% if page_obj.has_next %}
      <a href="?page={{ page_obj.next_page_number }}">
        next
      </a>
    {% else %}
      <span>next</span>
    {% endif %}
  </li>
</ul>

Depending upon your CSS framework, if you use one, there may already be some pre-prepared styles to help you out with this.

This is all well and good, until you want paginated search results. Then, you can no longer rely on being able to rely on using ?page=N, as this would remove any search terms you were already using. Also, if you were using ajax to fetch and display stuff, you may need to use the whole URL, rather than just the query string.

Instead, we can use a Django form for searching, and just add in the pagination bits.

We will build a page that displays an optionally filtered list of people.

Our Person model will be deliberately simple:

from django.db import models

class Person(models.Model):
    name = models.CharField(max_length=256)

Likewise, our form will be simple. All we need to do is have the form able to filter our queryset.

from django import forms

class PersonSearchForm(forms.Form):
    query = forms.CharField(label=_('Filter'), required=False)

    def filter_queryset(self, request, queryset):
        if self.cleaned_data['name']:
            return queryset.filter(name__icontains=self.cleaned_data['query'])
        return queryset

Finally, we will need to subclass a ListView. We’ll mixin from FormMixin, so we get the form-handling capabilities:

from django.views.generic.edit import FormMixin
from django.views.generic import ListView

class FilteredListView(FormMixin, ListView):
    def get_form_kwargs(self):
        return {
          'initial': self.get_initial(),
          'prefix': self.get_prefix(),
          'data': self.request.GET or None
        }

    def get(self, request, *args, **kwargs):
        self.object_list = self.get_queryset()

        form = self.get_form(self.get_form_class())

        if form.is_valid():
            self.object_list = form.filter_queryset(request, self.object_list)

        context = self.get_context_data(form=form, object_list=self.object_list)
        return self.render_to_response(context)

There’s a little bit to comment on there: we override the get_form_kwargs so we pull our form’s data from request.GET, instead of the default.

We also override get, so we filter results if the form validates (which it will if there was data provided). We delegate responsibility for the actual filtering to the form class.

Everything else is just standard.

We will want to actually use this view:

people_list = FilteredListView.as_view(
    form_class=PersonSearchForm,
    template_name='person/list.html',
    queryset=Person.objects.all(),
    paginate_by=10
)

Now we need to render this.

<form id="person-list-filter" action="{% url 'person_list' %}">
  <input name="{{ form.query.html_name }}" value="{{ form.query.value }}" type="search">
  <button type="submit" name="page" value="1">{% trans 'Search' %}</button>
</form>

<div class="results">
  {% include 'person/list-results.html' %}
</div>

You may notice that the search button will result in page=1 being used. This is deliberate.

Our person/list-results.html is just the same as what our person/list.html looked like before, with the addition of the pagination template inclusion.

{% include 'pagination.html' with form_target='person-list-filter' %}

<ul class="people">
  {% for object in object_list %}
    <li>
      <a href="{% url 'person_detail' pk=object.pk %}">
        {{ object }}
      </a>
    </li>
  {% endfor %}
</ul>

Our pagination.html is very similar to how our other template above looked too, but using <button> elements instead of <a>, and we will disable those that should not be clickable. Also, the buttons contain an attribute indicating which form they should be bound to.

<ul class="pagination">
  <li>
    <button
      form="{{ form_target }}"
      {% if page_obj.has_previous %}
        name="page"
        value="{{ page_obj.previous_page_number }}"
        type="submit"
      {% else %}
        disabled="disabled"
      {% endif %}>
      prev
    </button>
  </li>

  {% for page_number in paginator.page_range %}
    <li class="{% if page_number == page_obj.number %}active{% endif %}">
      <button
        name="page"
        value="{{ page_number }}"
        type="submit"
        form="{{ form_target }}"
        {% if page_number == page_obj.number %}
          disabled="disabled"
        {% endif %}>
        {{ page_number }}
      </button>
    </li>
  {% endfor %}

  <li>
    <button
      form="{{ form_target }}"
      {% if page_obj.has_next %}
        name="page"
        value="{{ page_obj.next_page_number }}"
        type="submit"
      {% else %}
        disabled="disabled"
      {% endif %}>
      next
    </button>
  </li>
</ul>

We are getting close now. This will be enough to have clicking on the next/previous or page number buttons resubmitting our search form, resulting in the page reloading with the correct results.

But we can do a bit better. We can easily load the results using AJAX, and just insert them into the page.

We just need one additional method on our View class:

class FilteredListView(FormMixin, ListView):
    # ...

    def get_template_names(self):
        if self.request.is_ajax():
            return [self.ajax_template_name]
        return [self.template_name]

    # ...

and one addition to our view declaration:

people_list = FilteredListView.as_view(
    form_class=PersonSearchForm,
    template_name='person/list.html',
    ajax_template_name='person/list-results.html',
    queryset=Person.objects.all(),
    paginate_by=10,
)

I’ll use jQuery, is it makes for easier to follow code:

// Submit handler for our form: submit it using AJAX instead.
$('#person-list-filter').on('submit', function(evt) {
  evt.preventDefault();

  var form = evt.target;

  $.ajax({
    url: form.action,
    data: $(form).serialize(),
    success: function(data) {
      $('#results').html(data)
    }
  });
});

// Because we are using buttons, which ajax submit will not send,
// we need to add a hidden field with the relevant page number
// when we send our request.
$('#person-list-filter').on('click', '[name=page]', function(evt) {
  var $button = $(evt.target).closest('button');
  var $form = $button[0].form;

  if (!$form.find('[type=hidden][name=page]')) {
    $form.append('<input type="hidden" name="page">');
  }

  $form.find('[type=hidden][name=page]').val($button.val());

  $form.submit();
});

That should do nicely.

There is another thing that we need to think about. If we leave the next/prev buttons, then we need to handle multiple clicks on those buttons, which fetch the subsequent page, and possibly cancel the existing AJAX request.

I do have a solution for this, too, although it complicates things a fair bit. First, we need to add some attributes to the next/prev buttons:

<ul class="pagination">
  <li>
    <button
      form="{{ form_target }}"
      {% if page_obj.has_previous %}
        name="page"
        value="{{ page_obj.previous_page_number }}"
        type="submit"
        data-increment="-1"
        data-stop-at="1"
      {% else %}
        disabled="disabled"
      {% endif %}>
      prev
    </button>
  </li>

  {% for page_number in paginator.page_range %}
    <li class="{% if page_number == page_obj.number %}active{% endif %}">
      <button
        name="page"
        value="{{ page_number }}"
        type="submit"
        form="{{ form_target }}"
        {% if page_number == page_obj.number %}
          disabled="disabled"
        {% endif %}>
        {{ page_number }}
      </button>
    </li>
  {% endfor %}

  <li>
    <button
      form="{{ form_target }}"
      {% if page_obj.has_next %}
        name="page"
        value="{{ page_obj.next_page_number }}"
        type="submit"
        data-increment="1"
        data-stop-at="{{ paginator.num_pages }}"
      {% else %}
        disabled="disabled"
      {% endif %}>
      next
    </button>
  </li>
</ul>

And our click handler changes a bit too:

$('#person-list-filter').on('click', 'button[name=page]', function() {
  var page = parseInt(this.value, 10);
  var $form = $(this.form);
  // Only update the value of the hidden form.
  if (!$form.find('[name=page][type=hidden]')) {
    $form.insert('<input name=page type=hidden>');
  }
  $form.find('[name=page][type=hidden]').val(page);
  // Increment any prev/next buttons values by their increment amount,
  // and set the disabled flag on any that have reached their stop-at
  $form.find('[data-increment]').each(function() {
    this.value = parseInt(this.dataset.increment, 10) + page;
    // We want to disable the button if we get to the 'stop-at' value,
    // but this needs to happen after any submit events have occurred.
    if (this.dataset.stopAt) {
      setTimeout(function() {
        this.disabled = (this.value == this.dataset.stopAt);
      }.bind(this), 0);
    }
  });

  $form.submit();
});

Since this was posted, I have written a number of pages that use this pattern. Some of the improvements that could be made are listed below:

It’s possible to have these results automatically update as the user types. Obviously, this only makes sense if we have AJAX submission happening!

$('#person-list-filter').on('keyup', function() {
  this.submit();
})

If you have lots and lots of results, you probably won’t want to show every button. Often you will see the first few, and a couple either side of the current page (and sometimes the last few). This is almost possible to do with pure CSS, but not quite. I do have a solution for this, but it’s probably worthy of a complete post of its own.

Another situation that is likely to happen is this:

  • User clicks on a page other than page 1 of results. Let’s say page N.
  • User enters text in search field which results in fewer than N pages of results being available.
  • User gets error message.

We can fix this with an overridden method:

class FilteredListView(FormMixin, ListView):
    # ...

    def paginate_queryset(self, queryset, page_size):
        try:
            return super(FilteredListView, self).paginate_queryset(queryset, page_size)
        except Http404:
            self.kwargs['page'] = 'last'
            return super(FilteredListView, self).paginate_queryset(queryset, page_size)

    # ...

You’ll also need to add in a get_prefix() method if you are using an old Django, but really you should just upgrade.

Updated: I’ve added in some more error checking into the templates, to prevent exceptions when attempting to render previous and next page links (thanks inoks).

Updated: I’ve changed to use the preferred urlpattern syntax. (thanks knbk).

Updated: Delegate to the form for filtering. Add discussion of other extensions. Add button[form] attributes.