django-socialregistration and 'closed' sites.

I develop a ‘closed’ system. It isn’t that not just anyone can use it - hell, we’d love to have more customers, but the users of the front-facing site (myROSS) are not the same as our customers, who use the backend indirectly through our client application.

The outcome of this is that we do not allow for registration on myROSS. The only way you can become a user of myROSS is if your company uses The ROSS System, and you have been added by your manager, franchisee or whatever.

However, I have been looking at django-socialregistration a bit lately: it is a very clean way to allow users to register and then login using Facebook, Twitter OAuth, and OpenID. Since one of the reasons that many of our ‘users’ do not use myROSS as much as we would like is because it is another password to remember, and also because most of our users are younger employees of Quick Service Restaurants, we are hoping that allowing them to use their Facebook or Twitter credentials will mean the barrier to use of myROSS is reduced.

So, I would like to be able to use socialregistration to allow currently registered users to connect their myROSS accounts to one or more of their other accounts (Twitter, Facebook, OpenID), and then allow them to use that as a means of authentication. But at the same time, I want to prevent non-current users from registering (or more to the point, prevent a current user from accidentally creating a duplicate account).

The first part is already handled by socialregistration: when a logged in user clicks on a link that connects to Twitter, for instance, and then approves the connection on the Twitter website, their account is then associated. Data is stored in the myROSS database noting which twitter account id matches which myROSS account. They can then log in using Twitter. I assume the same happens with Facebook (which I don’t use), and OpenID (which I haven’t tried yet - although I do use it for other sites).

What I need to do is prevent non-logged in users from registering.

The way socialregistration’s workflow works is that when a non-logged in user authenticates with the outside authentication system, they are redirected to a setup page, which allows them to either create a username, or generates a username for them. We need to just hook in at this time, and instead of asking for a new username/email address, ask them to enter their current myROSS credentials. At that point, we can store the information as is done if they were already authenticated.

Rather than hack the socialregistration source, we can use the hooks that the authors there have kindly left us. Specifically, we only need to override one class, the UserForm. We can do this in the view, by changing the arguments passed in to the ‘setup’ view.

If you copied-and-pasted the contents of the socialregistration file, then you can just modify the declaration of the ‘socialregistration_setup’ url route, and add in the new form class. Otherwise, you can just override that url route. I included the whole socialregistration.urls, so now the relevant section of my project looks like:

1 # routes for socialregistration: note the override of the first one.
2 (r'^auth/setup/$', 'socialregistration.views.setup', {'form_class':auth_additions.forms.ExistingUserForm}),
3 (r'^auth/', include('socialregistration.urls')),

There is also an import of my auth_additions.forms module.

Now, the exciting bit: what goes in the ExistingUserForm class?

 1 from django.contrib.auth.forms import authenticate
 2 from django.utils.translation import gettext as _
 3 from django import forms
 5 class ExistingUserForm(forms.Form):
 6     username = forms.CharField()
 7     password = forms.CharField(label=_("Password"), widget=forms.PasswordInput)
 9     def __init__(self, user, profile, *args, **kwargs):
10         super(ExistingUserForm, self).__init__(*args, **kwargs)
11         self.user = user
12         self.profile = profile
14     def clean(self):
15         username = self.cleaned_data.get('username')
16         password = self.cleaned_data.get('password')
18         if username and password:
19             self.user = authenticate(username=username, password=password)
20             if self.user is None:
21                 raise forms.ValidationError(_("Please enter a correct username and password. Note that both fields are case-sensitive."))
22             elif not self.user.is_active:
23                 raise forms.ValidationError(_("This account is inactive."))
25         return self.cleaned_data
27     def save(self, request=None):
28         self.profile.user = self.user
30         return self.user

Django templates not loading?

Had an annoying issue today after upgrading an installation of django to the trunk. All of a sudden, my admin interface would not load. I had errors stating that the template admin/login.html could not be loaded.

Now, django.contrib.admin was in settings.INSTALLED_APPS, and django.template.loaders.app_directories.Loader was in settings.TEMPLATE_LOADERS.

So, why was django throwing an exception? To find out, I stepped into a shell:

1 >>> from django.template.loader import find_template_loader
2 >>> loader = find_template_loader('django.template.loaders.app_directories.Loader')
3 >>> loader.load_template('admin/login.html')

This was where I realised something was wrong. I don’t get the error now (as I have fixed it), but it complained about being not allowed to open the file. As in a permissions error.

Looking up the location in a new shell, I was able to see that all of the files in the django.contrib.admin.templates directory were only able to be read by root. For some reason, python install had set the mode of these files to 0600, instead of the expected 0644. A quick sudo chmod -r ag+r templates (from inside the django.contrib.admin directory) fixed it.

Dvorak and VMWare Fusion

I made the choice a year or so ago to go Dvorak: I was on holidays and I spent the time I was free doing a typing tutor. I really liked how quickly I was able to make the transition. Most words and sentences can be written with little movement of the fingers off the home row. I also use the Programmers Dvorak layout, including swapping the orientation of the numeric keypad to be more like a phone, not a calculator. This was the only part I needed to actually swap the keycaps on my keyboards around on, as I touch-type on the main keyboard, but tend to look a bit more when entering values.

I have noticed that quite a few applications don’t handle a different keymap that well. It appears they are using the keycodes, which are independent of the values that should be presented to the operating system. I can understand why this is in many cases: for games for instance, you want to be able to continue to walk around using the aswd keys in World of Warcraft regardless of the layout of the keymap. I guess it makes a bit of sense to pass through this stuff to the virtual machine too: after all, it has its own keymap to interpret.

With VMWare Fusion, however, the translation of Mac Shortcuts to Windows Shortcuts is slightly broken. Because on my keymap, the ⌘-keys match their keycode (didn’t want to re-learn all of the shortcut keys), then they are not passed through correctly. Luckily, they are editable in the VMWare Fusion preferences. Mine now look like this:

Now, I can hit ⌘S in notepad, under Windows, and it saves my document. And I can use the cut/copy/paste/undo shortcuts again!

Django Management.tmbundle

Did some work on my Django Management.tmbundle last night.

It now handles running tests when (a) The apps are not directly in the project root, but inside another folder, for instance; and (b) the app/ file has been split into seperate files.

The main reason I made this was so that I could run tests and have clickable links in the results window for the location of failing tests.

There is still much to do on this. I am considering re-writing it in python rather than ruby, so I can programmatically find the app name, rather than guess it. I also want to refactor the hell out of it and make it much nicer.

Anyway, if you are interested, you can find the most recent version at - and I think it also appears in TextMate’s getBundles bundle.

Typing the ° symbol on iPhone.

For future reference: press and hold 0, and the ° will appear.

On Django and Check Constraints

I’ve come to love Django, and now I seem to be spending not only my work time, but also my own time working on one django project or another. Today at work I came across the need to have check constraints in a database. For instance, this can be used to ensure that the start of an event is before the end of the event, or that a certain value has a particular range of values.

For the SQL savvy, it can look like:

…more columns here…
start datetime,
finish datetime,
count integer CHECK (count >= 0),
CONSTRAINT event_starts_before_finish CHECK (start < finish)

There are two types of constraint here: the column-level constraint, which in this instance can be done with django, and the table-level constraint, which in many systems cannot be a part of a column definition. In PostgreSQL, my DBMS of choice, it is possible to define a constraint that affects more than one column as part of a column, but I prefer not to. Note also that I am using the syntax form that allows me to name the constraint. This means I can alter or drop it later.

As I mentioned, django can do the first type of constraint, as long as it is a > 0 constraint, using one of the field types that subclasses PositiveInteger. However, there is no functionality built into django to do the latter. And I would like to use them.

It is possible to just add the constraints to an already existing database: indeed, that is what I did for work. Having the constraints at the database level means that, since I have more than one interface to my datastore (don’t ask: one of them is for an old SOAP interface I need to keep around), I want to ensure that even if someone accesses it outside of django, they cannot put in data that breaks these constraints. Similarly, if I use an interface other than the admin interface, or heaven forbid, open up the database in raw form, I cannot accidentally put in data that breaks this validation.

But, pure database level constraints don’t give very nice feedback in django. It is nicer to have the pretty red boxes around my field than the traceback. So, I want the validation to occur on the field level as well. As long as I am using django’s forms (and my API for RESTful access will use them for validation), then I will have these errors nicely presented.

So, to that end, I have created some code that allows for the definition and enforcement of both of these types of constraint.

The column form allows for a new keyword argument to a field:

count = models.IntegerField(constraint=">= 0")

Notably, the string that can be passed in must conform to the pattern “cmp value”, where cmp is one of the comparison operators (<, >, <=, >=, =, !=), and value is a valid value for this column type. It will be passed to the to_python() method of this field when comparing. There must be a string between the two parts.

The other form is a new attribute on the Meta class of a model.

check_constraints = (‘start < finish’,)  

This must be a tuple of strings, where a string is of the form “column cmp column”. Again, there must be a space either side of cmp, and each column name must be a valid column. Not meeting these criteria will result in a validation error.

From these definitions, the database constraints will be applied, and validation of forms will also occur.

I have deliberately made the constraints simple (ie, not callable objects) so that they can easily be converted to database constraints. For instance, they can effectively be transferred straight through to the database (with the addition of the column name in the case of the column constraint).

I am going to create a ticket in the django trac, and submit a patch. Guessing I should write up some test cases, though!

Setup Django with Passenger Prefpane

I am loving Django for web development. I didn’t have it set up to serve my (development) projects automatically until just now.

I had installed the Passenger Prefpane, which greatly simplifies the management of VirtualHost-based serving of sites, at least for Rails and other ruby-based frameworks. With a little work, you can use the same setup to serve Django projects.

Rather than re-detail the setup, I’ll just point you to the mod_passenger setup, and the Passenger Prefpane setup pages.

Now, to set up a Django project: obviously you need a django project. Create one, and note where it is located. I stick all of mine in ~/Sites.

Add a file to the root of this project, called It needs to contain the following data:

1 import os, sys
2 sys.path.append('/Users/matt/Sites') # Replace with your directory
3 os.environ['DJANGO_SETTINGS_MODULE'] = 'testing.settings' # replace with your projectname.
4 import django.core.handlers.wsgi
5 application = django.core.handlers.wsgi.WSGIHandler()

Now, add the site to the Passenger prefpane. My site is the testing.local site:

Now visit the address and ensure that it works. You should get the basic you need to set up django message.

To get the admin media served by the standard apache setup, I created a link inside the /Library/WebServer/Documents directory to /Library/Python/2.5/site-packages/django/contrib/admin/media. This can be done inside the Terminal:

sudo ln -s /Library/Python/2.5/site-packages/django/contrib/admin/media /Library/WebServer/Documents/admin-media/

Then, change the ADMIN_MEDIA setting in your django projects to http://localhost/admin-media/. This is probably the weakest point in the setup, as it will only work for pages served to your machine, not others on your network.

1 # URL prefix for admin media -- CSS, JavaScript and images. Make sure to use a
2 # trailing slash.
3 # Examples: "", "/media/".
4 ADMIN_MEDIA_PREFIX = 'http://localhost/admin-media/'

I had some issues with mod_passenger serving the data for localhost (and arne, my actual machine’s name) from the first installed VirtualHost. To overcome this, I put in a new file into /etc/apache2/other/localhost.conf, which looks like:

1 <VirtualHost *:80>
2   DocumentRoot "/Library/WebServer/Documents"
3   <directory "/Library/WebServer/Documents">
4     Order allow,deny
5     Allow from all
6   </directory>
7 </VirtualHost>

This forces unnamed, or other sites to work as intended. Including the /User/*/Sites directories.


Prowl is awesome. Growl notifications can be forwarded to your iPhone.

But you can get notifications from anywhere. A Perl script is included, but that didn’t work on my server. So I wrote one in Ruby:

 1 #! /usr/bin/ruby
 3 # A ruby class for sending notifications to Prowl.
 5 require 'uri'
 6 require 'net/http'
 7 require 'net/https'
 9 class String
10   def urlencode
11     gsub( /[^a-zA-Z0-9\-_\.!~*'()]/n ) {|x| sprintf('%%%02x', x[0]) }
12   end
13 end
15 class Hash
16   def urlencode
17     collect { |k,v| "#{k.to_s.urlencode}=#{v.to_s.urlencode}" }.join('&')
18   end
19 end
21 class Prowler
22   def initialize user, pass
23     @url = URI.parse('')
24     @username = user
25     @password = pass
27     @http =, @url.port)
28     @http.use_ssl = true
29   end
31   def send_notification app, evt, desc
34     options = {
35       'application' => app,
36       'event' => evt,
37       'description' => desc
38     }
40     req ="#{@url.path}?#{options.urlencode}")
41     req.basic_auth @username, @password
42     @http.request(req)
43   end
45 end
47 # How to use?
48 # p ='username', 'password')
49 # p.send_notification('App','Event','Desc')


Started up a new iPhone app today. The logo at the bottom of the screen looked a bit familiar:

Here is the logo from the Adelaide City Council (which has been in use for about 7 years):

Wonder which came first…

If I overlay one one the other, we see they are not identical:


Fix Redmine not showing repository data

For some time, the git repository I am using to track changes for my Django-Management TextMate bundle was not working correctly in Redmine. I was able to connect it up using the settings, but was getting errors about the files not being found.

It turns out that git was not in my path. Putting a link from /usr/local/bin/git to /usr/bin/git fixed that all up.