KnockoutJS HTML binding

TL;DR: Don’t use KnockoutJS html binding lots of times in your page.

I’m in the middle of rewriting a large part of our application in HTML: for a lot of the interactivity stuff, anything more than just a simple behaviour, I’m turning to KnockoutJS.

Mostly, it’s been awesome. Being able to use two-way binding is the obvious big winner, but dependency tracking is also fantastic.

However, I have had some concerns with performance in the past, and this was always on my mind as I moved into quite a complicated part of the system.

Our approach is that we are not creating a single page application: different parts of the system are at different URLs, and visiting that page loads up the relevant javascript. This is a deliberate tradeoff, mostly because for the forseeable future, our software will not work without a connection to our server: most of the logic related to shift selection is handled by that. We aren’t about to change that.

While rewriting the rostering interface, I initially had Django render the HTML, and I added behaviours. This was possible, and quite fast, however as the behaviours became more complex, I was doing things like sending back scripts that caused other parts of the page to refresh themselves. It was all rather fragile.

So, I went back to KnockoutJS. After a while, I noticed significant slowdowns when dealing with pages that really shouldn’t have been that slow. I’d optimised the database access for the fetching of shifts (and indeed, it is much faster than before), but it felt like Knockout was very sluggish.

I do have quite a few ko.computed() objects, perhaps they were slowing it down? Notably, the function that filters which shifts should be shown where on the page.

So I put some console.time()/timeEnd() calls in place.

Nope: the initial parse of the data runs in less than half a millisecond: instantiating the objects took a while, but the filtering of shifts was taking much less than 100ms.

However, the initial call to ko.applyBindings() was taking several seconds.

The most annoying thing was that when the developer tools were open, it was taking far, far longer!

Eventually, through using the developer tools profiling, I discovered that the slowdown was because of repeated code like:

foo.innerHTML = bar;

Initially, I had thought this slowdown was in KnockoutJS itself, and played around with other ways of binding (such as using the knockout-repeat plugin). Still slow.

Eventually, however, I worked out that it was the act of interacting with the DOM in this manner that was slow. More specifically, the assignation to innerHTML was occurring in the html: binding.

Looking through my source code, I discovered code that looked like:

<span data-bind="html: icon"></span>

And, icon contained the HTML I wanted to put in there:

<i class="icon-ok"></i>

Which was a bad idea to begin with: it conflated UI with data to begin with. So, I replaced the code that looked like:

this.icon = '<i class="icon-ok"></i>';

With:

this.icon = {
  'icon-time': true
};

And then, in the HTML:

<i data-bind="css: icon"></i>

Bingo. All of a sudden, a page that took several seconds to re-render does so in around a second.

It’s important to note that this pattern was repeated several times for each shift: and we have possibly dozens of shifts on a page. When you really need to use the html binding that’s fine, just don’t stick it inside a loop (or worse still, inside a nested loop).

KnockoutJS persistence using Simperium

I really like KnockoutJS. I’ve said that lots of times, but I mean it. It does one thing, two-way bindings between a data model and the GUI elements, really well.

Perhaps my biggest hesitation in using it in a big project is that there is no built-in persistence layer. This would appear to be a situation where something like Backbone has an advantage.

And then, last week, I came across Simperium.

“So,” I thought, “what if you were able to transparently persist KnockoutJS models using Simperium?”

// Assume we have a SIMPERIUM_APP_ID, and a logged in user's access_token.
var simperium = new Simperium(SIMPERIUM_APP_ID, {token: access_token});
// mappingOptions is a ko.mapping mappingOptions object: really only useful
// if your bucket contains homogenous objects.
var store = new BucketMapping(simperium.bucket(BUCKET_NAME), mappingOptions);

var tony = store.all()[0];

var alan = store.create({
  name: "Alan Tenari",
  date_of_birth: "1965-02-06",
  email: "alan.tenari@example.com"
});

Now, tony is an existing object we loaded up from the server, and alan is one we just created.

Both of these objects are mapped using ko.mapping, but, this is the exciting bit, every time we make a change to any of their attributes, they are automatically persisted back to simperium.

There is a little more to it than that: we may want to only persist valid objects, for instance.

This totally gets me excited. And, I’ve already written a big chunk of the code that actually does this!

But for that, you’ll just have to wait…

Re: HTML is not XAML

This is a response to MVVM on MVC: HTML is not XAML. I attempted to post a comment, first from my iPad, but after a few characters the textarea became unresponsive. Then, from my iMac, I was able to enter a comment, but not post it. It seems like something weird is happening when it shows me the CAPTCHA, and it dismisses the dialog before I can do anything. Disabling JavaScript prevents me from commenting. (But to be honest, using DISQUS does the same).

I’ll start by saying that I’m not a Silverlight developer, indeed, I do nothing with any part of the Microsoft development stack. I don’t think I even have any Microsoft software installed on either my work or home machines. I have been doing a lot of stuff with KnockoutJS lately, though.

Jeremy makes some valid points about designerdeveloper interactions. Maybe I’m (un)lucky, but my interactions with a designer seem to be that (s)he provides me with an image, and I make the HTML match. Either that, or I do the design work myself. In that case, I design in the browser. Safari’s inspector allows you to alter CSS rules and view their impact live. This also means that my HTML is always as sparse as I can possibly make it.

Before I get to the main point, regarding using bindings inside the HTML, there is one thing I just need to point out. Jeremy has the code:

<div id="menuContainer">
  <ul id="menu">
    <li data-bind="foreach: menuItem">
      <span data-bind="text: name">Name</span>
    </li>
  </ul>
</div>

This would only create one <li> element, with multiple <span> elements. In addition, the Name text is superfluous, and would be replaced when the bindings were applied. To match the intent of the previously listed code, I think he meant (and I’m spelling some stuff differently):

<nav class="main-menu">
  <ul data-bind="foreach: menuItems">
    <li>
      <a data-bind="text: name, attr: {href: url}"></a>
    </li>
  </ul>
</nav>

Jeremy then goes on to discuss a way to have all of these bindings applied using code.

Personally, being able to bind the data declaratively is one thing that really draws me to KnockoutJS. It’s easy to see the HTML structure, and what data is bound to them. In fact, in some ways it reminds me lots of Cocoa Bindings.

One of his beefs is that designers may muck with the code. I think this could be easily remedied by a little education: don’t touch anything that has data-bind="...". This really isn’t that different to don’t touch anything’s id.

But a deeper problem is that by adding the bindings in code means that you can’t see from the HTML what flow control will be used to handle the layout. Assuming you are still able to apply the foreach binding to the ul in the example above if it had an id instead, it’s not obvious that there may be multiple items. Maybe that’s not the greatest example, as it is a list, so there probably will, but foreach can be used anywhere.

And there are more, too: if allows you to have bits that are not rendered (significantly different to just making them invisible). Plus, if you use the with binding, then you would need to keep in your head the nested structure of what you are accessing inside the with block. Do it in HTML, and you can see (assuming you have reasonable indenting).

Jeremy seems to come to the agreement (in one of his comments), that having the application of the bindings in the code makes things even more complicated, but I propose that it even makes them more brittle. No longer are you relying just on the names of the ViewModel’s attributes, but you are also relying on the ids of the HTML elements. And this is the kicker: a binding to a name that no longer exists in the ViewModel will fail when you try to view the page, meaning nothing will work (and you’ll see a nice red message in your console). What will the following code do if there is no matching HTML element?

$('#spelled-this-id-worng').attr('data-bind', 'value: theValue');

It does nothing.

But it does it silently.

Styling radio buttons like a boss segmented button

I quite like the concept of segmented buttons, where you have a list of related buttons, and can select one or more of them. In MacOS X, and iOS, the ones that are selected have a nice indented look.

I’m currently working on a GUI framework for KnockoutJS, and today I had reason to use this type of control. Initially, I had the following markup that I intended to style:

<nav>
  <ul class="segmented">
    <li><a>Organisation</a></li>
    <li><a>Users</a></li>
    <li><a>Units</a></li>
    <li><a>Tags</a></li>
  </ul>
</nav>

But, then it occurred to me that HTML radio buttons are a good fit for this use case. They can be set so that only one of them will be selected, which means you can actually get them to work without using any JavaScript to keep selected status in sync. And the bonus is that the labels will be clickable, so we don’t need JavaScript for associating them with the radio buttons.

<nav class="segmented">
  <input type="radio" name="seg-1" value="Organisation" id="seg-Organisation">
  <label for="seg-Organisation">Organisation</label>
  <input type="radio" name="seg-1" value="Users" id="seg-Users">
  <label for="seg-Users">Users</label>
  <input type="radio" name="seg-1" value="Units" id="seg-Units">
  <label for="seg-Units">Units</label>
  <input type="radio" name="seg-1" value="Tags" id="seg-Tags">
  <label for="seg-Tags">Tags</label>
</nav>

Now, there’s slightly more markup, but that’s okay. So, what does that look like?

Hmm, not quite what we want. We’ll actually want to hide the radio button widgets, and style the labels. Rather than try to have this in the page, here’s one I prepared earlier:

As you can see, this is with no JavaScript.

Obviously, this is fairly crappy styling: it just looks like some Windows buttons. Let’s tart it up a bit. This is the default styling for koui:

As a bonus, I’ve disabled one of the elements.

From the perspective of KnockoutJS, we can use the checked binding applied to the radio buttons to see which one is selected. If you were submitting this inside a form, you may want to not use display: none; on them, as they may not submit under certain browsers. For ko, however, it’s fine.

I’m going to be using this technique within koui: for segmented buttons, which I haven’t worked out a nice way to define bindings for, and for the tab_view binding, which associates these buttons with a view below, containing a choice of data based on the selection.

Update: There is one drawback with the technique I used here. It is detailed at How to fix the broken iPad form label issue.

It’s actually a rather simple solution: All you need to do is stop the propagation of the event. I had played around with re-firing the event onto the input element, but that might fire twice in some browsers, which might be bad (especially if you were using checkboxes, rather than radio buttons!)

KnockoutJS dirty extender.

Ryan Niemeyer is the man with respect to most things KnockoutJS, and I had been using a version of his smart dirty flag in some projects. I recall making it so it didn’t have to bind to a secondary property, but I may be mistaken.

Anyway, with Knockout 2.0, we get extenders. Now, it’s possible to do things like:

var thing = ko.observable(null).extend({dirty: true});

It will then look for ko.extenders.dirty, and call that function with two arguments: the observable, and the argument (in this case, true).

Thus, it is possible to re-implement his dirty flag as an extender:

ko.extenders.dirty = function(target, startDirty) {
  var cleanValue = ko.observable(ko.mapping.toJSON(target));
  var dirtyOverride = ko.observable(ko.utils.unwrapObservable(startDirty));
  
  target.isDirty = ko.computed(function(){
    return dirtyOverride() || ko.mapping.toJSON(target) !== cleanValue();
  });
  
  target.markClean = function(){
    cleanValue(ko.mapping.toJSON(target));
    dirtyOverride(false);
  };
  target.markDirty = function(){
    dirtyOverride(true);
  };
  
  return target;
};

The advantage I think mine has over Ryan’s is that you can mark an observable as dirty (thing.markDirty()), and it will stay dirty until you explicitly mark it as clean (thing.markClean()).

Otherwise, it’s just: thing.isDirty() and you are all good.

Alternatively, you could remove the two helper functions, and implement .isDirty() as a writeable observable, that tests the incoming value and sets the internal cleanValue if it needs to.

HSV to RGB in JavaScript

I am writing a set of UI widgets for use in web apps, using the excellent KnockoutJS library. One of the more challenging ones has been the colour picker. Rather than do what everyone else has done, I tried to ape the Apple Colour Picker. But this gives us values in HSV, which aren’t that useful for web.

So I came across a page that has a JavaScript HSV to RGB converter: http://jsres.blogspot.com/2008/01/convert-hsv-to-rgb-equivalent.html. And there are so many things wrong with that code that it hurts.

  • the declared variables r,g,b are not used at all.
  • RGB is defined as an Array, but used as an Object.
  • var_r and friends are not declared, and pollute the global namespace.

Plus, more I came across as I worked through the code.

So, I thought I’d clean it up, and make it a bit easier to follow.

Some of the bits that are ‘tricky’ are the use of toString(16), which converts a number to a base 16 representation, and the ("0" + value).slice(-2), which zero-pads a string.

The algorithm itself is fairly easy to follow: there are seven possible cases for the data conversion. If the saturation is 0, then RGB is #000000.

Otherwise, the value depends on the value of Math.floor(h/60). There is a simple lookup table (data), which stores three values based on hue, saturation and value, and then it’s just a matter of picking the correct two to use with the value, and returning that.

Binding to head elements in KnockoutJS

By default, KnockoutJS binds to <body>, by the look of things. If you have something like:

<html>
	<head>
		<title data-bind="text: title"></title>
	</head>
	<body>
		...
		<script>
			var vm = {
				title: ko.observable("This is the page title")
			};
			ko.applyBindings(vm);
		</script>
	</body>
</html>

The title will not be bound. Instead, you’ll need to use (and I’m using jQuery):

<html>
	<head>
		<title data-bind="text: title"></title>
	</head>
	<body>
		...
		<script>
			var vm = {
				title: ko.observable("This is the page title")
			};
			ko.applyBindings(vm, $('html')[0]);
		</script>
	</body>
</html>

You could also apply the binding twice, once to the head, and once to the body.

django and jQuery templates

KnockoutJS is a great way to create relationships between data objects, and interface elements. You can, for instance, bind a date value to an html input[type=date] element, and have it converted into a proper date object. You could then display data based on this, or do anything else you wanted.

KnockoutJS 1.2 (the currently stable version) defaults to using jQuery templates (jQuery-tmpl), which happen to use conflicting syntax to django templates.

For instance, if you were to have the following in your django template file:

{{if foo > bar }}
  <div>Stuff Here</div>
{{/if }}

Then django would attempt to process that, as it uses bits that look like django’s template engine’s value placeholder.

A workaround to this is to look at doing something like wrapping any jQuery templates in something that prevents django from interpreting it.

But I don’t like that solution. For starters, almost every text editor will try to syntax highlight data between <script> tags as javascript, even when explicitly marked as <script type="text/html"> or any other non-javascript mime type.

So, it would be nicest (and cleanest) to be able to have each jQuery template item in a separate file in my project.

Enter {% jquery_template %}. With a custom django templatetag, you can not only have it include the template in your django template, but it will automatically add the script tags, and even add an id.

For instance, you can do: <div class="highlight"><pre>{% jquery_template 'path/to/template.html' 'templateName' %} </pre> </div>

This will include the data from path/to/template.html, which it finds in any template location, but wrapped in <script type="text/html" id="templateName">.

I have a django app that contains this template tag, as well as some other useful stuff for jquery, and other javascript stuff (including knockoutjs). You can see this template tag at: jquery_template.py.

Hope it’s useful.

Knockout Collection

I am loving KnockoutJS. It makes it super easy to bind data values to UI elements in a declarative manner. You no longer have to worry about callbacks updating your data model and/or your view widgets.

The addition to KnockoutJS that I have been working on is a ‘collection’, that can be used to contain a set of objects, which can be fetched from a server, and each of which has it’s own resource URI that will be used to update or delete it.

For instance, we may have a collection URI:

GET "http://example.com/people/"

When we access this using a GET request, we might see something like:

 1 [
 2   {
 3     "first_name": "Adam",
 4     "last_name": "Smith",
 5     "links": [
 6       {"rel":"self", "uri": "http://example.com/people/552/"}
 7     ]
 8   },
 9   {
10     "first_name": "John",
11     "last_name": "Citizen",
12     "links": [
13       {"rel":"self", "uri": "http://example.com/people/32/"}
14     ]
15   }  
16 ]

Each linked resource contains the full (or as much as the logged-in user is able to see) representation. Example:

GET "http://example.com/people/552/"
1 {
2   "first_name": "Adam",
3   "last_name": "Smith",
4   "date_of_birth": "1910-02-11",
5   "email": "adam.smith@example.com",
6   "links": [
7     {"rel":"self", "uri": "http://example.com/people/552/"}
8   ]
9 }

Now, this is just the beginning. Obviously, we want to turn all of these fields into observables. I also wanted to know when any data had changed (so the “Save” button can be disabled when the object is not dirty). Clearly, being able to write the data back to the server, as well as create new objects, and delete them. Further, I needed to be able to do conditional reads and writes (only allow the object to be saved if no-one else has touched it since we last fetched it).

The place where the ko.mapping plugin broke down for me was that updating the resource from the full representation didn’t add the new fields that came back from the server. It may be that indeed this is possible (I think it is), but at the time, I could not see how to do this. It may be that I will rewrite this to use the ko.mapping stuff, but I’m not so sure right now.

Anyway, after a couple of revisions, I have a working framework.

To use it, you can just do:

 1 // Add a dependentObservable called 'name'.
 2 var processPerson = function(item) {
 3   item.name = ko.dependentObservable(function(){
 4     return item.first_name() + ' ' + item.last_name();
 5   });
 6 };
 7 
 8 var people = ko.collection({
 9   url: "http://example.com/people/",
10   processItem: processPerson
11 });

There is one main caveat at this stage:

  • It is expected that each object will have a ‘name’ property. If your server does not return one, you’ll need to setup a dependentObservable as shown in processPerson above.

First, the ko.collection object:

  1 ko.collection = function(options) {
  2   // Let jQuery know we always want JSON
  3   $.ajaxSetup({
  4     contentType: 'application/json',
  5     dataType: 'json',
  6     cache: false // This is browser cache! Needs to be set for Firefox.
  7   });
  8   
  9   options = options || {};
 10   var url = options.url;                  // Allow passing in a url.
 11   var processItem = options.processItem;  // Allow passing in a function to process each item after it is fetched.
 12   var etag;
 13   
 14   
 15   // Initial setup. We need to set these early so we can access them, even
 16   // if we have no data for them.
 17   var self = {
 18     items: ko.observableArray([]),
 19     selectedItem: ko.observable(null),
 20     selectedIndexes: ko.observableArray([]),
 21     filters: ko.observable({})
 22   };
 23   
 24   /*
 25   Message handling.
 26   
 27   We have a messages observableArray, but we use this dependent observable
 28   to access it. This allows us to have messages that expire.
 29   
 30   self.messages() => provide access to the array of messages.
 31   self.messages({
 32     type: "error|notice|warning|whatever",    => This will usually be used to apply a class
 33     message: "Text of message",               => This text will be displayed
 34     timeout: 1500                             => If this is non-zero, message expires (and 
 35                                                  is automatically removed after this many 
 36                                                  milliseconds)
 37   });
 38   
 39   Every message object gets given a callback function (.remove()), that,
 40   when executed, well immediately remove that message, and get rid of the
 41   timer that normally removes that message after timeout.
 42   
 43   The messages object is also given a flush() function, that will remove
 44   all of the messages within it.
 45   
 46   Not sure if I should move this to a seperate plugin?
 47   */
 48   var messages = ko.observableArray([]);
 49   self.messages = ko.dependentObservable({
 50     read: function() {
 51       return messages();
 52     },
 53     write: function(message) {
 54       var timeout;
 55       message.remove = function() {
 56         messages.remove(message);
 57         clearTimeout(timeout);
 58       };
 59       messages.remove(function(item) {
 60         return item.message === message.message;
 61       });
 62       messages.push(message);
 63       if (message.timeout) {
 64         timeout = setTimeout(function(){
 65           messages.remove(message);
 66         }, message.timeout);
 67       }
 68     }
 69   });
 70   self.messages.flush = function() {
 71     $.each(messages, function(message){
 72       message.remove();
 73     });
 74   };
 75     
 76   /*
 77   filteredItems : a subset of self.items() that has been passed through
 78                   all of the self.filters(), and selects only those that
 79                   match. A filter must be an object of the form:
 80                   {
 81                     value: ko.observable(""),
 82                     attr: "name",
 83                     test: function(test_value, obj_value) {}
 84                   }
 85                   
 86                   The filtering code handles getting the correct values to
 87                   pass to the test function, the attr is the name of the 
 88                   attribute on each member of self.items() that will be
 89                   tested.
 90                   Having 'value' passed in means we can have a default
 91                   value when app starts.
 92   */
 93   self.filteredItems = ko.dependentObservable(function() {
 94     var filteredItems = self.items();
 95     $.each(self.filters(), function(name, filt){
 96       filteredItems = ko.utils.arrayFilter(filteredItems, function(item){
 97         if (!filt.attr || !item[filt.attr]) {
 98           return true;
 99         }
100         return filt.test(filt.value(), item[filt.attr]());
101       });
102     });
103     return filteredItems;
104   });
105   
106   /*
107     This is really only used by a select[multiple] object, and is used in
108     conjunction with selectedIndexes.
109     
110     TODO: make this a writeable dependentObservable.
111   */
112   self.selectedItems = ko.dependentObservable(function() {
113     return self.items().filter(function(el){
114       return $.inArray(self.items().indexOf(el), self.selectedIndexes()) >= 0;
115     });
116   });
117   
118   /*
119     Filter self.items() finding only those that have at least one attribute
120     that is marked as dirty.
121   */
122   self.dirtyItems = ko.dependentObservable(function() {
123     return self.items().filter(function(el){
124       return el.isDirty();
125     });
126   });
127   
128   /*
129     Filter self.items(), finding only those that have at least one attribute
130     marked as conflicted.
131   */
132   self.conflictedItems = ko.dependentObservable(function() {
133     return self.items().filter(function(el){
134       return el.hasConflicts();
135     });
136   });
137   
138   self.setSource = function(newUrl) {
139     url = newUrl;
140   };
141   
142   /*
143     Fetch all items from the url we have for the index.
144     
145     It is allowable that the index does not return the full body of each
146     item, but instead only contains perhaps a name, and links for that
147     item. Then, we can use self.selectedItem().fetch() to get the full
148     data for the item.
149   */
150   self.fetchItems = function() {
151     if (!url) {
152       return;
153     }
154     var headers = {};
155     if (etag) {
156       headers['If-None-Match'] = etag;
157     }
158     $.ajax({
159       url: url,
160       type: "get",
161       headers: headers,
162       statusCode: {
163         200: function(data, textStatus, jqXHR) {
164           // Successful. If we already had objects, then
165           // we need to update that list.
166           $.each(self.items(), function(i, item){
167             // Is there an item in the new data items list that matches
168             // the item we are now looking at?
169             var matchingItem = data.filter(function(el){
170               links = el.links.filter(function(link){
171                 return link.rel==="self";
172               });
173               return links[0] && links[0].uri === item._url();
174             })[0];
175             if (matchingItem) {
176               // Update the item that matched.
177               item.updateData(matchingItem);
178               if (processItem) {
179                 processItem(item);
180               }
181               // Remove from data.
182               data.splice(data.indexOf(matchingItem), 1);
183               // Not sure if this should be here.
184               // item.isDirty(false);
185             } else {
186               // Not found in incoming data: remove from our local store.
187               // Will this break $.each(self.items(), ...) ?
188               self.items.remove(item);
189             }
190           });
191           
192           // Any items that we have left in data (which will be all if we
193           // haven't loaded this up before) now need to be added to items().
194           // On a clean fetch, this will be the first code that is run.
195           $.each(data, function(i, el){
196             var item = ko.collectionItem(el, self);
197             if (processItem) {
198               processItem(item, el);
199             }
200             self.items.push(item);
201           });
202           
203           // Finally, update the etag.
204           etag = jqXHR.getResponseHeader('Etag');
205         }
206       }
207     });
208   };
209   
210   /*
211     A shortcut method that allows us to bind an action to fetch the
212     data from the server for the currently selected item.
213   */
214   self.fetchSelectedItemDetail = function(evt) {
215     if (self.selectedItem && self.selectedItem()) {
216       self.selectedItem().fetch();
217     }
218   };
219   
220   /*
221     Create an item. I haven't implemented this yet, because I haven't 
222     figured out a way to see what fields are needed to be created when
223     there are no currently loaded items. I'm thinking about using a
224     Wizard in my application, so this might be overridden by the app.
225   */
226   self.createItem = function(evt) {
227     console.log("ADDING ITEM (NOT FINISHED YET)");
228     // The trick here is knowing what fields need to be created.
229     // self.items.push(ko.collectionItem({}));
230   };
231   
232   /*
233     Permanently remove the selectedItem, and delete it on the server.
234   */
235   self.removeSelectedItem = function(evt) {
236     if (self.selectedItem && self.selectedItem()) {
237       var sure = confirm("This will permanently remove " + self.selectedItem().name());
238       if (sure){
239         self.selectedItem().destroy();        
240       }
241     }
242   };
243   
244   /*
245     Iterate through self.items(), finding those that match all of the data
246     we pass in.
247     
248     For instance, you can do things like: 
249     
250       viewModel.findMatchingItems({date_of_birth: "1995-01-01"})
251     
252     This is used internally to find matches for objects when updating. Not
253     sure why it is exposed as a public member function though.
254   */
255   self.findMatchingItems = function(options) {
256     return self.items().filter(function(el){
257       var match = true;
258       $.each(options, function(opt, val) {
259         if (el[opt]() !== val) {
260           // Returning false causes $.each to stop, too.
261           return match = false;
262         }
263       });
264       return match;
265     });
266   };
267     
268   if (url) {
269     self.fetchItems();
270   }
271   
272   return ko.observable(self);
273 };

Second, the ko.collectionItem object. This may be eventually hidden in the collection object, as it isn’t really intended to be used seperately.

  1 ko.collectionItem = function(initialData, parentCollection) {
  2   var self = {
  3     isFetched: ko.observable(false)
  4   };
  5   var links = [];
  6   var etag = null;
  7   var url = null;
  8   var attributes = ko.observableArray([]);
  9   var collection = parentCollection;
 10   var dirtyFlag = ko.observable(false);
 11   
 12   /* Private methods */
 13   
 14   /*
 15     Given the incoming 'data' for this object, look through the fields for
 16     things that differ between the server representation and the client
 17     representation. Store both values for any differences in an attribute
 18     of the observable called conflicts().
 19     
 20     For each conflict, create a member function on the observable that
 21     allows you to resolve the conflict. When the last conflict is resolved,
 22     our etag is updated to the value the server gave us.
 23     
 24     This method returns true if all conflicts could be resolved (ie, the
 25     data in all fields was the same, just the etag had changed).
 26   */
 27   var parseConflicts = function(data, newEtag) {
 28     $.each(data, function(attr, value){
 29       if (attr !== "links") {
 30         if ($.compare(value, self[attr]() === undefined ? "" : self[attr]())) {
 31           // Server and client values match.
 32           // We need to do some funky stuff with undefined values, and treat
 33           // them as "". I don't really like this, but it works for now.
 34           self[attr].conflicts([]);
 35           self[attr].resolveConflict = function(){};
 36         } else {
 37           self[attr].conflicts([value, self[attr]() === undefined ? "" : self[attr]()]);
 38           self[attr].resolveConflict = function(chosenValue) {
 39             // Mark the entire object as dirty, so we can allow it to be
 40             // saved, even if we set it to the original value we had (which
 41             // differed from the server's value).
 42             self.isDirty(true);
 43             self[attr](chosenValue);
 44             self[attr].conflicts([]);
 45             if (!self.hasConflicts()) {
 46               // If this was the last conflict, we can use the new etag from
 47               // the server.
 48               etag = newEtag;
 49             }
 50           };
 51         }        
 52       }
 53     });
 54     var conflicts = self.hasConflicts();
 55     if (!conflicts) {
 56       etag = newEtag;
 57     }
 58     return !conflicts;
 59   };
 60   
 61   /*
 62   Given an object containing errors, we want to apply each of these
 63   errors onto the relevant field. We want to remove any errors that are
 64   already on any field.
 65   
 66   If we have any errors leftover, we need to notify globally, using the
 67   parentCollection's messages object.
 68   */
 69   var markErrors = function(errors) {
 70     $.each(attributes(), function(i,attr){
 71       if (!self[attr].errors) {
 72         self[attr].errors = ko.observableArray([]);
 73       }
 74       if (errors[attr]) {
 75         self[attr].errors(errors[attr]);
 76         delete errors[attr];
 77       } else {
 78         self[attr].errors([]);
 79       }
 80     });
 81     
 82     $.each(errors, function(field){
 83       parentCollection.messages({type:"error", message: field + ": " + errors[field].join("<br>"), timeout: 3000});
 84     });
 85   };
 86   
 87   /*
 88     Get the attributes ready for sending to the server.
 89     
 90     We can't just iterate through properties, as some will not apply. We
 91     use the convention that we will only send back properties that the
 92     server sent to us.
 93   */
 94   var prepareAttributes = function() {
 95     var data = {};
 96     $.each(attributes(), function(i,attr){
 97       data[attr] = self[attr]();
 98     });
 99     return data;
100   };
101   /* Public methods */
102   
103   /*
104     Update the data fields associated with this object from the provided
105     data.
106     
107     This may create new attributes, which need to be noted so we can send
108     those values back to the server.
109     
110     We can mark all updated attributes as not dirty, not conflicted, and
111     not having errors.
112   */
113   self.updateData = function(data) {
114     if (data.links) {
115       // We want to store the links, but not attach them to the object.
116       links = data.links;
117       delete data.links;
118       $.each(links, function(i, obj){
119         if (obj.rel === "self") {
120           url = obj.uri;
121         }
122       });
123     }
124     
125     $.each(data, function(attr, value){
126       if (attributes().indexOf(attr) < 0) {
127         self[attr] = ko.observable(value);
128         self[attr].errors = ko.observableArray([]);
129         self[attr].conflicts = ko.observableArray([]);
130         ko.dirtyFlag(self[attr], false);
131         // Need to add this last to cause the dirtyFields dependentObservable
132         // to work correctly when editing the last field.
133         attributes.push(attr);
134       } else {
135         self[attr](value);
136         self[attr].errors([]);
137         self[attr].conflicts([]);
138         self[attr].isDirty.reset();
139       }
140     });
141     // Put the links back in case a post-processor needs them.
142     data.links = links;
143   };
144   
145   self.serialize = function(evt) {
146     return JSON.stringify(prepareAttributes());
147   };
148   
149   /*
150     Discard any local changes, and pull the data from the server.
151   */
152   self.revert = function(evt) {
153     etag = null;
154     self.fetch();
155     parentCollection.messages({type:'warning', message:'The object "' + self.name() + '" was reverted to the version stored on the server.', timeout: 5000});
156   };
157   
158   /*
159     Attempt to save the data to the server.
160     
161     Only permitted to do this if we have successfully fetched the data
162     at some point.
163     
164     Notes: We use POST instead of PUT, in case we do not have access to
165            all of the fields of the object. PUT implies the complete resource
166            is being updated.
167            Errors may come back in {'field-errors': []}, or {'detail':[]}.
168            Currently, this makes assumptions about server type, which are
169            bad. I need to refactor the error handling code. (400,409)
170            Precondition Failed (412) needs to be handled differently, as
171            we need to fetch the data from the server if none was provided
172            as to the current state of the resource.
173            
174   */
175   self.save = function(evt) {
176     if (self.isFetched()) {
177       $.ajax({
178         url: url,
179         type: 'post', // We can't PUT in case we don't know about all fields.
180         headers: {'If-Match': etag},
181         data: self.serialize(),
182         statusCode: {
183           200: function(data, textStatus, jqXHR) {
184             // Object saved.
185             // Incase some fields were reformatted by the server, redo our data.
186             self.updateData(data);
187             etag = jqXHR.getResponseHeader('Etag');
188             parentCollection.messages({type:'success', message:'The object "' + self.name() + '" was saved.', timeout: 2500});
189             self.isDirty(false);
190           },
191           201: function(data, textStatus, jqXHR) {
192             // Object saved for the first time (created)
193             // Incase some fields were reformatted by the server, redo our data.
194             self.updateData(data);
195             etag = jqXHR.getResponseHeader('Etag');
196             url = jqXHR.getResponseHeader('Location');
197             parentCollection.messages({type:'success', message:'The object "' + self.name() + '" was created.', timeout: 2500});
198             self.isDirty(false);
199           },
200           400: function(jqXHR, textStatus, errorThrown) {
201             var data = JSON.parse(jqXHR.responseText);
202             if (data['field-errors']) {
203               markErrors(data['field-errors']);
204             }
205             parentCollection.messages({type:'error', message:'The object "' + self.name() + '" could not be saved. Please check the highlighted field(s).', timeout: 10000});
206           },
207           409: function(jqXHR, textStatus, errorThrown) {
208             // Errors saving the data. Likely to be validation errors.
209             // We should have a detail object with info to display.
210             var data = JSON.parse(jqXHR.responseText);
211             if (data.detail) {
212               markErrors(data.detail);
213             }
214             parentCollection.messages({type:'error', message:'The object "' + self.name() + '" could not be saved. Please check the highlighted field(s).', timeout: 10000});
215           },
216           412: function(jqXHR, textStatus, errorThrown) {
217             // Data was changed on server since we last fetched it.
218             // There may be conflicts to deal with.
219             // See if the server gave us a current version back...
220             var data, serverEtag;
221             if (jqXHR.responseText) {
222               data = JSON.parse(jqXHR.responseText);
223             } else {
224               $.ajax({
225                 url: url,
226                 async: false,
227                 success: function(newData, textStatus, jqXHR) {
228                   data = newData;
229                   serverEtag = jqXHR.getResponseHeader('Etag');
230                 }
231               });
232             }
233             if (parseConflicts(data, serverEtag)) {
234               // We were able to resolve all of the conflicts, now we can
235               // try to re-save; but only if it was the first time we saved,
236               // to prevent inifinite recursion.
237               if (evt) {
238                 self.save();
239               }
240             } else {
241               parentCollection.messages({type:'error', message:'The object "' + self.name() + '" has been modified on the server. Please check the changed field(s) and select the appropriate value(s).', timeout: 10000});
242             }
243           }
244         }
245       });
246     }
247   };
248   
249   /*
250     Permanently delete the object from the server.
251   */
252   self.destroy = function(evt) {
253     if (self.isFetched() && etag) {
254       console.log("DELETING ITEM");
255       $.ajax({
256         url: url,
257         type: 'delete',
258         headers: {'If-Match': etag},
259         success: function(data, textStatus, jqXHR) {
260           if (collection) {
261             collection.items.remove(self);
262           }
263           parentCollection.messages({type:'success', message:'The object "' + self.name() + '" was deleted.', timeout: 2500});
264         },
265         error: function(jqXHR, textStatus, errorThrown) {
266           // Display error message about not being able to delete?
267           parentCollection.messages({type:'error', message:'The object "' + self.name() + '" could not be deleted.', timeout: 10000});
268         }
269       });
270     }
271   };
272   
273   /*
274     (Re)Fetch the resource from the server.
275     
276     Handle conflicts if the arise (when the object has already been fetchd)
277   */
278   self.fetch = function(evt) {
279     var headers = {};
280     if (etag) {
281       headers['If-None-Match'] = etag;
282     }
283     $.ajax({
284       type: 'get',
285       url: url,
286       headers: headers,
287       statusCode: {
288         200: function(data, textStatus, jqXHR) {
289           // If we have an etag already, this means the object has been
290           // updated on the server, and we need to look for conflicts.
291           if (etag) {
292             var serverEtag = jqXHR.getResponseHeader('Etag');
293             // If we were unable to handle all conflicts, we need to exit.
294             if (!parseConflicts(data, serverEtag)) {
295               parentCollection.messages({type:'error', message:'The object "' + self.name() + '" has been modified on the server. Please check the changed field(s) and select the appropriate value(s).', timeout: 10000});
296               return;
297             };
298           }
299           
300           // Otherwise, we can update the data and the etag.
301           self.updateData(data);
302           etag = jqXHR.getResponseHeader('Etag');
303           self.isFetched(true);
304         },
305         304: function() {
306         }
307       },
308       error: function(jqXHR, textStatus, errorThrown) {
309         parentCollection.messages({type:"error", message:"There was an error fetching the data from the server"});
310       }
311     });
312   };
313   
314   
315   
316   /* Dependent Observables */
317   self.dirtyFields = ko.dependentObservable(function(){
318     return ko.utils.arrayFilter(attributes(), function(attr){
319       return self[attr] && self[attr].isDirty && self[attr].isDirty();
320     });
321   });
322   
323   self.conflictedFields = ko.dependentObservable(function() {
324     return ko.utils.arrayFilter(attributes(), function(attr){
325       return self[attr] && self[attr].conflicts && self[attr].conflicts().length > 0;
326     });
327   });
328   
329   var filterAttributes = function(property) {
330     return function() {
331       return ko.utils.arrayFilter(attributes(), function(attr){
332         return self[attr] && self[attr][property] && self[attr][property]().length > 0;
333       }).length > 0;
334     };      
335   };
336   
337   self.hasErrors = ko.dependentObservable(filterAttributes('errors'));
338   self.hasConflicts = ko.dependentObservable(filterAttributes('conflicts'));
339   
340   /*
341     An object is dirty when:
342       - any of its fields/attributes are dirty. (we aks them), OR
343       - we have explicitly marked it as dirty.
344       
345     We need to do the latter for when we have merged a conflict, by choosing
346     our value, which differed from the server. The local model would
347     normally think it wasn't dirty, but it differs from the server, and
348     does need to be saved.
349   */
350   self.isDirty = ko.dependentObservable({
351     read: function() {
352       return self.dirtyFields().length > 0 || dirtyFlag();
353     },
354     write: function(value) {
355       dirtyFlag(value);
356       if (!value) {
357         $.each(attributes(), function(attr){
358           if (self[attr] && self[attr].isDirty) {
359             console.log(attr);
360             self[attr].isDirty.reset();          
361           }
362         });
363       }
364     }
365   });
366   
367   /*
368     Can this object be saved back to the server?
369     Only when it is dirty, and has been fetched.
370   */
371   self.canSave = ko.dependentObservable(function() {
372     return self.isDirty() && self.isFetched();
373   });
374   
375   self._etag = function(){return etag;};
376   self._attributes = function(){ return attributes();};
377   self._url = function() {return url;};
378   
379   if (initialData) {
380     self.updateData(initialData);
381   }
382   
383   return self;
384 };