Alex Payne

Books

2010-08-15T00:00:00-07:00

Let’s Make a an Observer Library Most books and articles on javascript trends towards two extremes: basic instruction in the execution and idiomatic patterns of javascript as a language and basic usage of popular browser-focused utility libraries written in the language. In the first category I group The Definitive Guide, Crockford's The Good Parts, Resig's Pro JavaScript Techniques and Harnes & Diaz's Pro JavaScript Design Patterns; in the second, jQuery, Novice to Ninja, Mastering Dojo, and Practical Prototype and Scrtip.aculo.us.

In the wide gulf between these two sets there is almost nothing written. This leaves an intermediate javascript developer adrift. You understand basic language constructs, prototypal inheritance, and functional programming patterns. You've built a few browsed-based applications so you're accustomed to the DOM and user-generated event-based programming.

But, like me, you're probably stumped when it comes to writing your own reusable libraries or organizing a large application. The flexibility everyone lauds in language can be an idiomatic quagmire when you're between the basics and off-the-shelf utility libraries.

Given all this, I was super geeked when Alex Young started a new series of articles entitled Let's Make a Framework. This is some of the best middle ground writing about JavaScript I've read.

I'd like to add to the body of work in this area, so over the next few weeks I'll be series of posts that explore writing a javascript implementation of the observer pattern. Like Alex's series, this goal is creating an educational exercise for the reader, not to craft a ground-breaking library.

I'm going to cover

the basic observer pattern
event based programming
property change observation
data syncing through observation
existing libraries that use these techniques

Doing this, I'll focus on some modern development techniques:

behavior driven development
source control
API design

Let’s Make a an Observer Library. Part I: Library Architecture

Welcome to part 1 of Writing an Observer Library, a series of posts about building a JavaScript library to implement the various flavors of the observer pattern. In this part I’m going to discuss library architectures, and lay down the design for our framework.

This series is inspired by Alex Young's series of articles entitled Let's Make a Framework. Like Alex's, this series is designed to be educational. Also like Alex, we're going to define a set of practices to govern the library's crafting:

Verbose: Variable and method names should be verbose so things are easy to find and understand
Portable: Browsers and console should be catered for
Indentation: Two spaces
API-focused: a consistent, organized API is valued over other practices
Tested: Everything is tested (first) in JSpec
Versioning: We'll be keeping code in github
Documented: Everything should be well documented and a usage guide will be provided
Neighborly: We'll be sure not stop on global scope toes, avoid extending the prototype of included types, and use existing libraries if they provide functionality outside the scope of the what we're writing (dependencies are ok, folks)

Has Many, with Precedence

2010-06-20T00:00:00-07:00

Every so often I have a has_many ActiveRecord relationship with a twist: one (and only one) of the many possible related items is special in some way and I want to be get and set it like other associations. Imagine a Customer has several phone numbers, but only one is their primary contact number.

ActiveRecord has a few strategies for multiple, similar relationships. You could use association extensions and define a sub-assocation like so:

class Customer < ActiveRecord::Base
  has_many :phone_numbers do
    def priamry
      find(:first, :conditions => {:is_primary => true})
    end
  end
end

which will allow you to retrieve the primary phone number, but not set it:

c = Customer.find(1)
c.phone_numbers.primary
# => <PhoneNumber id: 1 number: '5551234', is_primary true>
c.phone_numbers.primary = PhoneNumber.new(:number => '5551235')
# => NoMethodError: undefined method `primary='

ActiveRecord lets you define multiple relationships to the same class with some configuration:

class Customer < ActiveRecord::Base
  has_many :phone_numbers
  has_many :primary_phone_numbers, 
           :class_name => 'PhoneNumber', 
           :conditions => {:is_primary => true}
end

This will let you ask for either collection, assign new or existing objects to either collection, and even use the build and create methods on the collection to get new objects with the attributes pre-assigned to match the :conditions you supplied:

c = Customer.find(1)
c.phone_numbers
#    [<PhoneNumber id: 1 number: '5551234', is_primary true,
#    <PhoneNumber id: 2 number: '5551234', is_primary false >]
c.primary_phone_numbers
#    [<PhoneNumber id: 1 number: '5551234', is_primary true]

The problem here is that you always get a collection back, and will need additional code to enforce the existence of a single primary phone number for a customer.

c = Customer.find(1)
c.phone_numbers
#    [<PhoneNumber id: 1 number: '5551234', is_primary true,
#    <PhoneNumber id: 2 number: '5551234', is_primary false >]
c.primary_phone_numbers << PhoneNumber.new(:number => '555 9081')
#    [<PhoneNumber id: 3 number: '5559081', is_primary true]
c.primary_phone_numbers
# two primary phone nubmers!
# [<PhoneNumber id: 1 number: '5551234', is_primary true,
#   <PhoneNumber id: 3 number: '5559081', is_primary true]

You might think, at first, the solution is to has has_one

class Customer < ActiveRecord::Base
  has_many :phone_numbers
  has_one  :primary_phone_number, 
           :class_name => 'PhoneNumber', 
           :conditions => {:is_primary => true}
end

This will work fine for retrieving, but you'll run into trouble when trying to assign. Part of the job of a has_one relationship is ensuring that the foreign key appears only once for the type of association.

c = Customer.find(1)
c.phone_numbers
#    [<PhoneNumber id: 1 number: '5551234', is_primary true,
#    <PhoneNumber id: 2 number: '5551234', is_primary false >]
c.primary_phone_number 
#    <PhoneNumber id: 1 number: '5551234', is_primary true>
c.primary_phone_number = Number.new(:number => '555 0919')
#    <PhoneNumber id: 3 number: '5550919', is_primary true>
c.reload
c.phone_numbers
# foreign keys cleared
# [<PhoneNumber id: 3 number: '5550919', is_primary true>]

The solution I'm currently using is to flip the relationship (and where the foreign key is stored) by making a Customer belong_to primary PhoneNumber

class Customer < ActiveRecord::Base
  has_many :phone_numbers
  belongs_to  :primary_phone_number, :class_name => 'PhoneNumber'
end

Now we can get and set a primary phone number without overwriting the old ones. Unfortunately we have the complementary problem to the one above. When we ask for phone_numbers the collection won't include the primary phone number (since it lacks the customer_id in the phone_numbers table)

  c.phone_numbers
  #    [<PhoneNumber id: 1 number: '5551234'>]
  c.primary_phone_number 
  #    nil
  c.primary_phone_number = Number.new(:number => '555 0919')
  #    <PhoneNumber id: 3 number: '5550919'>
  c.reload
  c.phone_numbers
  # doesn't include the primary phone number
  # [<PhoneNumber id: 1 number: '5551234'>]

In the future, We'll be able to solve this with a before_add callback whenever this patch is applied (right now they only work for has_many associations):

belongs_to  :primary_phone_number, 
            :class_name => 'PhoneNumber', 
            :before_add => :add_to_phone_numbers

def add_to_phone_numbers(phone_number)
  self.phone_numbers << phone_number
end

Until then, you can work around the limitation with some alias_method_chain trickery:

belongs_to :phone_number

def phone_number_with_callback=(phone_number)
  self.phone_numbers << phone_number
  self.phone_number_without_callback = phone_number
end

alias_method_chain :phone_number=, :callback

Multiparameter assignment and attr_accessor

2010-06-16T00:00:00-07:00

I've needed multiparamter assignment on non-persisted attributes for a few projects (usually on for Date and Time objects). Typically, I cheat and send separate attribute and have a before_validation callback to combine them into a date. I finally sat down and sussed out how to do it properly:

attr_accessor :initial_billing_day, 
              :initial_billing_month, 
              :initial_billing_year
              
composed_of :initial_billing_date, 
  :class_name => "Date", 
  :mapping => [
    %w(initial_billing_day day), 
    %w(initial_billing_year year), 
    %w(initial_billing_month month)
  ],
  :constructor => proc {|y, m, d|
    return nil unless y && m && d
    Date.new(y,m,d)
  }

Format Parsing in Javascript

2010-06-07T00:00:00-07:00

Riff is a a recent spin-off project from Jesse Sielaff’s work to bring a Ruby-like language interpreter to the browser. Riff is a plugin for Racc (a LARL(1) parser generator for Ruby). Riff extends the output generation portions of Racc and writes javascript instead of Ruby. If you’re familiar with LALR grammars this should add a handy tool to your javascript arsenal.

Jesse’s been using Riff in his latest stab at Red, but we were eager to throw it at a smaller text format and see how it compared to a solid hand-written parser. We took a weekend and paired on rewriting Jan Lehnardt’s javascript implementation of Chris Wanstrath’s mustache template language.

Our mustache.js has three little differences from Jan’s (we wanted to keep as close to Chris’ Ruby/Python mustache implementations as possible):

no explicit iterator {{.}} (use {{ toString }})
partial context is stored in Mustache.partials instead of passed as an argument. You can switch this context to a different object when you need to.
we have Mustache.render(template, object) instead of Mustache.to_html(template, object, partials, callback)

Other than those, the two parsers are roughly identical in functionality but quite distinct in implementation. Jan’s mustache.js is a handwritten parser, our’s is generated from Riff and Racc.

A generated parser is broken up into two main parts: grammar definition and semantic action. Take a look at Mustache’s grammar. The first part defines the grammar of the language (it’s pretty small), from which a finite state machine is generated. The part of the grammar enclosed in { and } defines the semantic rule conversion (e.g. what to do when a token is found).

For example, line 13 of our grammar looks like this:

"{{{" "text" "}}}"  { $$ = Array(4, $2); }

This means when you find }, create a new Array object with 4 (which is the arbitrary code number we gave to “escaped text” nodes) as the first element and the string value of the “text” node as the second element. What you choose to do in your parser depends how you’d like to handle evaluation. You might create javascript literals, use a jquery class, etc. We’ve opted to create a tree of nodes using Array objects in javascript, but a different solution might have been:

"{{{" "text" "}}}"  { $$ = { node_type:'unescaped-text', content: $2 } }

The grammar file is run through Riff and converted to generated parser. Everything up to line 214 is generated, the remainder is copied over from the ---- footer section of the grammar file.

Initially the footer for Mustache contained every function necessary to parse, evaluate, and render a mustache formatted file. Jesse later separated this into two separate sections – a parser and an evaluator – and the end results are pretty awesome (big reveal later, I promise).

Starting on line 214, we have two major functions of Mustache: compile and next_token. compile sets up a state for parsing and calls do_parse (which is generated by Riff from the grammar, not written by the programmer). next_token is called from inside do_parse and walks through the input string character by character, searching for semantically meaningful elements (e.g. {{#something}}).

This part can be a little dense if you’re unfamiliar with look-ahead parsing. If you can’t visualize the pointer moving through a string, see the StringScanner branch, where Jesse uses his javascript port of Ruby’s string scanner class and regular expressions to manipulate the string pointer.

So, this code gets you a tree of Array objects. The evaluator part of the library takes this tree and renders it into a string with the appropriate values inserted. The main mojo in the evaluator is the render function. This is the only function you’ll call in your code with Mustache.render(template, object). The evaluator is organized into one function for semantic type (e.g. evaluate_inverse_block_node for {{^foo}} inverse blocks)

Internally, render checks the type of template object you passed in. Up until now, we’ve talked about templates only as strings in mustache format. If you pass in a String template, it runs through Mustache.compile and comes out the other end as the series of Arrays mentioned above, is evaluated, and finally is converted back into a String with data inserted in appropriate locations.

Now, imagine we could start with a series of Arrays and skip the parsing stage. Obviously, nobody wants to develop their templates like this:

[0,[0,[0,[0,[0,[0,[0,[0,'<img src="',[3,'src']],'" width="'],[3,'width']],'" height="'],[3,'height']],'" alt="'],[3,'name']],'">']

we want to write templates like this:

<img src="{{src}}" width="{{width}}" height="{{height}}" alt="{{name}}">

In production, of course, the computer doesn’t care what your templates look like. If you’re working with javascript on the server, you can pre-compile your templates with Mustache.compile. Jesse also whipped up an example pre-compiler for mustache in Ruby in case you have different server needs.

This let’s us a use mustache.js, mustache.evaluator.js, and String templates while developing and move to just mustache.evaluator.js (which is a tiny, tiny file) and compiled templates when we’re ready to move into production and want a speed-boost.

How’d Riff do?

Not bad for a weekend pairing project. We went from nothing to a spec’ed, running format parser in two afternoons. Adjusting for comments and code style (Jesse is more liberal with his white space around curly braces), there’s 20% fewer lines of code over the handwritten parser. Run through a javascript minifier (Dean Edward’s in this example), Jan’s parser is 8,747 bytes, Jesse’s 6,292 bytes, and in production, you can get by with just the evaluator at 2,244 bytes.

Organizationally, I prefer separating out language definition, parsing, and evaluation. I personally find it reads more easily than putting everything in a single structure. Plus, it gives us the benefit of using compiled templates.

Speedwise, Jan’s Mustache ranges from identical up to 4x faster (depending the example), averaging 1.3x faster than ours. But as Jan kindly noted code size and simplicity are probably more important than incremental speed gains. Plus, if you’re desperate for cycles, using compiled templates is 14x faster (9x faster than Jan’s).

Here are the speed test results (run in Firefox, on my last-gen Macbook):

template	janl	jesse	jesse compiled
basic	207ms	356ms	25ms
complex	676ms	885ms	74ms
friend	202ms	227ms	22ms
long	192ms	450ms	29ms
looping	278ms	354ms	25ms
partial	892ms	945ms	81ms
	2447	3217	256

If you find yourself hand-parsing text in javascript, give Riff a try. We think you’ll like it.

Automating iTunes Gift Cards

2010-01-14T00:00:00-08:00

I'm finishing up a project for the University of Michigan School of Public Health that involves using iTunes Gift Cards as a participant incentive. They've pre-bought about $20,000 of Gift Cards and given me the ids.

We could just have participants copy/paste their code into iTunes, but I wanted to create a smooth user experience for redeeming.

The base url for iTunes Redeeming is https://buy.itunes.apple.com/WebObjects/MZFinance.woa/wa/redeemLandingPage (you can drag links from iTunes into a text editor to get their url) but I wanted to pre-fill the code field or automatically submit my request with the code supplied.

I suspected I could send a query parameter along with that url, but not being able to view source, I had no idea what to name it. So, I went to a place where I've seen automatic click-to-redeem before, Apple's own iTunes Facebook fan page.

Sure enough their fan page just sends a request to the same url with a query parameter of ?code=123456ABCDEF. Pass that along with the url and it will send your redemption code along without user intervention and will display inline error messages if your code is bunk for some reason

Rails Foundations Workshops - Winter 2010

2010-01-07T00:00:00-08:00

If you’re in South East Michigan and serious about learning (or better understanding) Rails our upcoming Rails Foundations workshop series is for you.

This season, we're trying something new: name your own price workshops

In the past we've offered the workshop for free and for pay. When free, we fill up same day and usually have 20+ people on the wait list. When we charge we end up with about half as many participants as we'd like. In the spirit of adventure (and capitalism!) we're letting people name their own price. $5? $50? $5000? Totally up to you.

At the end of a short bidding period, we're going to take a group of top bidders, and charge them all the lowest price from the group.

Rails Foundations focuses on giving attendees a solid understanding of major Rails components and how they interrelate. You’ll find we teach no Rails magic, trickery, or shortcuts.

We have a different approach for Rails training than most shops. If you’re looking for a grand tour of Rails, we recommended Big Nerd Ranch, the Pragmatic Studio, or fellow Michiganders Idea Foundry. If you want to explore just the core Rails concepts in greater depth (preparing you to do wider exploration on your own), we think you’ll enjoy the series.

If you’ve never used Rails before, the workshops are right for you if

You’ve programmed in an OOP language
You’ve written for the web, but didn’t really use HTTP fully
You like to keep up on current trends in development

If you’ve programmed in Rails, the workshops will clear up some misconceptions if

Your controllers have goofy actions like javascript_save and about_us
Your routes don’t go beyond map.connect ':controller/:action/:id'
Your views are so abstracted you’re not sure what HTML they even output anymore
You didn’t know that belongs_to is actually a class method that takes a symbol as its first argument

The course starts is just under two weeks, so grab a spot quickly!

Scoped DOM Selection

2009-08-24T00:00:00-07:00

While Jesse operates on the next version of Red, his never-quite-ready Ruby runtime built on top of Javascript, I’ve been playing with RedShift, our Ruby DOM manipulation library by exploring Ruby idioms that would make common tasks more elegant.

We already have a ready? method on the Document class that is called when the DOM has been parsed and is ready. It’s the equivalent of jQuery’s $(document).ready(fn), Prototype’s document.observe("dom:loaded", fn), and MooTools’ window.addEvent('domready', fn) functions.

I was playing with this concept of meta-events and had the idea of using them for scoping code based on selectors. When you combine all your javascript into a single file to reduce the number of HTTP requests for a page, you need a nice way of trigger certain behavior only for a subset of pages. I usually have a few classes and ids attached to the body tag of any page:

<body id='show' class='people admin-viewing'>

This presence of these identifiers will trigger a subset of all possible document behavior. To replacing many if statements I extended our DOM-selection method to encapsulate this behavior in an optional block.

Document['#show.people'] do
end

Although succinct, this fails aesthetically because it doesn’t obviously announce what the hell the block is for. I moved the block behavior to a specific found? method to mirror the syntax of Document.ready? as a kind of meta-event:

Document['#show.people'].found? do
  # code here only takes place if the selector is found
  Document['#show.people']['a'].add_class('whatever')
end

Missing from this example was the ability to do sub-selects from within the block (which gives a nice speed boost and reduces duplication). I updated it to take a block argument.

Document['div#navigation'].found? do |page_nav|
  # find all the links inside the navigation div
  page_nav['a'].add_class('whatever')
end

I’m eager to try pattern on existing projects, so I extended jQuery to accomplish the same goals:

$('div#navigation').found(function(){
  // only execute if the selector finds elements
  this; // is the jQuery object of $('div#navigation')
});

Rails Foundations Workshops

2009-07-08T00:00:00-07:00

If you’re in South East Michigan and serious about learning (or better understanding) Rails our upcoming Rails Foundations workshop series is for you. In the past we’ve only offered this workshop to our friends and colleagues or by recommendation. Based on the feedback of some people we highly respect we’ve decided to make this series generally available.

Rails Foundations focuses on giving attendees a solid understanding of major Rails components and how they interrelate. You’ll find we teach no Rails magic, trickery, or shortcuts.

If you’ve never used Rails before, the workshops are right for you if

You’ve programmed in an OOP language
You’ve written for the web, but didn’t really use HTTP fully
You like to keep up on current trends in development

If you’ve programmed in Rails, the workshops will clear up some misconceptions if

Your controllers have goofy actions like javascript_save and about_us
Your routes don’t go beyond map.connect ':controller/:action/:id'
Your views are so abstracted you’re not sure what HTML they even output anymore
You didn’t know that belongs_to is actually a class method that takes a symbol as its first argument

The course starts is just under two weeks, so grab a spot quickly!

Get Your API Right

2009-07-02T00:00:00-07:00

Every project I’ve worked on in the last two years has heavily involved the use of web APIs. Libersy at the time (no idea about now) had an architecture that was extensively API based, even for communication between internal applications (an architecture I strongly argued against, bee tea dubs). Since then I’ve futzed with web APIs almost exclusively. From very narrow focused uses like University of Michigan’s Bluestream Service, to more broad but still fairly local APIs like the Ann Arbor District Library’s soon-to-be-updated API, all the way to APIs of major web applications like Twitter and Flickr.

Constant exposure has turned me into a bit of a snob: I can’t stand working with a poorly designed API! If you’re about to design or release an API for the web and want to avoid the ire of your developers, I’ve summed up the best (and worst) of what I’ve seen into 8 rules:

1) Use HTTP

I’ll grant that HTTP isn’t perfect but it’s at least well understood and nearly universal. Nobody is going to want to write against your custom application layer atop UDP. Nobody cares about the RPC system you built in 1997 that is “running just fine.” Unless you have some major reasons HTTP cannot work and are willing to provide solid client libraries in a dozen or more languages, stick with HTTP.

2) Use Your Verbs

Every HTTP request comes with a verb. These verbs have specific meaning and should be used correctly. Use GET to retrieve items, POST to add new items to your service, PUT to update existing items, DELETE to remove items, and HEAD for uses similar to GET where the programmer doesn’t need body content (e.g. cache checking).

If you’re concerned that not all client libraries implement the HTTP verbs (web browsers, for example, can’t submit PUT and DELETE requests) allow for HTTP verb faking. The emerging convention is passing a _method parameter as part of the request body with the verb as a lowercase value (put,delete, and head).

If you don’t use all the verbs, you won’t be able to give your users access to all possible actions on your data without resorting to specialty urls. It’s possible to create an API that uses only POST requests:

POST /photos/create
POST /photos/show/decafbebad
POST /photos/update/decafbebad
POST /photos/delete/decafbebad

By doing this you’re encoding the intended action directly in the url; this is needlessly redundant. HTTP already gives to a slot for specifying action (the verb). Just use it:

POST   /photos
GET    /photos/decafbebad
PUT    /photos/decafbebad
DELETE /photos/decafbebad

Twitter’s API tries to have it both ways with destroying favorites. You can use either POST or DELETE as your software allows. To support client libraries that can’t DELETE, the deletion is additionally referenced in the URI (http://twitter.com/favorites/destroy/id.format).

Flickr only uses GET and POST necessitating specialty urls.

If you’re not allowing clients to create new data, or update/delete existing data on your system then you do not have an API. You have a feed. There’s nothing wrong providing read-only access to your data (it’s laudable, in fact), but I’m often disappointed to hear “Yeah! We have an API” only to find the person really meant they offered a number of customizable data feeds as XML.

The current Ann Arbor Library API is really a data feed.

Shopify’s API gets it right.

3) Keep Your URL/URIs Consistent

One of the most important principles of REST (which is quickly becoming the preferred method of organizing web APIs) is “every resource is uniquely addressable using Uniform Resource Identifiers”. In addition to making this URI unique, it’s important to make them patterned and consistent.

The Bluestream API, for example, uses the following four URIs to create a resource, retrieve a resource, find items related to that resource, and see the resource’s history of edits.

POST /ams/upload
GET  /ams/rest/asset/A1001001A06B17B43948J49293
GET  /ams/rest/related/A1001001A06B17B43948J49293
GET  /ams/rest/history/A1001001A06B17B43948J49293

Two problems with the URIs above. First, the URI for creating new assets is wildly different than the other URIs. You’re better off sticking with a pattern: POST /ams/rest/asset/.

Second, the collection of assets related to A1001001A06B17B43948J49293 is conceptually a sub-asset. Typically you’ll see the collection nested within the full URI of its conceptual parent: /ams/rest/asset/A1001001A06B17B43948J49293/related. Same goes for the history of edits: /ams/rest/asset/A1001001A06B17B43948J49293/history.

Most data hierarchies will be fairly flat like Twitter’s so you frequently won’t need to expose urls with deeply nested data relationships. But if you do, many client libraries are designed to interact with REST APIs that follow a patterned URI scheme.

The pattern of Digg’s data feeds is easy to understand. You barely need any more documentation than a sentence for the urls of each type. Sadly, you can’t get data in using these urls but “endpoints for participating” are coming soon.

4) Use Your Status Codes

Every HTTP response comes back with a status code. Like HTTP request verbs, these responses statuses have specific meaning. Use them correctly! Many API providers send back only two codes: 200 for requests that worked and 500 for requests that didn’t work.

I once snapped this shot while using the Ann Arbor District Library’s current API:

Sending back 200 OK when you mean 404 Not Found is decidedly not OK.

So, when someone PUTs to a URI that only accepts GET and POST send back a 405 Method Not Allowed. When someone attempts to access data without authenticating, send back 401 Unauthorized. Most client libraries will convert these status into native errors/exceptions keeping the amount of body parsing required (I got back data with 200 OK, but was it a “200 OK here’s what you wanted” or a “200 OK shit is not ok?”)

5) Expose (And Accept) Multiple Data Formats

You should both expose data and accept incoming data in at least XML and json/jsonp. These two data formats represent the standard data serializations of the web. The format of these should allow for nested, related data. There is some variance in how this data should be formatted, but there is an emerging pattern

For XML:

Enclose a data type in an element of it’s name, not as bare data

Do this

<?xml version="1.0"?>
<person>
  <name>...</name>
  <age>...</age>
</person>

not this

<?xml version="1.0"?>
<name>...</name>
<age>...</age>

Enclose a collection in a tag of its type, pluralized. Each item should be in an element of its type

Do this

<?xml version="1.0"?>
<dogs>
  <dog>
    <name>...</name>
  </dog>
  <dog>
    <name>...</name>
  </dog>
</dogs>

not this

<?xml version="1.0"?>
<dog>
  <name>...</name>
</dog>
<dog>
  <name>...</name>
</dog>

Although most client libraries will happily parse either, the first allows you to place additional useful data about the collection as an attribute of the collection itself:

<?xml version="1.0"?>
<count>2</count>
<last-added>...</last-added>
<dogs>
  ...
</dogs>

Use elements for data, not attributes.

Client library support for XML attributes is inconsistent and what distinguishes data in elements from data in attributes isn’t always clear.

Do this:

<asset>
  <id>A1001001A06C13B45909C08771</id>
  <name>AMS_TEST_ITEM</name>
  <display-name>AMS_TEST_ITEM</display-name>
  <size>80638</size>
  <thumbnail>
    <is-defult>true</is-default>
    <url>http://www.example.com/ams/icons/gif.gif</url>
  </thumbnail>
</asset>

not this:

<asset ID="A1001001A06C13B45909C08771">
  <entity name="AMS_TEST_ITEM" display-name="AMS_TEST_ITEM" />
  <metadata name="AMS_SZ" display-name="Size" namespace="Info">80638</metadata>
  <thumbnail isDefault="true">http://www.example.com/ams/icons/gif.gif</thumbnail>
</asset>

One valid use of attributes is to provide metadata. A common example is including suggested type casting, since XML (unlike json) only sends string data:

<asset>
  <id type='integer'>A1001001A06C13B45909C08771</id>
  <name>AMS_TEST_ITEM</name>
  <display-name>AMS_TEST_ITEM</display-name>
  <size type='integer'>80638</size>
  <thumbnail>
    <is-defult type='boolean'>true</is-default>
    <url>http://www.example.com/ams/icons/gif.gif</url>
  </thumbnail>
</asset>

Support for this in client libraries is spotty, however.

For json the structure is more formalized since json parses into native javascript date types. There are no attributes, only slots; strings, integers, booleans, nested object literals, nested arrays, and other formats can be sent as data in these slots. There’s really only one decision needed for json:

Have the outer object contained in a slot named for its type, or don’t

This one varies from API to API, both formats work well:

{
  'name' : '...',
  'age'  : 22,
  'dogs' : [
    {'name' : 'fido', 'breed' : 'mutt', 'is_spayed' : true},
    {'name' : 'killer', 'breed' : 'poodle', 'is_spayed' : false}
  ]
}

{
  'person' : {
    'name': '...',
    'age' : 22,
    'dogs' : [...]
  }
}

The latter will allow you to send back additional data about the response

{
  'access' : 'read-only',
  'person' : {
    'name': '...',
    'age' : 22
  }
}

But either format is acceptable.

For both XML and JSON, accept images as either multipart/post or a url

Images are tricky to manage via an API over the web (check out all the trouble that Twitter has had). Sending a multipart/post with the image data as content is probably the correct answer, but some http libraries (notably Ruby’s) have buggy or incomplete multipart/post implementations. Do your developers a favor and accept urls referencing images in addition to data POSTed as multipart body content.

6) Protect Your Users with OAuth

One thorny issue of user-specific API data is how you allow third party access. Twitter, until recently, required users to give their twitter username and password to third parties. You don’t ever want your users handing out their passwords! Instead, like Twitter, MySpace, Google Data, Get Satisfaction, Tripit, and many others, control access to your users’ data with OAuth. OAuth (not be mistaken with the very different OpenID) lets your users give access to a third party without giving away their credentials.

Libraries for adding OAuth to your API and for third-parties to consume data are available in all major languages.

7) Don’t Shut Off HTTP Authentication Entirely

OAuth isn’t always the answer. For cases where the end user isn’t involving a third party directly (e.g. a desktop or iPhone application that only communicates with your servers), allowing the user to simply enter their user name and password provides a more desirable user experience. At the very least, allow HTTP Authentication to obtain a token for these kinds of applications and then use that token for all other requests.

8) Document, Document, Document

Finally, fully document your API. You’ll want to demonstrate the following: valid URIs, required and optional data, HTTP status codes and what causes them, sample requests, and sample responses.

Having your documentation on the web in a structured format is best. PBWorks (formerly PBWiki) is very popular for API documenting: Twitter, Digg, and a few others are using it.

Depressed about Michigan

2009-06-10T00:00:00-07:00

I just got word of an upcoming career development event taking place in south-east Michigan next week and it's made me mildly depressed about the state of the software development industry in my area.

On one hand, I'm glad to see someone offering low-cost continuing education events. On the other hand it's 2009: there should not be a developer left in this industry who hasn't already investigated and/or implemented most of these techniques already on her (or his) own time or as part of a workplace requirement to remain updated on current industry trends.

You can't talk about "version control, continuous integration, and automated testing" as "new techniques" just like you can't talk about "C#, Java, Python, Scala, GWT, Silverlight," as "new languages".

Sad Panda.