be Groovie

Routes 1.4 Release and Web Services

This is slightly old as Routes 1.4 was released about a week and a half ago, but I thought it deserved some attention. There were a handful of fixes and some slightly major feature enhancements in 1.4.

From the changelog:

• Fixed bug with map.resource related to member methods, found in Rails version.
• Fixed bug with map.resource member methods not requiring a member id.
• Fixed bug related to handling keyword argument controller.
• Added map.resource command which can automatically generate a batch of routes

intended to be used in a REST-ful manner by a web framework.

• Added URL generation handling for a ‘method’ argument. If ‘method’ is specified, it

is not dropped and will be changed to ‘_method’ for use by the framework.

• Added conditions option to map.connect. Accepts a dict with optional keyword args

‘method’ or ‘function’. Method is a list of HTTP methods that are valid for the route. Function is a function that will be called with environ, matchdict where matchdict is the dict created by the URL match.

• Fixed redirect_to function for using absolute URL’s. redirect_to now passes all

args to url_for, then passes the resulting URL to the redirect function. Reported by climbus.

Web Resources

The map.resource command is based directly off the Simply Restful Rails plugin which adds support for various verb-oriented controller actions in a RESTful service style approach. The Simply Restful layout is more or less the exact service style laid out in the Atom Publishing Protocol.

It’s a great approach and it also meant providing a few other features to Routes that I hadn’t implemented previously, the most important being able to limit matching of a URL based on the HTTP method used. This is present in the new conditions clause for a Route:

map.connect('user/:id', controller='user', action='edit', 
    conditions={'method', ['GET', 'HEAD']})
map.connect('user/:id', controller='user', action='update',
    conditions={'method', ['PUT']})

The conditions clause can also accept your own function should you want to restrict the route to matching based off some other criteria (sub-domain, IP address, etc).

def stop_comcast(environ, match):
    if 'comcast.net’ in environ['REMOTE_HOST’]:
        return False
    return True

	
map.connect(’:controller/:action/:id’, conditions={'function’:stop_comcast})

David Heinemeier Hansson recently posted an entry about Resources on Rails discussing how important web services are. The other key point was to make it easier to write controllers that could not only give you easy browser access to your resources, but provide a web service API as well.

The two snippets shown above give you an edit and update capability that restricts matching based off the HTTP method (verb). Writing a huge mess of those for the rest of the functions needed for a full web service API like Atom is a bit of busy-body work, so in the opinionated style of Rails a single command wraps up the whole thing. In Routes it looks like this:

map.resource('user')

That will make the two routes at the top of this entry in addition to routes that handle PUT, and DELETE. It maps them out to a set of actions in the controller, and provides the capability to easily add more methods for specific verbs.

The map.resource command is still getting tuned up, and we’re integrating the additional functionality it provides back into Pylons as well. Josh Triplett also wrote some Python code that will parse HTTP Accept headers fully so that we can add some nice functionality to use in the controller to return the appropriate data given what the client is expecting (HTML, XML, JSON, etc.)

If you’re using a Python web framework that doesn’t use Routes… maybe its time to put a request in. :)

Is Rails a DSL? What is a DSL, and is it possible in Python?

I keep seeing blogs and blog comments pop up with some very odd notions. Many of these are summed up in this comment from a recent post by Tim on Ruby books

“It’s doubtful that the Python folks can come up with anything as compelling (or elegant) as Rails. Why? Because Ruby is so good at creating Domain Specific Langauges (DSLs). Ruby’s anonymous code blocks are a big part of what enables DSLs to be written so easily in the language. Python doesn’t have them. Python’s lambda’s (and closures in general) are crippled as well, which also doesn’t help Python’s cause.”

What is a Domain Specific Language (DSL)?

I always like to carefully define my terms before actually talking about them. So first I looked up several common definitions of a DSL. We have Martin Fowler’s entry on a DSL which describes it as:

“The basic idea of a domain specific language (DSL) is a computer language that’s targeted to a particular kind of problem, rather than a general purpose language that’s aimed at any kind of software problem.”

He further clarifies between two types of DSL:

“The most common unix-style approach is to define a language syntax and then either use code generation to a general purpose language, or write an interpreter for the DSL. Unix has many tools that make this easier. I use the term External DSL to refer to this kind of DSL.”

“The lisp and smalltalk communities also have a strong tradition of DSLs, but they tend to do it another way. Rather than define a new language, they morph the general purpose language into the DSL. (Paul Graham describes this well in Programming Bottom-Up.) This Internal DSL (also referred to as an embedded DSL) uses the constructs of the programming language itself to define to DSL.”

To summarize, an Internal DSL satisfies one of the following:

• Uses a general purpose language, then morphs it into a language to fit the domain
• Adds functionality to a general purpose language such that solving a specific domains problem is easier

An External DSL:

• Completely new language syntax
• Typically requires code generation or interpreter for DSL

Martin Fowler espouses on how very dynamic languages like Lisp, Smalltalk, and Ruby are more conducive to internal DSLs as the meta programming capabilities and dynamic nature make it easy to extend and customize the functionality for your domain. I completely agree with his contention that an Internal DSL is limited by the syntax and structure of the language.

DSL’s in Python and Ruby

While the anonymous code blocks in Ruby are useful and syntactically nice, similar functionality can be done in Python through the use of generators and iterators (more powerful generator functionality similar to Ruby’s anonymous blocks are planned for Python 2.5).

Ruby also lets you extend built-in types, which is utilized by Rails to add a few helper functions to core Ruby data types. Python and Ruby both have closures that operate slightly differently, Ruby’s closure is definitely more “full featured”. However this typically isn’t a problem because of list comprehension’s, generator expressions, and generators/iterators.

Zope is a very clear example that its quite possible to build an internal DSL in Python. Whether they built it in a clear, easy to use way is up for someone else to debate. SQLObject makes dealing with databases as easy as ActiveRecord, and there’s many other examples of internal DSL’s built in Python (Take a look at twill, a web application testing language implemented in Python).

Is Rails a DSL?

First, it should be obvious that if Rails is a DSL, its an Internal DSL. The most noticeable strides towards being a web programming DSL in Rails:

• Over a dozen core Ruby classes are extended/modified
• Dozens of helper functions
• New classes (syntactic sugar) defining database access and web paradigms

Given how loose the definition of internal DSL is, I think its clear that Rails qualifies, so does Django, TurboGears, Aquarium, etc.

If you prefer to be more strict about an internet DSL, and argue that a lay programmer needs to be able to do a reasonable amount of ‘work’ without knowing the base language implementing it, Rails starts to move apart. You can do a reasonably large amount of work in Rails without knowing Ruby. A lot can also be done in Zope without knowing Python.

In either case, you can’t really start to make advanced applications in any of the frameworks without knowing the general purpose language its built on. DSL’s can and have been built in Python, and I believe if you go out and start counting them up, you’ll see more DSL’s implemented in Python than in Ruby.

Anyways, hopefully for people that run around re-iterating something they heard somewhere about DSL’s and Python vs. Ruby, this clarified something. It’s easy to create DSL’s in Python, lots of people do, and there’s lots of them. Python and Ruby are both great for making them, outside of Rails I don’t know any other DSL’s in Ruby. Perhaps someone can compile a list of all the internal DSL’s written in both Ruby and Python?

Routes 1.0 almost ready

I’ve almost got a 1.0 ready of Routes. After reading Kevin Dangoor’s post on the mysterious 1.0 I’ve come to the conclusion that Routes is 1.0 ready. In case you aren’t familiar with Routes, I’d suggest taking a look at one of my earlier posts about it.

So where is it? It’s in the latest svn for now, because I’d like to actually have a nicer site with more full fledged documentation before the release. For example, some docs on how framework creators should go about integrating it, as well as more detailed and thorough documentation on usage. I also have to finish up one little change that will make Routes fully compatible with the WSGI spec when the path/script info is split-up (which happens when a WSGI app is put under a URL-space).

Rails routes suffers from a lack of documentation as well, and the most complete source of information is buried in the test units for it. There’s also good docs in the Agile Web Dev for Rails book, but that’s not exactly a free public resource. So I’ll likely be “porting” my docs back to Rails, especially since Nicholas Seckar has been a great help while working on this project.

Whats New?

Short answer, not a lot. The main thing was actually very trivial to implement, named routes. These act essentially as a short-cut for when you want to pull some possibly long pre-defined route defaults. Short-cuts are good of course, as they save you a bit of typing and in this case also make your URL’s more flexible should you decide to change how to get to a “named” route.

Here’s what a fairly basic Route setup looks like:

m.connect(':controller/:action/:id')
m.connect('', controller='home', action='splash')

To implement Named Routes I added the ability to specify an additional string before the keywords. Here’s an example:

m.connect(':controller/:action/:id')
m.connect('home', '', controller='home',action='splash')

Now take a look at using it inside a template:

	

Without named routes
	

url_for(controller='home’,action='splash’)
url_for(controller='home’,action='splash’, id=4)

	

With named routes
	

url_for('home’)
url_for('home’, id=4)

This is slightly different from the Rails approach as I was mainly interested in keeping it “Pythonic”. So there’s no extra symbols or functions created when using a named route.

As you can see, it can save a bit of typing as using a named route is sort of like having a set of keywords inserted for you.

A 1.0 Release

Kevin’s article gave me a lot of food for thought regarding whether I should keep incrementing the Routes version. I can’t see any reason not to just go 1.0 as my version of Routes will be feature equivalent (and then some) with the Rails version, is heavily unit-tested, and is used in a production environment.

I’m confident in the reliability of the code and its being used by quite a few people in production environments (myself included). Being 1.0 doesn’t mean its done, it just means its hit a point functionality wise where I’ve accomplished everything I wanted the “finished” product to have. Even 1.0 software has bugs, and when I think of more features I’d like to add, I’ll start on my way to 1.1 or maybe even 2.0.

Hopefully I’ll have the docs ready to go sometime before November at which point a fully redesigned web-page will be put up. Until then, if any of this has you interested, head over to the Routes site or take a look at the unit tests to get a feel for using it. If you want to see it in action, try out the latest version of Myghty which contains an easy-to-use project template using Routes integration (currently with the 0.2 Routes which is lacking Named Routes).

Best of breed Controllers for MVC web frameworks

I’ve been doing a lot of comparisons and research into various Python web frameworks lately, mainly focused on MVC oriented frameworks. Some of them have templating languages they come with, some not so much. The thing I have started to notice is how similar the Controller in them all is starting to look. First though, lets see what they all can do when it comes to the Controller.

For comparison’s sake, I’m bringing out some examples using CherryPy, Myghty, Bricks, Aquarium, Ruby on Rails, and Django.

Please note I’m not including frameworks built using any of the ones mentioned as their Controller looks pretty much the same. I’m also not an expert in all these frameworks as I only have so much time in the day, so if I seem vague on something its because I don’t know for sure. Feel free to comment (or correct) and clear up any vagueness I express.

Dispatch and Responder Styles

All the frameworks I’ve mentioned implement a style of object publishing (sometimes referred to as dispatch-style) that maps to the URL coming in. If you have another name you want to call this, go for it, I haven’t seen anything consistent on what to call this stuff.

Three different styles seem to exist that make sure an object responds to the URL:

1. User creates a regular expression to explicitly map a URL to an object (*Explicit Mapping*)
   • (r'^polls/(?P<poll_id>\d+)/vote/$', 'myproject.apps.polls.views.polls.vote') (from Django tutorial)
2. Object is mapped out according to the URL (*Implicit Mapping*)
   • /admin/login/hello/ —> admin/login.py/hello() (Myghty)
3. Programmatic rule set determines object that responds (*Programmatic Mapping*)
   • map.connect ‘:controller/:action/:id’ (Rails)
   • m.connect(’:controller/:action/:id’) (Routes)

In the Python world, three different styles also exist for how that object (or function) is defined. The first one is most popular, and is similar to how Rails sets up the Controller. They are:

1. Create a class inheriting from a Controller object, methods of the class instance are called to respond (*Class-method*)
2. Create a callable object that responds (*Callable*)
3. Create a function thats directly called (*Function*)

Note that the last 2 are functionally equivalent for the most part, however a callable object has the typical advantages (or disadvantages) of persistence in some web frameworks, as such if a web framework only shows one of those styles I won’t mention the other. Curious what frameworks use what styles from the above two sets?

For this listing, I’ll refer to the mapping scenario as dispatch-style and the method for defining the response as the responder-style, also please note that some frameworks support multiple dispatch-styles and responder-styles.

Aquarium:

• Dispatch-Style Mapping ¹:
  • Explicit, Implicit, Programmatic
• Responder-Styles:
  • Callable (Though it looks almost exactly like the Class-method style)

Bricks:

• Dispatch-Style Mapping:
  • Implicit
• Responder-Styles:
  • Class-method

CherryPy:

• Dispatch-Style Mapping:
  • Explicit (Not with regexp’s though, each class is attached off the root to determine what URL leads to it)
• Responder-Styles:
  • Class-method (Must be explicitly exposed with @cpg.expose)

Django:

• Dispatch-Style Mapping:
  • Explicit
• Responder-Styles:
  • Function (Packaged in the same module if they’re related)

Myghty:

• Dispatch-Style Mapping ¹:
  • Explicit, Implicit, (Programmatic with Routes in upcoming 0.99)
• Responder-Styles:
  • Class-method, Callable, Function

Rails:

• Dispatch-Style Mapping ¹:
  • Programmatic
• Responder-Styles:
  • Class-method

¹: User can create custom dispatchers to determine responder object

Who did it first?

If we throw in two more frameworks that utilize CherryPy, like Subway, and TurboGear, we can start to see that using a responder-style of Class-method is pretty popular. What’s even more interesting is that all the Class-method Controllers look remarkably similar, and I don’t think its because they’re copying each other.

The Aquarium project appears to be approximately 4 years old now, but from looking at the mailing list had little or no users until only 2-3 years ago (One poster from 2001 describes his aquarium plants turning brown…). From the sourceforge changelogs, the Controller stuff was added with 1.4 release in 2004. CherryPy has been around for 3+ years and I’m presuming it had the Controller stuff in it from the get-go as it seems a pretty core function of CherryPy. Rails has been out for a year or so, and was used in a form internally for BaseCamp for awhile predating that.

If you really want to trace back this style of Controller, you’d most likely find yourself looking at Java Python code (See Phillip J. Eby’s comment below). I’m really just trying to point out here, that even if one of these projects saw something they liked from the other, none of them were the originators. They all took a good concept, and implemented it differently, though they’ve all started to look remarkably similar.

A Controller Line-Up

So lets take a look at some of the examples on the pages of each respective website. Django is the only one that really stands out, as they’re not using a Class to contain the methods, rather they’re grouping them as plain functions inside a module.

Aquarium

class BaseController(Controller):

    def call(self, callNext, *args, **kargs):
        “”“Save a reference to myself in ctx.controller.”“”
        self._ctx.controller = self
        callNext(*args, **kargs)

    def doMessageAddAction(self):
        “”“Add a new message.
        ...does a bunch of stuff…

I mentioned that it was Callable but functioned in a way like a Class-method. This appears to be because Aquarium puts a context and maybe other initialization stuff per request in the __call__ bit, then has it actually calls a different method to respond.

Bricks

from bricks.controller.crud import CRUD
from bricks import user

	
class Test(CRUD): 
    # Override the default delete() method
    def delete(self, rowid):
        return CRUD.delete(self, rowid)
    delete.expose = user(level=1, app='app’)

	
controller = Test('todo’)

Bricks adds an interesting ability here by attaching function attributes to indicate what authorization is needed to execute the function called. Some of the other frameworks shown here do similar things through function decorators.

CherryPy

from cherrypy import cpg

	
class HelloWorld:
    def index(self):
        return “Hello world!”
    index.exposed = True

	
cpg.root = HelloWorld()

Notice that the class instance is attached to the location off cpg.root thats desired. To map a URL to a class instance farther down, say under /booga instead of /, you’d do that last line like so:
cpg.root.booga = HelloWorld()

Pretty handy I think, and it beats writing regular expressions (though doesn’t appear as flexible). One of the reasons I prefer the Programmatic dispatch-style. Moving on..

Django

from django.core.extensions import get_object_or_404, render_to_response
from django.models.polls import choices, polls
from django.utils.httpwrappers import HttpResponseRedirect

	
def vote(request, poll_id):
    p = get_object_or_404(polls, pk=poll_id)
    ... does a bunch more stuff to use Django-supplied functionality…

If you imagined the functions inside a class, instead of a module, it’d look rather similar to the others….

Myghty

class HelloWorld:
    def doit(self, m, **params):
        m.write(“hello world!”)

	
helloworld = HelloWorld()

At this point, the similarity between Bricks, CherryPy, and Myghty should be pretty obvious. Aquarium is close as well, while Django is only a wee farther from what’s appearing to be the norm.

Rails

class HelloController < ApplicationController
    def index
        return render :text => "hello world!"
    end
end

Doesn’t look too different from some of our Python MVC frameworks…

What it All Means

I have no idea. Really, I just thought it was fascinating that these different frameworks started looking so similar. Some of them have been incredibly similar before others, but obviously these approaches catch on and find themselves adopted into other frameworks.

I do start to wonder at what point they’ll be similar enough that the barrier to switching web frameworks is narrowed down to only a few dozen lines of code to change here and there. Sure, going to/from Rails is going to be tough as that also means a language change, but for Python programmers tempted by a feature a different web framework offers I won’t be surprised to see more people hopping back and forth.

With regard to the Python web frameworks, none of them appear to agree on a template language. Several of them use Cheetah, some use Myghty’s template abilities, one uses Kid, and several of them have their very own template language.

It’s also interesting to consider whether or not having more possible dispatch/responder styles is beneficial or detrimental to the popularity of a specific framework. Rails creator David Hansson purposely includes only one style for each as he emphasizes with a concept known as convention over configuration. He provides defaults for pretty much everything, and following conventions makes it easier to write code as it enforces a consistent programming and naming style. I can’t help but notice that of the frameworks I mentioned above, the ones with less (apparent) options and better defaults tend to have more users (With the exception of Bricks as its still in alpha).

You might have noticed I designated Rails as being capable of user-programmed dispatching. While Rails uses its routes system by default, the thing people miss is that they’re just defaults. You can get in there and totally alter the way it works if you want, Ruby doesn’t completely close classes so you could just re-open them and override any aspect of its dispatch system you like.

I think this a valuable lesson that some of the more “verbose” examples above might want to consider. If there’s lines of code in a controller that you have to put in there every-time, find a way to make it the default case so that boiler-plate isn’t needed. It only needs to be a default, an advanced programmer will know how to override it should they need to. There’s obvious limitations to this since things work differently in Python, but there’s still Python ways to do it in a way that feels “right”.

For the Python Web Frameworks I missed

I know, I didn’t cover SkunkWeb, Spyce, Snaklets, Webware and more. Please feel free to add the relevant information regarding your framework in the comments. It’d be helpful if you could indicate what Dispatch-Styles and -Responder-Styles your framework supports.

If I failed to properly cover one of the above frameworks, bringing that to my attention is most appreciated.

The comment system supports the Textile format.

Where’s Single Sign-On? Part 2

In a recent Wired article regarding One Login, reference is made to a new social style network called GoingOn. The article spends most of its time focusing on one site that hopes to aggregate functionality that currently is split between Blogger, Flickr, Friendster, and Bloglines (for the most part). However, the thing it misses is what I previously discussed regarding the lack of a working distributed identity system.

After looking around more, I’m happy to say there are indeed working identity systems out there. Unfortunately the most promised of them, the Liberty Alliance doesn’t seem to have much oomph behind it, but two others that I previously didn’t know about are now out there.

The first is from the folks at Microsoft, which they’ve called an Identity Meta-System (or something like that), which is described over at vnunet. It seems to be rather tied (or at least integrated heavily) to Microsoft technology (go figure!), and will be included in Indigo and other various Micrsoft technologies. As a mainly open-source coder, this has little appeal to me, nor am I about to start using Microsoft API’s to write my websites and web code. The standards utilized by Microsoft for their Federated Identity are generally known as WS-* for some reason I’m too lazy to investigate.

The second is much more appealing (to interested users and web developers), and has actually been around for a very long time in a primitive form (2000 is ancient by web standards). The home site appears to be the identity commons, and the current sole Identity Broker is 2idi, the organization behind the standards is XDI. They’ve made the entire code-base they run the Identity Broker on, open-source under the Affero General Public License to ensure that users are never locked into just one Identity Broker (Yea!).

If you’re curious how the Microsoft and Liberty Alliance methodology differs, idcommons has a useful FAQ addressing the differences.

The most exciting aspect for me, is that all the technology behind the XDI approach is completely open-source, and geared towards maximum user flexibility and empowerment. The user gets to move data between Identity Brokers, and every care has been made to ensure the user is never locked into a single Identity Broker. Actually, the most exciting part, is that it works right now. :)

They’re currently preparing to switch to a SAML-2.0 backed code-base, however the code they have only works from PHP, Java, and Perl. If you want to try it out, here’s how to get an i-Name, and you can try it out on those two sites. Also, a developer made a ISSO (I-name Single Sign-On) authentication system for WordPress which is pretty cool.

So what’s stopping ISSO from being used on more websites? It’s free, its open-source, its standards based, its not controlled by a commercial corporation….

It needs Python libraries!

I should mention, when I first wrote this as far as I knew, there was no Ruby version. There still isn’t a public one, but Victor Grey is fairly close to a Ruby version with a full Rails rig to go with it which I’m rather looking forward to.

Anyone want to help? I’m tired of remembering a zillion usernames and passwords, and with ISSO on the horizon I shouldn’t need to, all the Python web frameworks will be a bit better (at least the sites that use usernames/passwords) with an easy way to use ISSO.

By the way, for a useful overview of SAML, there’s a very detailed write-up of SAML2 on xml.com.

Switched to Typo and some MT3 to Typo Migration Notes

So tonight I took the plunge, and converted the blog to Typo. It’s totally slick, and I’m digging it. Plus when I want to tweak stuff, I don’t have to mess around with Perl.

It’s currently lacking a feature I had grown accustomed to, Technorati auto-pinging. I’m thinking this would be a good thing to contribute, so I’ll start digging into the code and see what needs to be updated.

Migration was also a bit of a hassle, as it appears the MovableType 3 migration script only is happy if you use MySQL for your old and new blog. Getting it migrated to Typo required dumping the MovableType database using pg_dump, and loading it into the Typo database. Then the mt3.rb script needed to be edited to remove all the table prefixes. After this, the script happily imported all my prior posts, comments, and trackbacks.

Porting Routes from Rails

I’ve begun porting the Routes system from Rails over the last weekend. It’s definitely been a good brain workout, as the one Nicholas Seckar wrote that drives Rails is quite a brain-fuck. His version actually does quite a bit of code generation based on the routes one sets up. At first I tried to do a fairly direct port of the Ruby code behind this to Python, but my brain just couldn’t wrap itself around what he was doing (He’s told me that quite a few others have said the same).

This left me kind of discouraged, and I left the project (more of a thought at that point) alone and proceed to replicate most of Rails ActionMailer in Python instead. Partly because I liked how ActionMailer assembled mail, and partly because after hearing a talk by Alex Martelli on Metaclasses and other Python “black magic” I just had to find some excuse to use a metaclass.

Why not just use Rails?

Now, before anyone starts yelling “Just use Rails!” at me, I should explain something. I am using Rails, and I rather like it. However, most companies wouldn’t be too thrilled with their employee’s changing the entire codebase of the company every year when the latest, greatest, way to write webapps comes along. Plus, since I already use Python extensively, a lot of stuff that makes life pleasant in Rails-land can be fairly easily adopted to Python-land.

Do I really want to re-write 5k or more lines of Python libraries in Ruby just so I can use Rails? Should the entire web team learn a new language and framework just because its a bit better than the current way? The time spent doing all this wouldn’t be paid off for at least a year or more, if that (and maybe some other language+framework will be the new thing by then). When it comes down to normal Python libraries, I don’t save any lines of code by moving to Ruby. So in the end, it makes sense to adopt part of Rails that make my life easier.

In Case You Didn’t Know

The Routes system from Rails is used by Rails in a lot of functions. Virtually all functions that require a URL to be written (url_for, link_to, form_tag, etc) all use the Routes system to figure out what URL to put in place.

In the Python world, as far as I know (and I’ve looked quite a bit), there is absolutely no web framework that will convert from controller/action -> URL.

Update (9/26/05): Phillip J. Eby rightfully pointed out that other Python web frameworks can do this. I should’ve been more specific in that none of them operate in this manner. Since putting word out about Routes I also found out Aquarium can designate routes for dispatch/generation in this manner, however Aquarium cannot do automatic route lookup to find the shortest URL. This ends up hard-coding the template URL to the route.

Plenty of web frameworks are happy to convert a regular expression into a dispatch command and let you tweak it (Myghty, Django, etc.) but they are all helpless if you want to generate a URL from the commands they used to dispatch.

Progress

Currently, I have URL generation (controller/action -> URL) and URL recognition (URL -> controller/action) working and passing all the tests that I’ve ported over from the Rails route testing. Since my version of Routes is independent of the framework, its my hope that multiple Python web frameworks will be able to incorporate the finished Route system I’m building. The URL -> controller/action results in a dict, that the web framework can choose to deal with as it pleases.

The first framework I plan on integrating this with is Myghty as the resolver system is fully user-customizable, no hacking on the framework source is needed to plug it in. The dispatch model will then follow Rails and make the same decisions for the user (controller goes in appdir/controllers/, name_controller.py, etc.) that Rails does.

Once I add a few more tests, and try and clean up my code some, I’ll put it out there. I’m considering a BSD license right now, in the meantime, here’s a sample:

>>> from routes import Mapper
>>> m = Mapper()
>>> m.connect('page/:id', controller='content', action='show_page', id =1)
>>> m.connect(':controller/:action/:id')
>>>
>>> m.generate(controller='content',action='index')
'/content'
>>> m.generate(controller='content', action='show_page')
'/page'
>>> m.generate(controller='content',action='show_page',id=4)
'/page/4'

Speed

My implementation choice was to rely heavily on the speed of set operations using the built-in set/frozenset functions in Python 2.4. Under Python 2.3 the sets module provides an ImmutableSet which I’ve also tested my port with. I was actually quite impressed with how fast the sets module is in 2.3, but there is a three-fold speed increase using Python 2.4 built-in sets.

Even under 2.3 the speed should be more than sufficient for most applications, plus, this is a first revision of the code. I’d hardly expect it to be the peak of efficiency right now.

What’s Missing

I currently haven’t implemented memory of the match that got to the current page. Rails does this so if you’re inside a template that was matched by ‘post/show/20’ and you call link_to :id => 30 it would give you a url of ‘post/show/30’. It does this by remembering the match that got you there. Since this match would be setup inside the web framework, I’m considering how to implement it without tying one to a specific framework.

The other noticeable feature missing is Named Routes which act as a sort of macro on top of the normal url generation calls. This shouldn’t be too tough to add at a later point.

Parting Thoughts (ala Jerry Springer)

Before you submit some comment saying I’m an idiot for porting this and should drink more Ruby kool-aid, don’t, please. I’m already drinking it, and it tastes quite good. But just because I like Ruby, doesn’t mean my taste for other languages has changed. Languages have their strong points, and they have things they lack (Yes, they both lack some things). When I’m writing Python, my web framework of choice is so very close to perfection for me, Routes is one of those things that would make it that much sweeter.

If you’re interested in adapting this to your Python web framework and want to start playing with the code I have so far, send me an email and remember boys and girls, be safe.

Making Decisions for Others

Please Note: Reading the last post would help greatly for this one, and this post does end up comparing aspects of Django with Rails in terms of making decisions rather than actual usage.

As I mentioned in my last post, Rails has been greatly helped by the decisions it makes for you. It decides the layout of your application directory, the directories you put stuff in, where you put what parts, etc. These decisions thankfully were done by someone familiar with good application design and had most likely even read Martin Fowler’s Refactoring to Patterns.

What results is a clean, well thought out application that you didn’t even need to think out very much. Before you know it, you fall into these useful and powerful design patterns without even thinking about it (I’d hope you would at least, they’re good patterns for a reason).

A lot of web frameworks make the assumption you are familiar with these patterns already, will setup your application to follow them, and use them in a way that makes sense. Part of the reason for this is because some web framework developers are rather appalled at the thought of making these decisions for the framework user. To be making those decisions for the framework user is looked at as holding their hand, and typically the programmers who write frameworks don’t need any hand holding and don’t expect their users to either (I’m assuming this based on the frameworks I’ve tried).

For example, during Ian Bicking’s test run with Rails he noted that having such a directory hierarchy setup for him put him off a little and mentions people arguing about file layout. I’ve seen very little, if anyone else, even comment on the directory structure that Rails lays out for you. The other irony of this particular post is that a lot of the stuff Ian finds “wrong” about Rails is actually Ruby’s “fault” (Implicit passing of names, mix-in’s, etc.). But moving on….

Ian clearly doesn’t want to lay down the law for users of his software, neither do other Python web framework developers. Having choice in how you lay out your application, with no clear direction what goes where, is considered a good thing. I don’t think there’s anything wrong with this approach, if the users you’re aiming your framework at are expected to be experienced, professional, programmers that are quite familiar with web applications and what works best for them. This does mean the audience will not be that large, and I think thats clearly reflected in the small communities most Python web frameworks have. While their communities are small, I’ve found them to be highly skilled and you get great answers to questions on their respective mailing lists.

The one Python web framework that has almost overnight gone from hardly any users, to at least a few dozen (probably more, but even a few dozen is quite good for a Python web framework), is Django. Django makes a lot of decisions for the user, more so even than Rails by a pretty good amount. While Rails was designed as a web application toolkit, and extracted form a large webapp, Django is designed more as a CMS web application toolkit as its been extracted from newspaper-style web sites. Thus, many of the decisions Django makes for you assume you will want users, user-access to the database objects, etc.

The question is, if someone else is making decisions for you, how important is it that they be good decisions?

One of the decisions the Django team made originally irritated me quite a bit. The Django framework is probably about 95% MVC in nature. It has database files that clearly are acting as Models (which it calls models), logic oriented files traditionally called Controllers (that it calls views), and template files that display the results to the user in a typical View role (though Django calls them templates).

The FAQ on the Django site notes that if you squint, sure enough the observations I just made do pop out, however because of the 5% non-MVC I mentioned they decided the names didn’t work. Rather than calling them by what they obviously were (or closest to), they swapped around names in a way that I think is bound to confuse Django programmers when they go into a real MVC environment and tell someone they edited the view when in fact it was the controller. If they had come up with completely different names that didn’t relate to the MVC paradigm at all, I think that would help to avoid later confusion on the part of a Django user.

Despite the naming of these parts, the decision made for the user is clearly for a MVC-style (or influenced) environment which I think is great ( please change the naming! ;) ). So what is the point of this whole thing?

I think its a good thing for developers of toolkits and especially frameworks to make some more decisions for the end-programmer of their work. This makes it easier for not so experienced programmers to get started quickly, and if you’re using a good design pattern even the experienced dev’s wont have anything to complain about. The two frameworks that currently are getting a lot of attention make a lot of decisions for the programmer using them, and this definitely increases the user-base (assuming you want a large user-base). Even though I know good practices on laying out stuff, it makes my life that much easier when its done for me.

Who’s Attracted By Simplicity?

I’ve been reading a lot of various blogs lately, quite a bit of loud thinking and some 37 signals vs noise as well. A mantra I see over and over regards simplicity, and focusing on doing a few things extremely well. This is reflected in the 37 Signals products, Basecamp, and Backpack. It also bubbles over into the web framework these are all built on, Ruby on Rails.

The Rails framework has a lot of appeal, in large part because of how easy and simple it makes the vast majority of mundane web programming. The sacrifice is completely intentional, and summed up with “Convention over configuration”. This mantra is in the Agile Development with Rails book, and has been uttered on Loud Thinking as well.

Ironically, the people I know that dislike Rails do so because of the “lack of flexibility”. Many decisions have definitely been made for you when you create a Rails application, though I see no harm in this because every decision follows what most regard as the best practice and pattern for that context.

While Rails and Ruby are skills I’m currently learning, I spend a lot of time working with Python. The Python programmers I’ve talked to generally regard lack of flexibility as very unappealing, and would happily sacrifice simplicity or making any decisions for the programmer (I’m referring to Python toolkits/frameworks intended for use by programmers). The assumption is that the programmer will read how to use the tool, see that there’s a half dozen ways to use it, and plug it in as he best sees fit.

Having more ways to use it is a good thing, with no clear direction on any sort of best practice or pattern provided typically. Strangely enough the majority of toolkits and web frameworks for Python generally have rather small groups of users. Is this a coincidence?

My favorite framework, Myghty recently got the very cool ability to call module components implicitly . This added two more mechanisms of dispatch control to the resolver, very powerful stuff. It’s also rather confusing, not because how it works is actually difficult, but because now the way you should use it has become even less certain with the added choices.

The fact that there’s so many ways to use it has definitely hampered the ability of people to understand it and use it effectively. The powerful flexibility has made it powerful though, for totally different paradigms to be built on top of it. For example, you could replicate the pipelining methodology that AxKit uses by configuring the resolver to pass the different extensions through recursively until an “end” is signaled. Or you could configure Myghty to dispatch control to a Controller exactly how Rails does (which I’ve worked on doing).

In the end though, why should I waste my time replicating what another framework already has? I’ve got to admit, even though I know how to pick up the tools and plug them in, I’m still attracted to simplicity.