be Groovie

Routes 0.2 released

Routes 0.2 has been released. The change-log is pitifully short:

• Added prefix option
• Fixed Python 2.3 bug with thread-local singleton

But hey, its a small package to begin with so what the heck. Though its only 0.2 I’m rather pleased with it so far, its performance is great and quite reliable so I’m using it in production environments already.

If you haven’t been following my blog long, Routes is a feature-complete implementation of Rails routes system. I talk more about my reasons for re-implementing Routes in Python in an earlier post so I won’t repeat them all here.

It’s fairly unique in the Python world as it will do a route lookup search to turn a dictionary back into the URL (URL Generation) that will ensure the same values are created. This allows you to generate URL’s from inside your web pages and easily add new URL schemes without touching all your web pages.

The Routes package is aimed directly at integration with Python web frameworks that support the MVC style paradigm as it returns a controller and action value with the assumption your framework will know what to do with it.

Tired of writing big regexp’s to match URL’s to a class/method for dispatch in your webapp? Pester the framework creator to integrate Routes. :)

Here’s some Python Web Frameworks that currently or will shortly have Routes support/integration:

• Aquarium has a URL Connector that uses Routes. Not sure if its been added to the core or only exists as an add-on right now.
• Myghty will have Routes integration packaged with it in 0.99 which is on track to be released this week hopefully. It will also bring Python Paste support and integration which will bring a whole bunch of goodies to webapp developers and users. (Note that Python Paste requires Python 2.4)

I’ll be talking more about Python Paste and why you should care later this week. Any comments/suggestions on Routes are greatly appreciated.

Presentation on SQLObject & FormEncode

I gave a presentation tonight on SQLObject & FormEncode for the Bay Area Python Users Group (BayPiggies). Here are the slides for the presentation, which actually have 3 extra slides I didn’t show during the presentation because I forgot they were there (doh!).

Anyways, there were rather important I think as they actually showed two common cases where you’d want to use SQLObject and FormEncode together. That is, for moving data from a CSV file to a database, and for dealing with web forms. The latter of which I previously described in much more detail on here but it would’ve been nice to go over in the presentation, ah well.

If you’re curious about FormEncode, I put a decent amount of info into it along with some examples. It’s worth checking out if you’re ever in need of data validation and converting data types.

Separating Web Development Environments in Myghty

On many occasions, its quite useful when developing web applications to have the webapp know whether its in a production/development/testing environment. Rails builds this into the framework and its rather easy to add this toggle throughout a Myghty webapp (or some other Python web framework) as well.

We’ll make use of an environment variable so that the webapp environment can be easily configured from lighttpd or Apache. This way just deploying the application under a different handler can toggle the web applications mode of operation. The other thing we’ll setup while we’re at it, is a variable to keep track of the absolute root of our web application. I’ve found it quite useful in many cases to be able to get at this information without hardcoding it in, this way its always accurate no matter where the program is.

There’s a few rather commands that’ll give you the information we’re looking for. It took me awhile to find this, so hopefully it’ll help someone else out there.

Getting our Absolute Location

Let’s assume our directory hierarchy looks like this:

webapp/
   templates/
   components/
   scripts/
      dispatch.fcgi

Maybe you have the script called by mod_python or lighttpd somewhere else, I’m assuming it’ll be inside the root of your web application somewhere. In this case, the handler called by lighttpd is dispatch.fcgi. So dispatch.fcgi needs to figure out what the absolute path of the directory is above it.

Here’s the code that figures this out:

import os

	
Set the prefix to our base path for the webapp
	

myloc = os.path.join(os.getcwd(), file)
prefix = os.path.normpath(myloc + ‘../..’)

The myloc assignment gets the absolute file-name with path of the current file, in this case dispatch.fcgi. Unlike a normal os.getcwd(), this call doesn’t care what directory we happen to be in when we import the module, it will always be the absolute file-name of that file.

In case you’re curious why this happens, __file__ will return the path of the file relative to the working environment its called from. So combining it with the full path name of the current working environment results in the complete absolute path of the module no matter what the current context or working directory.

The prefix assignment uses the normpath call to strip off the filename, and backup one directory to our webapp root. This leaves us with the absolute path to our webapp/ directory.

Setting and Using the Environment variable

Figuring out if we’re in a special runtime environment is quite easy and looks like this:

import os

	
MYGHTY_ENV = os.environ.get('MYGHTY_ENV’) or 'development’

Now we can just test MYGHTY_ENV to determine whether to contain errors ourself, or drop them to the web (as you would want in development mode). We default to being in development mode, since this is what you’d typically run your webapp in.

To switch it to production mode, here’s what the lighttpd config looks like:

fastcgi.server = ( 
   ".fcgi" => (
     "www" => (
        "min-procs" => 2,
        "max-procs" => 4,
        "socket" => "/tmp/webapp.socket",
        "bin-path" => "PATH/TO/webapp/scripts/dispatch.fcgi",
        "bin-environment" => ("MYGHTY_ENV" => "production" ),
        "idle-timeout" => 120
      )
    )
)

To set the extra environment variables with Apache, use mod_env with the SetEnv directive which would look something like this:

# Make sure you have mod_env loaded, this line assumed to be in the VirtualHost
# block of your config
SetEnv MYGHTY_ENV production

At this point, you might’ve noticed (if you’ve used Rails) how similar my Fast CGI setup with lighttpd looks when compared to the some of the Rails examples for a lighttpd + Fast CGI setup. This is intentional, as I’m adding a Routes dispatcher to Myghty so it makes sense to layout my web application in a similar directory hierarchy.

Anytime you need to toggle some behavior depending on your webapp’s runtime context, just import os and check it as I showed up above.

Where’s Single Sign-On? Part 2

In a recent Wired article regarding One Login, reference is made to a new social style network called GoingOn. The article spends most of its time focusing on one site that hopes to aggregate functionality that currently is split between Blogger, Flickr, Friendster, and Bloglines (for the most part). However, the thing it misses is what I previously discussed regarding the lack of a working distributed identity system.

After looking around more, I’m happy to say there are indeed working identity systems out there. Unfortunately the most promised of them, the Liberty Alliance doesn’t seem to have much oomph behind it, but two others that I previously didn’t know about are now out there.

The first is from the folks at Microsoft, which they’ve called an Identity Meta-System (or something like that), which is described over at vnunet. It seems to be rather tied (or at least integrated heavily) to Microsoft technology (go figure!), and will be included in Indigo and other various Micrsoft technologies. As a mainly open-source coder, this has little appeal to me, nor am I about to start using Microsoft API’s to write my websites and web code. The standards utilized by Microsoft for their Federated Identity are generally known as WS-* for some reason I’m too lazy to investigate.

The second is much more appealing (to interested users and web developers), and has actually been around for a very long time in a primitive form (2000 is ancient by web standards). The home site appears to be the identity commons, and the current sole Identity Broker is 2idi, the organization behind the standards is XDI. They’ve made the entire code-base they run the Identity Broker on, open-source under the Affero General Public License to ensure that users are never locked into just one Identity Broker (Yea!).

If you’re curious how the Microsoft and Liberty Alliance methodology differs, idcommons has a useful FAQ addressing the differences.

The most exciting aspect for me, is that all the technology behind the XDI approach is completely open-source, and geared towards maximum user flexibility and empowerment. The user gets to move data between Identity Brokers, and every care has been made to ensure the user is never locked into a single Identity Broker. Actually, the most exciting part, is that it works right now. :)

They’re currently preparing to switch to a SAML-2.0 backed code-base, however the code they have only works from PHP, Java, and Perl. If you want to try it out, here’s how to get an i-Name, and you can try it out on those two sites. Also, a developer made a ISSO (I-name Single Sign-On) authentication system for WordPress which is pretty cool.

So what’s stopping ISSO from being used on more websites? It’s free, its open-source, its standards based, its not controlled by a commercial corporation….

It needs Python libraries!

I should mention, when I first wrote this as far as I knew, there was no Ruby version. There still isn’t a public one, but Victor Grey is fairly close to a Ruby version with a full Rails rig to go with it which I’m rather looking forward to.

Anyone want to help? I’m tired of remembering a zillion usernames and passwords, and with ISSO on the horizon I shouldn’t need to, all the Python web frameworks will be a bit better (at least the sites that use usernames/passwords) with an easy way to use ISSO.

By the way, for a useful overview of SAML, there’s a very detailed write-up of SAML2 on xml.com.

Handling Form data with Formencode + SQLObject

Two of my favorite and most often-used Python packges are formencode and sqlobject. Using these packages together is done fairly often, but I’ve rarely seen any documentation describing a full form display, validation, and insert into a database of new data from said form.

In this example, I’ve tried to wean down the code as much as possible to get a very concise snippet (only 12 lines of web page Controller code) that will:

1. Display a new form to the user
2. Handle form submission
3. Validate the form
4. Coerce form values into the proper types for the database
5. Insert the form data to the database, or
6. Display the form with error messages, and maintain their values

The nice thing about using formencode, is that 3 of the 6 steps I listed above are handled by it in a fairly automatic way. Formencode will check the data given a form schema, coerce the values into the Python types you ask for, fill in errors should they occur, and maintain the existing values.

I’ll be using Myghty for this, but since all I’m really pulling from it is the request args, it should be pretty obvious what to change for whatever web framework makes you happy.

formencode

First, lets take a look at our basic form:

# myform.myt
<html>
<head><title>basic form</title></head>
<body>
<form action="/mypage" method="post">
Username: <input type="text" name="username" size="26" />
            <form:error name="username">
Age: <input type="text" name="age" size="3" />
            <form:error name="age">
<input type="submit" value="Send it" />
</form></body></html>

To validate this, we’ll setup a formencode form schema to run this through. I should note at this point, that the formencode web documentation kind of sucks. However, the doc strings are plentiful, and extremely useful for figuring out which validator to use in addition to having examples of each. To keep things organized, I usually place related form schema classes under the same module and import it as needed.

The other thing you might notice about the form, is that it has form:error fields. These are used by the formencode parser to put in the error message that the validation triggers. This lets us put the error messages right under the boxes they occured in. The drawback is that we have to process the form before first displaying it to strip out the form:error fields.

Here’s our simple schema to validate the above form:

from formencode import schema, validators
class UserInfoSchema(schema.Schema):
    allow_extra_fields = True
    filter_extra_fields = True

    username = validators.String(not_empty = True, max = 50)
    age = validators.Int(not_empty = True)

Hopefully the above should look pretty obvious. The allow_extra_fields bit is needed so that we can pass the entire request argument dict into formencode without it tripping up if there’s “extra” keys it didn’t expect (like the submit button). Since we’re going to be passing the dict we get back from formencode directly to sqlobject, we include filter_extra_fields to remove anything that our sqlobject isn’t going to like.

The form schema needs to include all the fields the database is going to take, since we’re stripping off anything it doesn’t mention. The Int validator not only ensures that the value is an int, but will change it into a Python integer in the process.

sqlobject

Now that we’ve handled validation and value coercion, lets have a look at the sqlobject class. I’m going to “cheat”, and assume your database for this was created like so (in Postgresql):

create table user_info (
    id serial primary key,
    username varchar(50),
    age int(2)
);

Since I’m feeling lazy, we’ll rely on SQLObject to pull the table info from the database giving us a SQLObject class like so:

from sqlobject import *
class UserInfo(SQLObject):
    class sqlmeta:
        fromDatabase = True
</pre>

	
Personally, I think if you just leave the whole thing empty and put pass in for the body, it should assume you want it populated from the database…. but the above will do the trick. Please note I’m using the sqlmeta class to define this, which is used in the recent svn builds of sqlobject. If you use the release on the site, you could replace those two lines with _fromDatabase = True instead.

	
Putting It All Together

	
Now that our form, validation, and sqlobject schema is all done its time for the meat of the matter… the web page controller. Getting this function called will vary depending on your web framework, so I’ll just assume you can figure out how to get it called, here’s what it looks like in Myghty using implicit module components:

from formencode import htmlform
from ourschema import UserInfoSchema
from oursqlstuff import UserInfo

	
def mypage(m):
    html = m.scomp(’/myform.myt’)   # load the form into a string
    form = htmlform.HTMLForm(html, UserInfoSchema())
    if m.request_args:
        form_result, errors = form.validate(m.request_args)
        if errors:
            errorForm = form.render(m.request_args, errors)
            m.write(errorForm)
        else:
            UserInfo(**form_result)  # database insert
            m.subexec(’/thankyou.myt’)
    else:
        m.write(form.render())

	

	
And there you have it. In a brief 12 lines, we handle displaying a new form to a user, and handle form submission, validation, and database insertion while ensuring that the string values are coerced as needed before database insertion. This task is done quite often in web sites, so making this task as painless as possible is a real time saver. 

	
Hopefully this will help out anyone out there, who was wondering about quicker and easier ways to handle cases like this. If you have any thoughts/suggestions on how to streamline this further, be sure to leave a comment.
 

Fragmenting A Framework Userbase

I’ve been thinking a lot lately about web programmers and the web frameworks they choose, or don’t choose, and why. I’m mainly going to talk about Python Web Frameworks as the majority of them have small communites, and possible reasons this could be.

I only started using Python for web development about a year ago, and it took me about a month to settle down on a web framework. In that time, I looked over at least a dozen different frameworks. There’s so many python web frameworks, quite a few people have actually setup entire pages and sections of their site just to covering them all.

I think part of the reason for the proliferation of frameworks is because of the nature of many Python programmers, as I briefly mentioned in a prior post on Making Decisions for Others.

The recent appearence of Django on the Python web framework scene I’m sure has quite a few other Python web framework developers wondering, “Why isn’t the web framework I made getting this much attention and use?”

A Common Base

Many of these same people would like to blame it on hype and good marketing. While that will certaainly boost initial usage, I don’t believe it will create a lasting user base. I think a huge driving factor behind Rails and Django, besides for the hype and marketing, is the fact that both of them make a lot of decisions for you. These decisions start the users all off at a common base of understanding.

The linear progression from:

1. Never used the framework
2. Wrote the tutorial app
3. Wrote their own basic webapp
4. Wrote an advanced web application

Makes it easy for people a step or two up, to help other new users join them. Because the steps they all take are the same steps to achieve greater understanding of the web framework, they can easily help new users get to where they are. Most, if not all the other Python web frameworks I’ve seen are so flexible its hard to have a common base of understanding amongst new users. The process looks more like this:

1. Never used the framework
2. Researched the frameworks options and choices to find a possible starting point
3. Wrote a basic web application using method X
4. Wrote an advanced web app using method X

The flexibility of the web framework becomes an obstacle to a strong user-base in this case, as it fragments the users by the methodology they’re using to build their webapp. It also reduces the common re-usable components available, since different users will utilize different options of the framework and have possibly very different starting points.

Have a Tutorial Application

Also lacking from many Python web frameworks is a clear and obvious Tutorial application. Ideally the front page of a Python web framework should be an obvious path to become an experienced user of said framework. Such as:

1. Install the framework
2. Write a basic tutorial application
3. Look here/there for instruction as need to write your own more complex application

A good tutorial should leave a user feeling confident that they know how to install and start with a common base for writing their own web applications. It’s also amazing how many problems people can have just getting a framework installed and running in a minimal configuration. Having a tutorial that leaves them with a functioning web application gives them a big leap forward.

Since many users will do the first tutorial web application, other new users can give help to even newer users that run into a problem. This is where the common base effect really provides some power.

Methods of Fragmentation

The Python frameworks I’ve tried and used have fragmented their starting points and users in various ways. All of them as a result of their “flexibility and power”. Here are a few common trends of fragmentation I’ve seen:

• Let the user choose various template language schemes (Use ZPT, or Cheetah, or…)
• Let the user choose from web paradigm (MVC, page-driven, pipelined…)
• No base or example configuration for a fully working webapp (So everyone sets up their first application slightly differently)

The last one I listed, is probably the easiest to solve, especially with useful web framework template creators like Python Paste. Obviously, removing the first two will be seen by many Python web framework developers as undesirable. I think it’d really help the users though, as it gives them more in common with each other. If they all use the same paradigm, and the same template language with your framework, their ability to help each other increases and they feel confident they made the “right” choice as well.

Assumptions

I’ve assumed for the purpose of this post, that Python web framework makers are interested in having a large user-base. This isn’t always the case, I’m sure some just want a small, very experienced user-base that isn’t going to be asking basic questions like, “I can’t connect to my database like you show in the tutorial”.

I can understand that, but for the other Python web framework makers out there, try and consider some of the things I mentioned. There are a lot of Python coders out there, and a lot of them can live without having 4 template language choices and 2 different design paradigms. So when adding that feature that’d let people get so much “power and flexibility”, will it fragment your user-base?

Language Specific Comparisons

I’ve read quite a bit of Paul Graham’s articles regarding Lisp, how awesome Lisp is, how much of a dufus one might be for using a language programmed for dufuses.

Now, before I start, I should mention that these posts are quite old, and I don’t really want to start a flame fest over this again. However, I couldn’t help but notice today with a problem I had in my code, how easy Python made the solution (Zach points out the Lisp solution is quite succinct as well).

Without a doubt, Lisp excels at recursion, function/code generation, and closures. This frequently leaves Lisp looking like a god when you see how many lines of code other languages take to replicate the examples Paul chooses to compare (which always revolve around the things Lisp excels at as Paul Prescod points out)

Today, I came across a fairly common case, where I had a function taking keyword arguments and collecting them all. That would look like this in Python:

def somefunc(**kargs):

Now, if I want to take two of those keyword being passed in, and set some defaults so they’re not required but will always have something set in the function body, its rather easy:

def somefunc(keyone='default', keytwo='anotherdefault', **kargs):

So in one line, I have now added two defaults that will be available in my function body for use. How succinct is this in Lisp? What about in Ruby?

I cite two languages that came off very well in the Accumulator Generator shoot-off. They Ruby doesn’t do quite as well in this case, which I’ve actually encountered far more than the code generation cases Graham is apt to cite. (I actually like Ruby and am now using it quite a bit, I’ll be quite happy when it has keyword args)

What’s even worse is where in this thread that I referenced above, Graham at the end says he has no clue how one would create a basic class to handle accumulation in Lisp. I find that rather disturbing that something so easy in Python has Graham saying, “God only knows.” how to do it in Common Lisp Richard points out below that this is because PaulG is rather keen on macros, and not so big on OO. That makes complete sense to me as Lisp did not start out with OO features, those were added later when it was all the rage. When working in an elegant functional language like Lisp I can see why one would never have to consider OO.

That alone should indicate that many technical comparisons between languages can be easily skewed towards a language by using examples that heavily favor built-in abilities of the language one chooses to boast about.

In the end, I’m left with the belief that different languages have different applications. Claiming one language is the be-all, and is always better for any task is about as false as claiming that a language has no problems or issues.

Within certain realms it does make sense to compare languages, scripting vs scripting, functional vs functional, etc. But leaping to a comparison of functional/dynamic-typed vs non-functional/static-typed is typically going to result in some strange claims.

Anyways, if you feel like commenting, try and come up with an example of where Language X (that you use) has a very succinct solution compared to Language Z (all the others). It’d be great to compare some examples and see areas in which different languages fall flat on their face when it comes to succinctness. (Ie, in most dynamic languages, you’d have to add several lines of code to ensure variables are the type you want. A feature/annoyance of static-typed languages)

Update: An anonymous user kindly informs me that there’s no foundation for my claim that some languages are better in certain realms than others, unfortunately the anonymous user fails to say why.

Ruby has no keyword arguments currently (Ruby 2 will have them and keyword collectors ** as Python does). To even approximate my Python example in Ruby, you’d first need to declare the argument as optional which has the side effect of packaging it into an Array. Wheras in Python **kargs packages up the rest of the key/vals under a dictionary. If someone would like to write out the full translation in Ruby, I’d be happy to put it up here, but I doubt its going to be pretty (until Ruby 2).

Zach was helpful and provided an example showing that the specific task I cited is fairly short in Lisp as well, looking like this:

(defun somefunc (&rest kwargs
                 &key (keyone "default") (keytwo "anotherdefault")
                 &allow-other-keys)
                 ; ...
                 )

I would like to make it very clear that my point is not that Python is better, but that technical comparisons can be warped to favor certain languages. This is the same point Paul Prescod makes, and what I’d actually like to see is more technical comparisons that make this point obvious.

Porting Routes from Rails

I’ve begun porting the Routes system from Rails over the last weekend. It’s definitely been a good brain workout, as the one Nicholas Seckar wrote that drives Rails is quite a brain-fuck. His version actually does quite a bit of code generation based on the routes one sets up. At first I tried to do a fairly direct port of the Ruby code behind this to Python, but my brain just couldn’t wrap itself around what he was doing (He’s told me that quite a few others have said the same).

This left me kind of discouraged, and I left the project (more of a thought at that point) alone and proceed to replicate most of Rails ActionMailer in Python instead. Partly because I liked how ActionMailer assembled mail, and partly because after hearing a talk by Alex Martelli on Metaclasses and other Python “black magic” I just had to find some excuse to use a metaclass.

Why not just use Rails?

Now, before anyone starts yelling “Just use Rails!” at me, I should explain something. I am using Rails, and I rather like it. However, most companies wouldn’t be too thrilled with their employee’s changing the entire codebase of the company every year when the latest, greatest, way to write webapps comes along. Plus, since I already use Python extensively, a lot of stuff that makes life pleasant in Rails-land can be fairly easily adopted to Python-land.

Do I really want to re-write 5k or more lines of Python libraries in Ruby just so I can use Rails? Should the entire web team learn a new language and framework just because its a bit better than the current way? The time spent doing all this wouldn’t be paid off for at least a year or more, if that (and maybe some other language+framework will be the new thing by then). When it comes down to normal Python libraries, I don’t save any lines of code by moving to Ruby. So in the end, it makes sense to adopt part of Rails that make my life easier.

In Case You Didn’t Know

The Routes system from Rails is used by Rails in a lot of functions. Virtually all functions that require a URL to be written (url_for, link_to, form_tag, etc) all use the Routes system to figure out what URL to put in place.

In the Python world, as far as I know (and I’ve looked quite a bit), there is absolutely no web framework that will convert from controller/action -> URL.

Update (9/26/05): Phillip J. Eby rightfully pointed out that other Python web frameworks can do this. I should’ve been more specific in that none of them operate in this manner. Since putting word out about Routes I also found out Aquarium can designate routes for dispatch/generation in this manner, however Aquarium cannot do automatic route lookup to find the shortest URL. This ends up hard-coding the template URL to the route.

Plenty of web frameworks are happy to convert a regular expression into a dispatch command and let you tweak it (Myghty, Django, etc.) but they are all helpless if you want to generate a URL from the commands they used to dispatch.

Progress

Currently, I have URL generation (controller/action -> URL) and URL recognition (URL -> controller/action) working and passing all the tests that I’ve ported over from the Rails route testing. Since my version of Routes is independent of the framework, its my hope that multiple Python web frameworks will be able to incorporate the finished Route system I’m building. The URL -> controller/action results in a dict, that the web framework can choose to deal with as it pleases.

The first framework I plan on integrating this with is Myghty as the resolver system is fully user-customizable, no hacking on the framework source is needed to plug it in. The dispatch model will then follow Rails and make the same decisions for the user (controller goes in appdir/controllers/, name_controller.py, etc.) that Rails does.

Once I add a few more tests, and try and clean up my code some, I’ll put it out there. I’m considering a BSD license right now, in the meantime, here’s a sample:

>>> from routes import Mapper
>>> m = Mapper()
>>> m.connect('page/:id', controller='content', action='show_page', id =1)
>>> m.connect(':controller/:action/:id')
>>>
>>> m.generate(controller='content',action='index')
'/content'
>>> m.generate(controller='content', action='show_page')
'/page'
>>> m.generate(controller='content',action='show_page',id=4)
'/page/4'

Speed

My implementation choice was to rely heavily on the speed of set operations using the built-in set/frozenset functions in Python 2.4. Under Python 2.3 the sets module provides an ImmutableSet which I’ve also tested my port with. I was actually quite impressed with how fast the sets module is in 2.3, but there is a three-fold speed increase using Python 2.4 built-in sets.

Even under 2.3 the speed should be more than sufficient for most applications, plus, this is a first revision of the code. I’d hardly expect it to be the peak of efficiency right now.

What’s Missing

I currently haven’t implemented memory of the match that got to the current page. Rails does this so if you’re inside a template that was matched by ‘post/show/20’ and you call link_to :id => 30 it would give you a url of ‘post/show/30’. It does this by remembering the match that got you there. Since this match would be setup inside the web framework, I’m considering how to implement it without tying one to a specific framework.

The other noticeable feature missing is Named Routes which act as a sort of macro on top of the normal url generation calls. This shouldn’t be too tough to add at a later point.

Parting Thoughts (ala Jerry Springer)

Before you submit some comment saying I’m an idiot for porting this and should drink more Ruby kool-aid, don’t, please. I’m already drinking it, and it tastes quite good. But just because I like Ruby, doesn’t mean my taste for other languages has changed. Languages have their strong points, and they have things they lack (Yes, they both lack some things). When I’m writing Python, my web framework of choice is so very close to perfection for me, Routes is one of those things that would make it that much sweeter.

If you’re interested in adapting this to your Python web framework and want to start playing with the code I have so far, send me an email and remember boys and girls, be safe.

Making Decisions for Others

Please Note: Reading the last post would help greatly for this one, and this post does end up comparing aspects of Django with Rails in terms of making decisions rather than actual usage.

As I mentioned in my last post, Rails has been greatly helped by the decisions it makes for you. It decides the layout of your application directory, the directories you put stuff in, where you put what parts, etc. These decisions thankfully were done by someone familiar with good application design and had most likely even read Martin Fowler’s Refactoring to Patterns.

What results is a clean, well thought out application that you didn’t even need to think out very much. Before you know it, you fall into these useful and powerful design patterns without even thinking about it (I’d hope you would at least, they’re good patterns for a reason).

A lot of web frameworks make the assumption you are familiar with these patterns already, will setup your application to follow them, and use them in a way that makes sense. Part of the reason for this is because some web framework developers are rather appalled at the thought of making these decisions for the framework user. To be making those decisions for the framework user is looked at as holding their hand, and typically the programmers who write frameworks don’t need any hand holding and don’t expect their users to either (I’m assuming this based on the frameworks I’ve tried).

For example, during Ian Bicking’s test run with Rails he noted that having such a directory hierarchy setup for him put him off a little and mentions people arguing about file layout. I’ve seen very little, if anyone else, even comment on the directory structure that Rails lays out for you. The other irony of this particular post is that a lot of the stuff Ian finds “wrong” about Rails is actually Ruby’s “fault” (Implicit passing of names, mix-in’s, etc.). But moving on….

Ian clearly doesn’t want to lay down the law for users of his software, neither do other Python web framework developers. Having choice in how you lay out your application, with no clear direction what goes where, is considered a good thing. I don’t think there’s anything wrong with this approach, if the users you’re aiming your framework at are expected to be experienced, professional, programmers that are quite familiar with web applications and what works best for them. This does mean the audience will not be that large, and I think thats clearly reflected in the small communities most Python web frameworks have. While their communities are small, I’ve found them to be highly skilled and you get great answers to questions on their respective mailing lists.

The one Python web framework that has almost overnight gone from hardly any users, to at least a few dozen (probably more, but even a few dozen is quite good for a Python web framework), is Django. Django makes a lot of decisions for the user, more so even than Rails by a pretty good amount. While Rails was designed as a web application toolkit, and extracted form a large webapp, Django is designed more as a CMS web application toolkit as its been extracted from newspaper-style web sites. Thus, many of the decisions Django makes for you assume you will want users, user-access to the database objects, etc.

The question is, if someone else is making decisions for you, how important is it that they be good decisions?

One of the decisions the Django team made originally irritated me quite a bit. The Django framework is probably about 95% MVC in nature. It has database files that clearly are acting as Models (which it calls models), logic oriented files traditionally called Controllers (that it calls views), and template files that display the results to the user in a typical View role (though Django calls them templates).

The FAQ on the Django site notes that if you squint, sure enough the observations I just made do pop out, however because of the 5% non-MVC I mentioned they decided the names didn’t work. Rather than calling them by what they obviously were (or closest to), they swapped around names in a way that I think is bound to confuse Django programmers when they go into a real MVC environment and tell someone they edited the view when in fact it was the controller. If they had come up with completely different names that didn’t relate to the MVC paradigm at all, I think that would help to avoid later confusion on the part of a Django user.

Despite the naming of these parts, the decision made for the user is clearly for a MVC-style (or influenced) environment which I think is great ( please change the naming! ;) ). So what is the point of this whole thing?

I think its a good thing for developers of toolkits and especially frameworks to make some more decisions for the end-programmer of their work. This makes it easier for not so experienced programmers to get started quickly, and if you’re using a good design pattern even the experienced dev’s wont have anything to complain about. The two frameworks that currently are getting a lot of attention make a lot of decisions for the programmer using them, and this definitely increases the user-base (assuming you want a large user-base). Even though I know good practices on laying out stuff, it makes my life that much easier when its done for me.

Who’s Attracted By Simplicity?

I’ve been reading a lot of various blogs lately, quite a bit of loud thinking and some 37 signals vs noise as well. A mantra I see over and over regards simplicity, and focusing on doing a few things extremely well. This is reflected in the 37 Signals products, Basecamp, and Backpack. It also bubbles over into the web framework these are all built on, Ruby on Rails.

The Rails framework has a lot of appeal, in large part because of how easy and simple it makes the vast majority of mundane web programming. The sacrifice is completely intentional, and summed up with “Convention over configuration”. This mantra is in the Agile Development with Rails book, and has been uttered on Loud Thinking as well.

Ironically, the people I know that dislike Rails do so because of the “lack of flexibility”. Many decisions have definitely been made for you when you create a Rails application, though I see no harm in this because every decision follows what most regard as the best practice and pattern for that context.

While Rails and Ruby are skills I’m currently learning, I spend a lot of time working with Python. The Python programmers I’ve talked to generally regard lack of flexibility as very unappealing, and would happily sacrifice simplicity or making any decisions for the programmer (I’m referring to Python toolkits/frameworks intended for use by programmers). The assumption is that the programmer will read how to use the tool, see that there’s a half dozen ways to use it, and plug it in as he best sees fit.

Having more ways to use it is a good thing, with no clear direction on any sort of best practice or pattern provided typically. Strangely enough the majority of toolkits and web frameworks for Python generally have rather small groups of users. Is this a coincidence?

My favorite framework, Myghty recently got the very cool ability to call module components implicitly . This added two more mechanisms of dispatch control to the resolver, very powerful stuff. It’s also rather confusing, not because how it works is actually difficult, but because now the way you should use it has become even less certain with the added choices.

The fact that there’s so many ways to use it has definitely hampered the ability of people to understand it and use it effectively. The powerful flexibility has made it powerful though, for totally different paradigms to be built on top of it. For example, you could replicate the pipelining methodology that AxKit uses by configuring the resolver to pass the different extensions through recursively until an “end” is signaled. Or you could configure Myghty to dispatch control to a Controller exactly how Rails does (which I’ve worked on doing).

In the end though, why should I waste my time replicating what another framework already has? I’ve got to admit, even though I know how to pick up the tools and plug them in, I’m still attracted to simplicity.