be Groovie

Viewing posts for Myghty

Feb 07 2006

Emulating Ruby’s anonymous blocks with Myghty

Ruby’s anonymous block capability is probably the main feature I find myself wishing Python had on more than one occasion. While the upcoming Python 2.5 PEP 323 provides for the “with” statement which will enhance generators to get a bit closer. Though as Ryan Tomayko notes, this still doesn’t make the block available within the “generator”.

There are of course, many uses for having the block (with its closure scope), available in the function/generator you’re using. I’ve seen this used to easily register code call-backs with a CleanUp/Initializaiton manager, and other cases where its preferable to actually retain the block in its entirety for later execution. It’s also very useful when you want the function to control execution of the block, and return its output in a modified form.

In these ways, it would appear that the generator enhancements won’t quite be bringing the full power of Ruby anonymous blocks to Python. However, I recently found out that a relatively recent feature in my favorite Python template language, Myghty, implements something quite close to this.

Myghty’s Component Calls with Content

In a previous entry on Formencode and SQLObject I noted how useful the component call with content can be. What I failed to note (I didn’t know at the time as well), was that not only is the function/component able to get the content of the content “block”, but its also able to execute it again and again.

Consider this example from my prior entry:


Hi, lets translate the content under:
<&| MODULE:mylib:translate, lang='es' &>
    This entire block of content will be sent in as a 
    single variable to translate.myt
    for use. This includes any <b>HTML tags and such</b> as well.
</&>

and the function it calls:


from mytranslater import translated

def translate(m, lang): body = “

The translated text is:

“ body += translated(lang, m.content()) m.write(body)

The m.content() call can be called as many times as you want the output of it, and it retains the scope of its original location. This in many ways emulates how Ruby can yield to the block and capture its output, however it is not possible to stash the block itself (in Myghty/Python).

Next up is to actually be able to use values from the function inside the block. In Ruby this is done in a very elegant fashion letting you declare what you’ll call them, then use them, like so:


SomelistOfints.each do |item|
    item += 2
end

To emulate this behavior, our block has to make a special function call and know the name of the value in advance. Here’s an example:


% randomvar = 423
<&| MODULE:mylib:translate, lang='es' &>
    This entire block of content will be sent in 
    as a single variable to translate.myt for use. 
    This includes any <b>HTML tags and such</b> as
    well. Here's something supplied by
    translate: <% m.content_args['value'] %>.
    Of course, <% randomvar %> is still in scope too.
</&>

And just for fun, our translate function will call the block with some different values:


def translate(m, lang):
    body = "<p>The translated and repeated text is:</p>"
    for val in range(0,4):
        body += m.content(value=val)
    body += "<p>That's it, nothing else.</p>"
    m.write(body)

The important thing to remember that makes the concept powerful is that the block above is called in the scope you saw it in the template. Whatever variables were available there are used as normal.

Close…

This method brings us close to Ruby’s anonymous blocks, as close as might be possible in Python. Unfortunately its only usable within Myghty (if not, please let me know), and its still not true anonymous blocks. At the very least, its close enough to make me happy for now. While I could just switch to Ruby entirely, there’s still way too many things about Python that I’d miss.

From the details of PEP 343, it appears that this full capability to pass the block in was purposely avoided as having flow control in the “macro” makes the code inscrutable. Hopefully someday the utility and power such functionality provides will result in it being available in Python. Or at the very least, some clever person can try a variant in PyPy and see what sticks.

And yes, I know nested functions can be used and passed around with their closure, but its pretty annoying to be nesting functions back and forth solely for that purpose. It feels like all we have is a hammer…

Further reference:

Myghty docs on Component-Call-With-Content Arguments

Comments
Dec 14 2005

Handling Form data w/Formencode+SQLObject Redux

In a prior post on handling form data I covered how to easily populate a form using FormEncode/SQLObject and validate/save the data.

One of the things I noted was that I had to take the HTML form output first, and use htmlfill with it to populate it with defaults and errors before displaying. This required some extra lines I wasn’t to thrilled with in the controller. After an evening of chatting with Ian Bicking about whether FormEncode could somehow be cleaned up to make this easier, Ian suggested I use a Myghty feature I had forgotten about.

Component Calling with Content blocks in Myghty

Somewhat ironic since I use Myghty all day and Ian doesn’t (afaik). It’s a very useful feature called Component Calls with Content.
It’s probably easiest to understand the cool ability this provides by seeing it. In Myghty, each template is known as a component, and complex arrangements can easily be put together by component inheritance and component calls between them.

Normal Python function calls and expressions in a Myghty component (template) looks like this:


Hi everyone, 2 + 2 = <% 2+2 %> and your name is <% lookup_name() %>
</pre>

	

To include content from other components, you can either call the component through the function call m.comp('/some/template.myt') or you can use the component call syntax:


Hi everyone
  <& /sidebar.myt &>

So here’s how we will use a component call with content:


Hi, lets translate the content under:
<&| /translate.myt, lang='es' &>
    This entire block of content will be sent in as a single variable to translate.myt
    for use. This includes any <b>HTML tags and such</b> as well.
</&>

And the translate.myt template might look like:


The translated text is:


lang
%args>

from mytranslater import translated
content = m.content()
%init>

This is probably a lot to absorb, as it utilizes a few different Myghty concepts. Components can have arguments they expect, here the translate.myt component expects a lang arg which is passed in. The content block is then available as m.content() inside the template. The <%init> block is called first and variables defined there are available inside the template.

So how does this help us with the original problem?

Module Components

So far, all of these abilities are possible in Myghty’s ancestor, Mason. One more powerful construct available in Myghty however is the ability to not only call components, but Module Components.

I’ve been familiar with these for awhile as I use them for Controllers. It had never occurred to me though, that they can call any function in a module as if it was a component, not just ‘Controllers’.

Module components allow you to do a variety of interesting things, mainly, calling functions, classes, or objects that are in Python modules. Let’s take a look at the last translate.myt example using this approach instead.


Hi, lets translate the content under:
<&| MODULE:mylib:translate, lang='es' &>
    This entire block of content will be sent in as a single variable to translate.myt
    for use. This includes any <b>HTML tags and such</b> as well.
</&>

And the mylib Python module (assumed to be in the search path):


from mytranslater import translated

def translate(m, lang): body = “

The translated text is:

“ body += translated(lang, m.content()) m.write(body)

Myghty will examine the function signature to determine what variables it wants and will search the current scope to make sure they’re passed in. Very handy. :)

Putting it All Together

So let’s see how this helps us out, first, rather than having to take the form and render it in the controller, we’ll push this into the template using a component call with content. So our new myform.myt looks like:


	
  1. myform.myt


basic form



Username:

Age:



&>
Comments
Oct 21 2005

Editing Myghty with TextMate

TextMate is a rather slick little text editor, with a bunch of cool automation stuff thats great for programmers. You’ve probably already seen it if you’ve watched any of those demo movies for some of the latest web frameworks.

Anyways, as I mainly use Myghty, I needed a syntax highlighting mode for its format. So I made a Myghty bundle for TextMate. This should also be useful for those using Mason as only a few minor adjustments regarding the underlying language highlighter need to be tweaked (Change source.python -> source.perl). This also has a few automation snippets to speed things up.

To install:

  • Unzip
  • Drag into ~/Library/Application Support/TextMate/Bundles/
  • Restart TextMate (Not absolutely sure this is needed though)

Enjoy!

Comments
Oct 18 2005

Hooked on Myghty

I’ve been programming web sites for many years, and have yet to come across a templating language as appealing as Mason / Myghty. To avoid confusion I’m going to talk about Myghty, but since its a direct port of Mason (plus some MVC stuff) all of my comments apply to Mason as well (unless otherwise noted). So if you find yourself stuck using Perl (or you prefer Perl) and something here sounds appealing, by all means use Mason as I did.

Despite Myghty only having come into existence approximately 14 months ago, the code-base is stable, very quick, and has been running in production environments for over 8 months. This is mainly because it started as a very direct port of Mason to Python, it then grew a few additional features that made it great for MVC use. The methodologies present in Mason (thus Myghty as well) are known to scale to very large and complex sites, as this list of Mason-powered sites shows. But it’s the little things added up that really make Myghty my template language of choice.

This is a rather lengthy post as I highlight and explain some core concepts of Myghty, please bear with me… also, if you’d like to follow along and try the examples out, its really easy to get started using Myghty with Paste.

Syntax

Template languages vary a lot in style, there are the very basic string replacement template languages all the way to more advanced template languages. Myghty definitely weighs in on the latter as it has many of its own concepts with regards to templating that produce a very robust and advanced templating system.

The syntax itself is also very appealing to me, as I’m a fan of templating languages that keep their guts inside < > signs. The only exception to this being if you need a quick line of Python, which is done just by having a % in front of it. Here’s what a loop would look like:


% for person in people:
   <b>Hi</b><% person %>
% #end

You’ll notice an additional line is needed to indicate the end of the loop. This is because Python uses white-space to determine blocks and Myghty needs to know when the indentation is over.

Components and Inheritance

In Myghty, each template is more properly referred to as a Component. When a component is directly called as a request (or sub-request), inheritance is applied. This is how a site’s skin is typically applied. Rather than having to specify in your template that it includes or extends some other template, in Myghty your component automatically inherits from an autohandler file above it.

An easier way to think of it is to consider your template directory layout as a Class, and all the templates in it are methods. Every directory inside that root one is another Class, and so on. The autohandler in this context acts much like your __init__ method. Here’s a little example:


	
  1. /autohandler



Sitename


% m.call_next()
Comments
Oct 04 2005

Python Paste Power

Mmm, tasty, a headline of P-word’s. Recently Ian Bicking went on a bit of a release spree with a whole bunch of goodness that I’m way too lazy to link to in this paragraph. The ones I’ve been waiting for were all in the line-up: Paste, Paste Script, and Paste Deploy. I blogged about paste and setuptools earlier and hadn’t followed up as I indicated I would partly because I was waiting on their official release.

So, what’s the big deal? Paste and its buddies solve a host of issues that commonly confront Python web developers and web administrators, the front page of Python Paste does a good job of explaining briefly why each group should care. Thus, I’ll move directly onto the fun that comes with using them.

Yes, I think its fun, but that might be just because I have a thing for installing web frameworks…. so I’ll cover using Paste as two groups of users, as a Web Developer, and as a Web Administrator.

Using Paste as a Web Developer

First off, to create a web application we need to setup a directory for the project. Since we want our web application to be Paste-enabled, the template for the web application should be using setuptools. That way when we package up our web application and give/sell it on the Internet, the people (Web Admins) using it will be able to install and run it easily.

So… lets see what framework I should use to whip up this little web application. I’ll run a paster command that lets me see what Paste-enabled web frameworks I have installed that come with new web application directory templates:


% paster create --list-templates
Available templates:
  basic_package:            A basic setuptools-enabled package
  myghty_modulecomponents:  Module Component Template
  myghty_routes:            Routes Template
  myghty_simple:            Simple Template
  paste_deploy:             A web application deployed through paste.deploy
  pylons:                   Pylons application template
  turbogears:               TurboGear application template

Cool, right? Well, maybe its just cause I like having a big toolbelt of web frameworks sitting around. Obviously not a lot of web frameworks have paste templates available yet, but more are adding support and I’ll be rather excited when that list is 20 or more long.

If you go and install the Paste packages, your list won’t be quite as long as mine (insert evil laugh) as I’m running the CVS of Myghty (0.99 release with these templates coming shortly), along with some other stuff I’ve whipped up. Merely installing a web framework package that’s Paste-enabled and has templates available will make it show up in that list.

Another interesting thing to consider here, is that there’s no requirement that only web frameworks are allowed to create directory templates. Maybe a web application you’re working on, is so powerful you want to make it easy for end-users to extend it with their own custom add-ons. You could provide a paster template that creates a plug-in ready directory template for custom themes. There’s a ton of power packed in the various ways you can extend these tools, building on the dynamic discovery stuff in setuptools.

Moving along though, I’ll go ahead and use Turbogears.


% paster create —template=turbogears pygo
Selected and implied templates: TurboPaste#turbogears TurboGear application template

Variables: package: pygo project: pygo
Creating template turbogears Creating ./pygo/ Recursing into package Creating ./pygo/pygo/ Copying init.py to ./pygo/pygo/__init__.py Copying model.py to ./pygo/pygo/model.py Recursing into templates Creating ./pygo/pygo/templates/ Copying init.py to ./pygo/pygo/templates/__init__.py Copying package-start.py_tmpl to ./pygo/pygo-start.py Copying dev.cfg_tmpl to ./pygo/dev.cfg Copying prod.cfg to ./pygo/prod.cfg Copying setup.py_tmpl to ./pygo/setup.py
Running /opt/local/bin/python setup.py egg_info

As you can see, it just setup our new project for us, which I called pygo because it was the first name that popped into my head. If you’ve actually used TurboGears, looking at that should have raised some red flags as its missing a bunch of stuff that a TurboGears project needs. This is mainly because the template was created by Ian as a demonstration of how to make a Turbogears-style template. Hopefully an upcoming version of TurboGears will be Paste-enabled so I can create a new project like this (hint hint). :)

Ah well, I’ll just make a basic Myghty project using Routes instead:


% paster create —template=myghty_routes pygo
Selected and implied templates: Myghty#myghty_routes Routes Template

Variables: package: pygo project: pygo
Creating template myghty_routes Creating ./pygo/
...Bunch more files…

So that’s all there is to creating new projects with the web framework of your choice (if its Paste-enabled). How do we go and start it up?


~% cd pygo
~/pygo% paster serve server.conf 
Starting server in PID 6090.

In the case of the myghty_routes template, it starts the server on port 5000. A quick look at the server.conf file makes it obvious:

[server:main]
use = egg:PasteScript#wsgiutils
host = 127.0.0.1
port = 5000

[app:main]
use = egg:pygo#paste

That’s really all the Paste you have to worry about as a web developer (though it has even more capabilities you’ll probably want to use). We can see here that its using wsgiutils from PasteScript to run the server. You can easily swap that out for any of the server support that flup offers such Fast CGI, SCGI, or AJP.

At this point, as a web developer, you’d go ahead and create your web application. Especially since the default myghty_routes template is pretty boring if you don’t put anything in it. But for this example, we’ll assume its done and distribute it as a single egg file for people:


~/pygo% python setup.py bdist_egg
running bdist_egg
running egg_info
... whole bunch of stuff here...
~/pygo% ls dist/
pygo-0.0.0-py2.4.egg

That’s it. This is easy, right? You’re ready to go ahead and give your egg to anyone running Python 2.4 now (You could make a source distribution and upload it to Cheese Shop just as easily). So let’s try and totally forget that we’re a web developer, and assume a different role.

Using Paste as a Web Administrator

Ah yes, the joys of setting up web applications you come across. Well, I’ve rarely gotten any joy out of it at least. Let’s take a look at how Paste does make it a lot easier. First, we’ll need to install the insanely useful Pygo webapp that some other thoughtful user created. If this Pygo application was released and uploaded to the Cheese Shop, I could install it like so:


% sudo easy_install Pygo                                  Searching for Pygo
Reading http://www.python.org/pypi/Pygo/
Best match: Pygo 0.0
Downloading http://cheeseshop.python.org/packages/2.4/P/Pygo...
Processing pygo-0.0.0-py2.4.egg
... more stuff happens...
Installed /usr/local/lib/python2.4/site-packages/pygo-0.0.0-py2.4.egg
Processing dependencies for Pygo

Now, since I didn’t actually upload it to Cheese Shop, I’ve faked that screen. But that’s pretty close to how it would’ve looked if I had released pygo and uploaded it to Cheese Shop.

After this one command, as a web administrator, we now have the pygo application installed. There shouldn’t be any need to be installing web applications over and over for every user that wants to run it as the vast majority of web applications get their customization and settings from a database.

What needs to be configured, is the instance of the web application that each user is running. We can set that up for each user inside our Paste configuration file like so:


[server:main]
use = egg:PasteScript#flup_scgi_thread
host = 127.0.0.1
port = 3000

[composit:main]
use = egg:Paste#urlmap
/blog/fred = fredpygo
/blog/janet = janetpygo

[app:fredpygo]
use = egg:pygo
database = mysql:/username@localhost/database

[app:janetpygo]
use = egg:pygo

These configuration files are quite flexible, and allow different instances of a web application to run at different locations. In that case, one of them is running for ‘fred’ at /blog/fred. Each block for an app can have additional arguments that setup the database to use, and other settings that should probably be customized for each user. I threw in an additional database argument for one of them as an example.

Maybe you’re not a web administrator, but a web user comfortable downloading and installing many of the other webapps out there (Typo, MovableType, phpBB, etc.). Setting up your own site, using whatever Paste-enabled Python webapps you want is just as easy. Ideally ISP’s and such would just run the easyinstall command to install whatever the ‘popular’ webapps of the week are, and they can update them easily, keep old versions around if needed, etc.

Making Python Web-Application Distribution/Use Easy

This is pretty powerful stuff, and its quite easy to use. Even if you’re not making a web application with the intent to distribute it to the world, Paste is still going to help you out in a lot of ways.

An ISP or webapp end-user only needs to know how to add your webapp to their config file and installing it is a breeze thanks to setuptools. I think this has an enormous benefit for the Python web community, as it’ll significantly increase the ease of use when it comes to installing and managing web applications. Python web developers will gain some good footing to create Python Web Applications that compete with and surpass PHP webapps, especially if more ISP’s start supporting Paste-enabled applications.

Well, that’s my hope at least as I’m sure the title of this section made clear.

For the Python web framework creators out there, I like writing web applications (with various web frameworks), and I really like how easy Paste makes it to create them, package them up, and use them. Please add Paste support to your web framework, I’ll be happy to help and the Paste mail list is very responsive.

Comments
Sep 30 2005

Best of breed Controllers for MVC web frameworks

I’ve been doing a lot of comparisons and research into various Python web frameworks lately, mainly focused on MVC oriented frameworks. Some of them have templating languages they come with, some not so much. The thing I have started to notice is how similar the Controller in them all is starting to look. First though, lets see what they all can do when it comes to the Controller.

For comparison’s sake, I’m bringing out some examples using CherryPy, Myghty, Bricks, Aquarium, Ruby on Rails, and Django.

Please note I’m not including frameworks built using any of the ones mentioned as their Controller looks pretty much the same. I’m also not an expert in all these frameworks as I only have so much time in the day, so if I seem vague on something its because I don’t know for sure. Feel free to comment (or correct) and clear up any vagueness I express.

Dispatch and Responder Styles

All the frameworks I’ve mentioned implement a style of object publishing (sometimes referred to as dispatch-style) that maps to the URL coming in. If you have another name you want to call this, go for it, I haven’t seen anything consistent on what to call this stuff.

Three different styles seem to exist that make sure an object responds to the URL:

  1. User creates a regular expression to explicitly map a URL to an object (*Explicit Mapping*)
    • (r'^polls/(?P<poll_id>\d+)/vote/$', 'myproject.apps.polls.views.polls.vote') (from Django tutorial)
  2. Object is mapped out according to the URL (*Implicit Mapping*)
    • /admin/login/hello/ —> admin/login.py/hello() (Myghty)
  3. Programmatic rule set determines object that responds (*Programmatic Mapping*)
    • map.connect ‘:controller/:action/:id’ (Rails)
    • m.connect(’:controller/:action/:id’) (Routes)

In the Python world, three different styles also exist for how that object (or function) is defined. The first one is most popular, and is similar to how Rails sets up the Controller. They are:

  1. Create a class inheriting from a Controller object, methods of the class instance are called to respond (*Class-method*)
  2. Create a callable object that responds (*Callable*)
  3. Create a function thats directly called (*Function*)

Note that the last 2 are functionally equivalent for the most part, however a callable object has the typical advantages (or disadvantages) of persistence in some web frameworks, as such if a web framework only shows one of those styles I won’t mention the other. Curious what frameworks use what styles from the above two sets?

For this listing, I’ll refer to the mapping scenario as dispatch-style and the method for defining the response as the responder-style, also please note that some frameworks support multiple dispatch-styles and responder-styles.

Aquarium:

  • Dispatch-Style Mapping 1:
    • Explicit, Implicit, Programmatic
  • Responder-Styles:
    • Callable (Though it looks almost exactly like the Class-method style)

Bricks:

  • Dispatch-Style Mapping:
    • Implicit
  • Responder-Styles:
    • Class-method

CherryPy:

  • Dispatch-Style Mapping:
    • Explicit (Not with regexp’s though, each class is attached off the root to determine what URL leads to it)
  • Responder-Styles:
    • Class-method (Must be explicitly exposed with @cpg.expose)

Django:

  • Dispatch-Style Mapping:
    • Explicit
  • Responder-Styles:
    • Function (Packaged in the same module if they’re related)

Myghty:

  • Dispatch-Style Mapping 1:
    • Explicit, Implicit, (Programmatic with Routes in upcoming 0.99)
  • Responder-Styles:
    • Class-method, Callable, Function

Rails:

  • Dispatch-Style Mapping 1:
    • Programmatic
  • Responder-Styles:
    • Class-method

1: User can create custom dispatchers to determine responder object

Who did it first?

If we throw in two more frameworks that utilize CherryPy, like Subway, and TurboGear, we can start to see that using a responder-style of Class-method is pretty popular. What’s even more interesting is that all the Class-method Controllers look remarkably similar, and I don’t think its because they’re copying each other.

The Aquarium project appears to be approximately 4 years old now, but from looking at the mailing list had little or no users until only 2-3 years ago (One poster from 2001 describes his aquarium plants turning brown…). From the sourceforge changelogs, the Controller stuff was added with 1.4 release in 2004. CherryPy has been around for 3+ years and I’m presuming it had the Controller stuff in it from the get-go as it seems a pretty core function of CherryPy. Rails has been out for a year or so, and was used in a form internally for BaseCamp for awhile predating that.

If you really want to trace back this style of Controller, you’d most likely find yourself looking at Java Python code (See Phillip J. Eby’s comment below). I’m really just trying to point out here, that even if one of these projects saw something they liked from the other, none of them were the originators. They all took a good concept, and implemented it differently, though they’ve all started to look remarkably similar.

A Controller Line-Up

So lets take a look at some of the examples on the pages of each respective website. Django is the only one that really stands out, as they’re not using a Class to contain the methods, rather they’re grouping them as plain functions inside a module.

Aquarium

class BaseController(Controller): def call(self, callNext, *args, **kargs): “”“Save a reference to myself in ctx.controller.”“” self._ctx.controller = self callNext(*args, **kargs) def doMessageAddAction(self): “”“Add a new message. ...does a bunch of stuff…

I mentioned that it was Callable but functioned in a way like a Class-method. This appears to be because Aquarium puts a context and maybe other initialization stuff per request in the __call__ bit, then has it actually calls a different method to respond.

Bricks

from bricks.controller.crud import CRUD
from bricks import user

class Test(CRUD): # Override the default delete() method def delete(self, rowid): return CRUD.delete(self, rowid) delete.expose = user(level=1, app='app’)

controller = Test('todo’)

Bricks adds an interesting ability here by attaching function attributes to indicate what authorization is needed to execute the function called. Some of the other frameworks shown here do similar things through function decorators.

CherryPy

from cherrypy import cpg

class HelloWorld: def index(self): return “Hello world!” index.exposed = True

cpg.root = HelloWorld()

Notice that the class instance is attached to the location off cpg.root thats desired. To map a URL to a class instance farther down, say under /booga instead of /, you’d do that last line like so:
cpg.root.booga = HelloWorld()

Pretty handy I think, and it beats writing regular expressions (though doesn’t appear as flexible). One of the reasons I prefer the Programmatic dispatch-style. Moving on..

Django

from django.core.extensions import get_object_or_404, render_to_response
from django.models.polls import choices, polls
from django.utils.httpwrappers import HttpResponseRedirect

def vote(request, poll_id): p = get_object_or_404(polls, pk=poll_id) ... does a bunch more stuff to use Django-supplied functionality…

If you imagined the functions inside a class, instead of a module, it’d look rather similar to the others….

Myghty

class HelloWorld: def doit(self, m, **params): m.write(“hello world!”)

helloworld = HelloWorld()

At this point, the similarity between Bricks, CherryPy, and Myghty should be pretty obvious. Aquarium is close as well, while Django is only a wee farther from what’s appearing to be the norm.

Rails

class HelloController < ApplicationController
    def index
        return render :text => "hello world!"
    end
end

Doesn’t look too different from some of our Python MVC frameworks…

What it All Means

I have no idea. Really, I just thought it was fascinating that these different frameworks started looking so similar. Some of them have been incredibly similar before others, but obviously these approaches catch on and find themselves adopted into other frameworks.

I do start to wonder at what point they’ll be similar enough that the barrier to switching web frameworks is narrowed down to only a few dozen lines of code to change here and there. Sure, going to/from Rails is going to be tough as that also means a language change, but for Python programmers tempted by a feature a different web framework offers I won’t be surprised to see more people hopping back and forth.

With regard to the Python web frameworks, none of them appear to agree on a template language. Several of them use Cheetah, some use Myghty’s template abilities, one uses Kid, and several of them have their very own template language.

It’s also interesting to consider whether or not having more possible dispatch/responder styles is beneficial or detrimental to the popularity of a specific framework. Rails creator David Hansson purposely includes only one style for each as he emphasizes with a concept known as convention over configuration. He provides defaults for pretty much everything, and following conventions makes it easier to write code as it enforces a consistent programming and naming style. I can’t help but notice that of the frameworks I mentioned above, the ones with less (apparent) options and better defaults tend to have more users (With the exception of Bricks as its still in alpha).

You might have noticed I designated Rails as being capable of user-programmed dispatching. While Rails uses its routes system by default, the thing people miss is that they’re just defaults. You can get in there and totally alter the way it works if you want, Ruby doesn’t completely close classes so you could just re-open them and override any aspect of its dispatch system you like.

I think this a valuable lesson that some of the more “verbose” examples above might want to consider. If there’s lines of code in a controller that you have to put in there every-time, find a way to make it the default case so that boiler-plate isn’t needed. It only needs to be a default, an advanced programmer will know how to override it should they need to. There’s obvious limitations to this since things work differently in Python, but there’s still Python ways to do it in a way that feels “right”.

For the Python Web Frameworks I missed

I know, I didn’t cover SkunkWeb, Spyce, Snaklets, Webware and more. Please feel free to add the relevant information regarding your framework in the comments. It’d be helpful if you could indicate what Dispatch-Styles and -Responder-Styles your framework supports.

If I failed to properly cover one of the above frameworks, bringing that to my attention is most appreciated.

The comment system supports the Textile format.

Comments
Sep 26 2005

Routes 0.2 released

Routes 0.2 has been released. The change-log is pitifully short:

  • Added prefix option
  • Fixed Python 2.3 bug with thread-local singleton

But hey, its a small package to begin with so what the heck. Though its only 0.2 I’m rather pleased with it so far, its performance is great and quite reliable so I’m using it in production environments already.

If you haven’t been following my blog long, Routes is a feature-complete implementation of Rails routes system. I talk more about my reasons for re-implementing Routes in Python in an earlier post so I won’t repeat them all here.

It’s fairly unique in the Python world as it will do a route lookup search to turn a dictionary back into the URL (URL Generation) that will ensure the same values are created. This allows you to generate URL’s from inside your web pages and easily add new URL schemes without touching all your web pages.

The Routes package is aimed directly at integration with Python web frameworks that support the MVC style paradigm as it returns a controller and action value with the assumption your framework will know what to do with it.

Tired of writing big regexp’s to match URL’s to a class/method for dispatch in your webapp? Pester the framework creator to integrate Routes. :)

Here’s some Python Web Frameworks that currently or will shortly have Routes support/integration:

  • Aquarium has a URL Connector that uses Routes. Not sure if its been added to the core or only exists as an add-on right now.
  • Myghty will have Routes integration packaged with it in 0.99 which is on track to be released this week hopefully. It will also bring Python Paste support and integration which will bring a whole bunch of goodies to webapp developers and users. (Note that Python Paste requires Python 2.4)

I’ll be talking more about Python Paste and why you should care later this week. Any comments/suggestions on Routes are greatly appreciated.

Comments
Sep 09 2005

Separating Web Development Environments in Myghty

On many occasions, its quite useful when developing web applications to have the webapp know whether its in a production/development/testing environment. Rails builds this into the framework and its rather easy to add this toggle throughout a Myghty webapp (or some other Python web framework) as well.

We’ll make use of an environment variable so that the webapp environment can be easily configured from lighttpd or Apache. This way just deploying the application under a different handler can toggle the web applications mode of operation. The other thing we’ll setup while we’re at it, is a variable to keep track of the absolute root of our web application. I’ve found it quite useful in many cases to be able to get at this information without hardcoding it in, this way its always accurate no matter where the program is.

There’s a few rather commands that’ll give you the information we’re looking for. It took me awhile to find this, so hopefully it’ll help someone else out there.

Getting our Absolute Location

Let’s assume our directory hierarchy looks like this:

webapp/
   templates/
   components/
   scripts/
      dispatch.fcgi

Maybe you have the script called by mod_python or lighttpd somewhere else, I’m assuming it’ll be inside the root of your web application somewhere. In this case, the handler called by lighttpd is dispatch.fcgi. So dispatch.fcgi needs to figure out what the absolute path of the directory is above it.

Here’s the code that figures this out:


import os
  1. Set the prefix to our base path for the webapp

myloc = os.path.join(os.getcwd(), file)
prefix = os.path.normpath(myloc + ‘../..’)

The myloc assignment gets the absolute file-name with path of the current file, in this case dispatch.fcgi. Unlike a normal os.getcwd(), this call doesn’t care what directory we happen to be in when we import the module, it will always be the absolute file-name of that file.

In case you’re curious why this happens, __file__ will return the path of the file relative to the working environment its called from. So combining it with the full path name of the current working environment results in the complete absolute path of the module no matter what the current context or working directory.

The prefix assignment uses the normpath call to strip off the filename, and backup one directory to our webapp root. This leaves us with the absolute path to our webapp/ directory.

Setting and Using the Environment variable

Figuring out if we’re in a special runtime environment is quite easy and looks like this:


import os

MYGHTY_ENV = os.environ.get('MYGHTY_ENV’) or 'development’


Now we can just test MYGHTY_ENV to determine whether to contain errors ourself, or drop them to the web (as you would want in development mode). We default to being in development mode, since this is what you’d typically run your webapp in.

To switch it to production mode, here’s what the lighttpd config looks like:


fastcgi.server = ( 
   ".fcgi" => (
     "www" => (
        "min-procs" => 2,
        "max-procs" => 4,
        "socket" => "/tmp/webapp.socket",
        "bin-path" => "PATH/TO/webapp/scripts/dispatch.fcgi",
        "bin-environment" => ("MYGHTY_ENV" => "production" ),
        "idle-timeout" => 120
      )
    )
)

To set the extra environment variables with Apache, use mod_env with the SetEnv directive which would look something like this:


# Make sure you have mod_env loaded, this line assumed to be in the VirtualHost
# block of your config
SetEnv MYGHTY_ENV production

At this point, you might’ve noticed (if you’ve used Rails) how similar my Fast CGI setup with lighttpd looks when compared to the some of the Rails examples for a lighttpd + Fast CGI setup. This is intentional, as I’m adding a Routes dispatcher to Myghty so it makes sense to layout my web application in a similar directory hierarchy.

Anytime you need to toggle some behavior depending on your webapp’s runtime context, just import os and check it as I showed up above.

Tags: Python Myghty
Comments
Aug 05 2005

Making Decisions for Others

Please Note: Reading the last post would help greatly for this one, and this post does end up comparing aspects of Django with Rails in terms of making decisions rather than actual usage.

As I mentioned in my last post, Rails has been greatly helped by the decisions it makes for you. It decides the layout of your application directory, the directories you put stuff in, where you put what parts, etc. These decisions thankfully were done by someone familiar with good application design and had most likely even read Martin Fowler’s Refactoring to Patterns.

What results is a clean, well thought out application that you didn’t even need to think out very much. Before you know it, you fall into these useful and powerful design patterns without even thinking about it (I’d hope you would at least, they’re good patterns for a reason).

A lot of web frameworks make the assumption you are familiar with these patterns already, will setup your application to follow them, and use them in a way that makes sense. Part of the reason for this is because some web framework developers are rather appalled at the thought of making these decisions for the framework user. To be making those decisions for the framework user is looked at as holding their hand, and typically the programmers who write frameworks don’t need any hand holding and don’t expect their users to either (I’m assuming this based on the frameworks I’ve tried).

For example, during Ian Bicking’s test run with Rails he noted that having such a directory hierarchy setup for him put him off a little and mentions people arguing about file layout. I’ve seen very little, if anyone else, even comment on the directory structure that Rails lays out for you. The other irony of this particular post is that a lot of the stuff Ian finds “wrong” about Rails is actually Ruby’s “fault” (Implicit passing of names, mix-in’s, etc.). But moving on….

Ian clearly doesn’t want to lay down the law for users of his software, neither do other Python web framework developers. Having choice in how you lay out your application, with no clear direction what goes where, is considered a good thing. I don’t think there’s anything wrong with this approach, if the users you’re aiming your framework at are expected to be experienced, professional, programmers that are quite familiar with web applications and what works best for them. This does mean the audience will not be that large, and I think thats clearly reflected in the small communities most Python web frameworks have. While their communities are small, I’ve found them to be highly skilled and you get great answers to questions on their respective mailing lists.

The one Python web framework that has almost overnight gone from hardly any users, to at least a few dozen (probably more, but even a few dozen is quite good for a Python web framework), is Django. Django makes a lot of decisions for the user, more so even than Rails by a pretty good amount. While Rails was designed as a web application toolkit, and extracted form a large webapp, Django is designed more as a CMS web application toolkit as its been extracted from newspaper-style web sites. Thus, many of the decisions Django makes for you assume you will want users, user-access to the database objects, etc.

The question is, if someone else is making decisions for you, how important is it that they be good decisions?

One of the decisions the Django team made originally irritated me quite a bit. The Django framework is probably about 95% MVC in nature. It has database files that clearly are acting as Models (which it calls models), logic oriented files traditionally called Controllers (that it calls views), and template files that display the results to the user in a typical View role (though Django calls them templates).

The FAQ on the Django site notes that if you squint, sure enough the observations I just made do pop out, however because of the 5% non-MVC I mentioned they decided the names didn’t work. Rather than calling them by what they obviously were (or closest to), they swapped around names in a way that I think is bound to confuse Django programmers when they go into a real MVC environment and tell someone they edited the view when in fact it was the controller. If they had come up with completely different names that didn’t relate to the MVC paradigm at all, I think that would help to avoid later confusion on the part of a Django user.

Despite the naming of these parts, the decision made for the user is clearly for a MVC-style (or influenced) environment which I think is great ( please change the naming! ;) ). So what is the point of this whole thing?

I think its a good thing for developers of toolkits and especially frameworks to make some more decisions for the end-programmer of their work. This makes it easier for not so experienced programmers to get started quickly, and if you’re using a good design pattern even the experienced dev’s wont have anything to complain about. The two frameworks that currently are getting a lot of attention make a lot of decisions for the programmer using them, and this definitely increases the user-base (assuming you want a large user-base). Even though I know good practices on laying out stuff, it makes my life that much easier when its done for me.

Comments
Jul 05 2005

The Myghty Python Web Framework

Myghty is a powerful web application framework that builds on the strengths of Perl’s Mason using my favorite language, Python. While Myghty also handles page templating, it has some very powerful features that make it a full fledged framework in my opinion.

Wondering if Myghty is something you should check out?

I would highly recommend looking into Myghty if you’re looking for any of the following:

  • Clean, unobtrusive templating language for integrating Python data and HTML
  • An easy way to write pages that inherit their look/feel without needing .include statements
  • Re-usable code that you can carry from one project to the next
  • MVC programming paradigm , or page-driven paradigm (Myghty doesn’t care, it can be anywhere in between as well)
  • Advanced caching ability, tunable to just the time intensive sections of a page
  • Run as WSGI, FCGI, SCGI, mod_python, or stand-alone with hardly a single change to your code (to switch between them)
  • Architecture that scales, from a single developer’s personal project, to a multi-developer’s company website

History

Myghty first came into existence a little over a year ago when Mike Bayer looked around at the Python web application frameworks and couldn’t find one that really sucked him in. Having used Mason for various projects, but not finding anything as elegant as it for Python he decided to port Mason to Python.

This ported code was based on the Perl code almost line by line, resulting in a Python web application framework that worked almost exactly like Mason, except with Python as the base language. It wasn’t very long before Mike started adding to Myghty, such as Module Components, and threaded features not found in Mason.

Terms Used

MVC – Model, View, Controller paradigm is typically used to describe an approach that separates the code in these 3 steps rather than mingle them all in one place. Model usually refers to the database and its abstraction, the Controller handles the logic of dealing with the request, getting the data from the Model, and passing it to the View for display to the user.

Component – A component in Mason is what a template is known as. A template might display a full web-page, just one small section, or send data to other components. Components typically correspond to files on a filesystem, and appear as normal HTML pages with a few lines of Python in them, although they’re far from normal HTML pages…

Resolver – Used to describe the process of determining what to do with a given URL. Myghty has some incredible flexibility here.

Distinctive Features

Module Components

In Python, your modules contain all your Python code. In Myghty, the Component contains a mixture of Python with HTML. To handle a MVC approach to development, Myghty has Module Components. These are a cross of a Myghty Component with a Python Module, that leaves you with a fairly normal Python Module. It’s called directly from Myghty in a similar way to how Myghty would call a Component. These Module Components can be explicitly defined in the configuration, or called implicitly.

The Module Component allows you to directly handle URL’s, and respond from within the Module by sending data back to the client or processing data then sending it on to a normal Component for processing. Module Components work within the Resolving process, and have control passed to them according to a configuration mapping . The mapping uses regular expressions to match URL’s to modules that contain the method to be called, like so:


module_components = [
    {r'myapplication/home/.*' : 'myapp.home:HomeHandler'}, 
    {r'myapplication/login/.*' : 'myapp.login:LoginHandler'}, 
    {r'.*/cart' : 'myapp.cart:process_cart'}, 
    {r'.*' : 'myapp.home:HomeHandler'} 
]

This would map the URL http://yourapp/myapplication/home/blah to the HomeHandler component class, in the home module file, under the myapp package. The method in the HomeHandler class called can be done in two different ways, implicitly, or explicitly. For more details on this, read the Myghty docs on Module Components

While other frameworks implement an MVC approach, Myghty’s goes a step farther by letting you tie it into the resolution options used by the Resolver, and even use your own resolvers…

Advanced Resolver

In version 0.97alpha2 (maybe a better naming scheme could be used? :) the resolver implementation was heavily rewritten to allow massive flexibility in how a URL is handled. When Myghty handles a request, it goes through a fairly complex process to determine what to do with it. This is called the default strategy:

  1. Translate the URL against the path translation options – This can act as an internal URL rewriting scheme, changing a URL to be handled differently
  2. Resolve a dhandler – This rule matches conditionally. It is only called if the rules under it don’t find a match, it then strips off part of the URL its looking for and adds dhandler. So a search for /article/view/38 becomes a search for /article/view/dhandler if 38 isn’t found. Read more about dhandlers
  3. Check the URI Cache – The URI cache is searched for the incoming URI at this point, to see if a component has matched it before. This rule is also executed on a conditional basis.
  4. Upwards Search – The 3rd conditional rule of the bunch, this rule is called during inheritance when a component searches parent paths for an autohandler. Read more about autohandlers
  5. Resolve a Module – Matches a Module Component to the incoming URI.
  6. Resolve a File – Matches a file under the component roots specified for a match to the incoming URI. This will directly call a Myghty template file on the system in response to a URL.

As soon as any of the non-conditional rules matches, the file or module component method is served up. Myghty gets an additional speed boost by keeping compiled versions of all your template files around, so that they’re only compiled the first time they’re called. By using the basic Python way of compiling the templates as .pyc files, it also ensures that they’re re-compiled the second you update one of them. This way you’re always sure you’re viewing the latest update of your template.

So where’s the real power in all this?

These are just the default rules. They can be completely, and utterly customized. You can switch the order around, you can remove resolver rules entirely, and you can even write your very own resolver rules.

Maybe you’ve decided you want to serve some of the components from a database instead of the file system. You could write a custom resolver rule that does just that, and loads the entire component from the database as well. You could write your own version of the Path Translate module to get its rewrite rules from a database table. As if this wasn’t powerful enough, Myghty also lets you have different resolver strategies for different contexts.

Conclusion

Myghty is a very powerful web framework. It has features that go beyond any pure “templating” language I’ve encountered. If you’re not quite happy with what you’re using right now, or something I described above makes you giddy with thoughts of power, maybe its time to take Myghty for a test drive.

In future posts, I’ll describe more of the features I use daily in Myghty, and share code tidbits that let you get complex tasks done easily.

Comments
Page 1 of 1