Advanced Caching with Django and Beaker
After seeing more than a few blog posts and packages attempt to provide more advanced caching capability for Django, it occurs to me I should actually just blog how to use Beaker in Django, rather than just keep mumbling about how “Beaker already does that”. So, if you’ve needed caching in Django that goes beyond using just one backend at a time, or maybe can actually cope with the Dog-Pile Effect, this is the blog entry for you (Until I flesh it out further into actual docs on the Beaker site).
Install Beaker
This is simple enough, if you have easy_install available, just:
easy_install -U Beaker
Or if you prefer to download tar files, grab the Beaker 1.4 tar.gz file
Configuring the Cache
Setting up Beaker’s cache for your Django app is pretty easy. Since only a single cache instance is needed for an app, we’ll set it up as a module global.
Create a beakercache.py file in your Django project with the following contents:
from beaker.cache import CacheManager
from beaker.util import parse_cache_config_options
cache_opts = {
'cache.type’: 'file’,
'cache.data_dir’: 'cache/data’,
'cache.lock_dir’: 'cache/lock’
}
cache = CacheManager(**parse_cache_config_options(cache_opts))
There’s a lot more options available, such as memcached, configuring multiple cache backends at once, etc. Now that you know how to provide the configuration options, further customization can be done as needed using the Beaker configuration docs. (Note the very handy cache region configurations which make it easy to toggle cache backend configurations on the fly!)
Using the Cache
Beaker provides a conveinent decorator API to make it easy to cache the results of functions. In this example we’ll just sleep and make a string including the time, add this to your views.py:
import time
from datetime import datetime
from django.http import HttpResponse
from YOURPROJECT.beakercache import cache
def hello(request):
@cache.cache(expire=10)
def fetch_data():
time.sleep(4)
return 'Hello world, its %s’ % datetime.now()
results = fetch_data()
return HttpResponse(results)
In this case, the cached data is in a nested function with the decorator. It could of course be in the module elsewhere as well.
Hook the view function up in your urls.py, and hit the view. The first time it will wait a few seconds, then it will return the old time until the cache expires (10 seconds in this case).
The cached function can also accept positional (non-keyword) arguments, which will be used to key the cache. That is, different argument values will result in different copies of the cache that require those arguments to match.
That’s it, it’s really quite easy to use.
Update: It occurs to me this post does say ‘advanced’, and that example wasn’t very advanced, so here’s something a bit more interesting. Let’s configure cache regions to make it easy to toggle how long and where something is cached. Cache regions allow you to arbitrarily configure batches of settings, a ‘region’. Later you can then indicate you want to use that region, and it uses the settings you configured. This also make its easy to setup cache policies and change them in a single location.
In this case, we’ll have ‘long_term’ and ‘short_term’ cache settings, though you can of course come up with as many regions as desired, with the name of your choice. We’ll have the long_term settings use the filesystem, since we want to retain the results for quite awhile and not have them pushed out like memcached does. The short_term settings will use memcached, and be cached for only 2 minutes, long enough to help out on those random slashdog/digg hits.
In the beakercache.py file:
from beaker.cache import CacheManager
from beaker.util import parse_cache_config_options
cache_opts = {
'cache.type’: 'file’,
'cache.data_dir’: 'cache/data’,
'cache.lock_dir’: 'cache/lock’,
'cache.regions’: 'short_term, long_term’,
'cache.short_term.type’: 'ext:memcached’,
'cache.short_term.url’: ’127.0.0.1.11211’,
'cache.short_term.expire’: '1200’,
'cache.long_term.type’: 'file’,
'cache.long_term.expire’: '86400’,
}
cache = CacheManager(**parse_cache_config_options(cache_opts))
Now in our views.py:
import time
from datetime import datetime
from django.http import HttpResponse
from testdjango.beakercache import cache
def hello(request):
@cache.region('long_term’)
def fetch_data():
time.sleep(15)
return 'Hello world, its %s’ % datetime.now()
results = fetch_data()
return HttpResponse(results)
def goodbye(request):
@cache.region('short_term’')
def fetch_data():
time.sleep(4)
return 'Bye world, its %s’ % datetime.now()
results = fetch_data()
return HttpResponse(results)
