You Must Construct Additional Pylons (on other VMs)

Occasionally on the Pylons IRC Channel or mailing list someone will reference the game Starcraft, where "Pylons" are structures that act as a power source. Sometimes the game vocally instructs you to construct additional Pylons.

You could say that last month we've constructed a couple additional Pylons, and by that I mean made Pylons run on different environments:

o Pylons on Jython trunk!
o Pylons on Google App Engine via Ian Bicking's appengine-monkey and mako trunk

With the development versions of both Pylons and its dependencies (and the Mako jython branch) I'm now able to create the flicksearch tutorial app from the Pylons Official Docs on Jython.

The Mako jython branch uses Python 2.5's _ast module, which Jython now supports thanks to Frank Wierzbicki's recent work. Mako's also using Armin Ronacher's handy ast module that'll hopefully find its way into the stdlib. Frank and I are still working on smoothing out a couple of our _ast's rough edges, hence the separate Mako jython branch, but Mako is in good shape with 95% of its 214 tests passing (with most of those failures due to the lack of PEP 263 source code encodings, which we'll get to supporting on Jython soon).

According to the Pylons on Jython buildbot, only a few of the Pylons dependency's tests fully pass -- but most of the failing tests are due to unicode issues (again, lack of PEP 263), CPython dict ordering assumed in doctests, and a few other issues that aren't important enough to cover here. Pylons own test failures are only due to unicode issues and lack of support for the Kid and Cheetah templating engines, which Pylons is only testing for the sake of testing alternative templating engines anyway.

The biggest problem with Pylons on Jython so far is running it in development mode with paster serve --reload, which hot reloads your WSGI app whenever a change to one of its files is made. The paster reloader is working but the reloading process is very slow on Jython. Reloading is done by spawning 2 processes; when a file changes, the child process serving your app exits and the parent creates it anew (last time I looked Django and CherryPy's reloaders also work this way). Since CPython startup time, including loading of the entire WSGI app, is pretty quick, this is the safest and probably easiest way to accomplish a reload.

Unfortunately Jython suffers from a lenghty startup time. It takes about 1.5 seconds best case scenario just to get the >>> prompt in Jython on my 2.33ghz MacBook Pro. It takes almost 10 seconds to get paster serve --reload to fully load a simple Pylons app; that's the parent process startup time plus the child process startup time plus the app load time.

I've been experimenting with using Nailgun to improve startup time, but it actually isn't helping Pylons' startup time as much as I thought it might. Jython must have a startup time bottleneck or two that we need to find here.

Nailgun also poses other problems, such as when System.exit()'ing resources like sockets and files are left open unless they're explicitly cleaned up. Worse yet, threads are left running -- and Pylons' paster serve by default loads 10 listener threads.

If we can't drastically improve the startup time issue in Jython sooner rather than later, we can probably come up with a reloader for Jython for the time being that doesn't require a full restart. These can be tricky and in general are avoided, though.

Here's a couple screenshots of Pylons on Jython, showing off a handy feature of dynamic languages on the JVM; just having the ability to play around with Java in an interactive interpreter:

o Pylons-dev WebError on Jython
o Pylons paster shell on Jython

Pylons is also making progress towards an additional release, 0.9.7, which will utilize the work done by the PyCon sprinters (particularly on WebHelpers), WebOb, WebError (and maybe WebTest) and will also include optional support for a basic SQLAlchemy configuration for new projects. As well as Alpha Jython support.