Tuesday, May 13, 2008

Jython @ JavaOne 2008

I attended a couple days of JavaOne last week and luckily avoided the norovirus outbreak. However I did notice something else spreading through Moscone Center; interest in languages on the JVM. For example:
  • The number of sessions/BoFs covering Dynamic Language topics: around 8 on Groovy, 5 on JRuby, 2 on Jython and also a couple on Scala and Rhino each, with some very good turnouts. That's not even including CommunityOne. Also featured was a Scripting language bowl: a faceoff between JRuby, Groovy, Scala and Jython.
  • The JavaOne book store was stocked with just about every JRuby and Groovy book out there. Of the top 10 selling books at JavaOne, 3 were about languages on top of the JVM (Groovy and JavaFX). Wednesday's top 10 sellers also included JRuby committer Ola Bini's Practical JRuby on Rails as well as another Rails book. I'm hoping to see a whole lot of Python books on the shelf next year.
  • A number of Ruby/JRuby folks (and not just the ones employed by Sun either) told me that they are quite happy with NetBeans' support for Ruby. It has code completion and even some refactoring support. Tor Norbye and the NetBeans crew are now working on JavaScript support (Look Mom, JavaScript type inference) and are slated to add Python support next. Ted Leung has already begun talking to them about the details.
  • Ruby is still on the rise, and JRuby is a big contributing factor. JRuby definitely had the attention of many attendees and definitely has a growing userbase in Java land.
  • Groovy and the Groovy on Grails combo is also on the rise. linkedin.com, Sky TV and SAP's Composition on Grails Product are a few notable users of Grails. IBM's Project Zero (aka the WebSphere sMash product) is also utilizing Groovy, as well as their own PHP on the JVM implementation.
  • Sun's own new scripting lanuguage on top of the JVM, JavaFX, was of course all over the place.
  • John Rose: "I think we are on the right track here, letting the JVM grow independently of the Java language. (The language, if it has room to grow, will catch up.) James Gosling expressed a similar sentiment at his February Java Users Group talk “The Feel of Java, Revisited”, when he said he sometimes felt more interested in the future of the JVM than that of the Java language. “I don’t really care about the Java language. All the magic is in the JVM specification.” (Yes, I think that is hyperbolic. No, he is not abandoning Java.) I love both Java and the JVM, and I am pushing on the JVM this year."
  • The potential Java 7 new features. They're still potential because the Java 7 JSR isn't out yet. While some are nice (like John Rose's JSR 292, which will be a big help for JVM languages like Jython), some are arguably not nice. I hear more disdain than ever over where Java the language is going, which makes other languages on top of the JVM even more desirable.
  • I met many new people (more than I expected) who have used Jython at some point in their career, and some who are using it now (like pushToTest's Frank Cohen). They're all eager about its recent progress and can't wait for the 2.5 release.
  • Reminder: the Java world is large. Over 10,000 attendees, probably 10 times as many as this year's PyCon. All potential Python converts, right?
Ok, I mostly covered what I observed about different JVM languages at JavaOne, not just Jython. What I'm getting at here is that there's a lot of potential for the Jython 2.5 release (and all that code waiting around on Pypi to run on it) in this large Java ecosystem.

Sunday, May 4, 2008

You Must Construct Additional Pylons (on other VMs)

Occasionally on the Pylons IRC Channel or mailing list someone will reference the game Starcraft, where "Pylons" are structures that act as a power source. Sometimes the game vocally instructs you to construct additional Pylons.

You could say that last month we've constructed a couple additional Pylons, and by that I mean made Pylons run on different environments:

o Pylons on Jython trunk!
o Pylons on Google App Engine via Ian Bicking's appengine-monkey and mako trunk

With the development versions of both Pylons and its dependencies (and the Mako jython branch) I'm now able to create the flicksearch tutorial app from the Pylons Official Docs on Jython.

The Mako jython branch uses Python 2.5's _ast module, which Jython now supports thanks to Frank Wierzbicki's recent work. Mako's also using Armin Ronacher's handy ast module that'll hopefully find its way into the stdlib. Frank and I are still working on smoothing out a couple of our _ast's rough edges, hence the separate Mako jython branch, but Mako is in good shape with 95% of its 214 tests passing (with most of those failures due to the lack of PEP 263 source code encodings, which we'll get to supporting on Jython soon).

According to the Pylons on Jython buildbot, only a few of the Pylons dependency's tests fully pass -- but most of the failing tests are due to unicode issues (again, lack of PEP 263), CPython dict ordering assumed in doctests, and a few other issues that aren't important enough to cover here. Pylons own test failures are only due to unicode issues and lack of support for the Kid and Cheetah templating engines, which Pylons is only testing for the sake of testing alternative templating engines anyway.

The biggest problem with Pylons on Jython so far is running it in development mode with paster serve --reload, which hot reloads your WSGI app whenever a change to one of its files is made. The paster reloader is working but the reloading process is very slow on Jython. Reloading is done by spawning 2 processes; when a file changes, the child process serving your app exits and the parent creates it anew (last time I looked Django and CherryPy's reloaders also work this way). Since CPython startup time, including loading of the entire WSGI app, is pretty quick, this is the safest and probably easiest way to accomplish a reload.

Unfortunately Jython suffers from a lenghty startup time. It takes about 1.5 seconds best case scenario just to get the >>> prompt in Jython on my 2.33ghz MacBook Pro. It takes almost 10 seconds to get paster serve --reload to fully load a simple Pylons app; that's the parent process startup time plus the child process startup time plus the app load time.

I've been experimenting with using Nailgun to improve startup time, but it actually isn't helping Pylons' startup time as much as I thought it might. Jython must have a startup time bottleneck or two that we need to find here.

Nailgun also poses other problems, such as when System.exit()'ing resources like sockets and files are left open unless they're explicitly cleaned up. Worse yet, threads are left running -- and Pylons' paster serve by default loads 10 listener threads.

If we can't drastically improve the startup time issue in Jython sooner rather than later, we can probably come up with a reloader for Jython for the time being that doesn't require a full restart. These can be tricky and in general are avoided, though.

Here's a couple screenshots of Pylons on Jython, showing off a handy feature of dynamic languages on the JVM; just having the ability to play around with Java in an interactive interpreter:

o Pylons-dev WebError on Jython
o Pylons paster shell on Jython

Pylons is also making progress towards an additional release, 0.9.7, which will utilize the work done by the PyCon sprinters (particularly on WebHelpers), WebOb, WebError (and maybe WebTest) and will also include optional support for a basic SQLAlchemy configuration for new projects. As well as Alpha Jython support.