Ruby on Grails? Why the hell not?

I've spent some time this weekend looking over the Grails codebase. After a truly gigantic checkout (Grails bundles production JARs for all the libraries it integrates) and after swimming through a substantial, single-project sea of source files, I think I'm starting to follow it (though a tour by someone familiar with Grails internals would be nice). But there's something else I've learned in the process...

There's really no reason this couldn't use Ruby just as easily as Groovy.

Let's look at some really dumb numbers. Grails, as it turns out, is actually over 2/3 Java code, unlike Rails which is 100% Ruby. Grails contains around 436 Java source files for about 54kloc of code and 273 Groovy source files for about 23kloc of code. So Java outnumbers Groovy by a 2:1 ratio. And yes, Java's probably more verbose than Groovy in most cases. But my point stands, and I'll back it up with more...

Then there's the libraries. The Grails binary release weighs in at a whopping 26MB compressed and 49MB uncompressed (*wow*). But 31MB of that is third-party libraries completely independent of Groovy, things like Hibernate, XFire, Quartz, Apache Commons, and so on. Almost all of which (or perhaps all of which) are pure Java code. So the ratio is even more heavily toward Java.

These numbers and my exploration of the code base lead me to a surprising conclusion: Grails is not a "Groovy on Rails" by any means. It's a large number of popular, well-established Java libraries stitched together mostly by Java code with a thin layer at the front (20kloc is not much) of Groovy code for end-users to see and work with. And I guarantee you it's not the Groovy part that's most interesting here. Hell, you could write that part in Java if you followed enough "convention over configuration", and you certainly could write it in Ruby.

I decided to do this exploration after realizing there's nothing about Ruby or JRuby that couldn't be used to wire a bunch of Java libraries together--indeed, that's one of the biggest selling points of JRuby, the fact that you can write plain old Ruby code but call Java libraries almost seamlessly (with day-by-day improving integration points and performance numbers). So why not take Grails and do a port to Ruby? Groovy code isn't far off from Ruby in many ways, and everything you can do in Groovy you can do in Ruby (plus more), so the port should be pretty straightforward, right?

Then I find out that Grails is mostly Java code. And instead of being annoyed or repulsed by this revelation, I was intrigued. Why not make Grails work with Ruby as well? And really, why not make it work with any scripting language? Why limit that nicely-integrated back end by forcing everyone to use Groovy, regardless of preference? Have we learned nothing from a decade of "100% Pure Java"?

So I propose the following.

• I believe it would be possible to completely isolate Grails' tastier bits from Groovy. I don't claim it would be easy, but Grails is fairly well interfacified, injectificated, and abstractilicious, so making that dividing line clear and bold is certainly possible (and heck, that's the point of dependency injection and component frameworks anyway, isn't it?). Really, I could start implementing a few of those interfaces right now with Ruby code, but that's not the right approach.
• Grails' value is not in Groovy, but in the clean integration of established libraries behind the "Convention Over Configuration" mantra. There's nothing specifically Groovy (or Ruby) about that religion, and there's nothing Groovy does in Grails that another JVM language couldn't do just as easily. This is *truth*. And Grails would benefit tremendously by expanding to other languages. Being tied to Groovy alone will hold it back (just as a framework tied to any one language will be held back, Ruby included).
• I also believe that if the Grails project doesn't make an official effort in this direction, others are likely to take up that mantle themselves. Though the debates will forever rage about which JVM dynlang should carry us into the future, the same facts that have guided programming for half a century still apply today: No one language will be enough to carry us forward; no languages will survive indefinitely; and the language you'll use in ten years you've probably not even heard of yet. Did anyone expect Java to be the most widely-used, widely-deployed language ten years ago? Well today, it is. It won't always be.
• There's also a very interesting angle here for language enthusiasts. Nobody doubts that the Java platform is officially becoming multi-lingual, possibly a complete reversal from years past. But there are going to be growing pains here, and we're already feeling them on some projects at Sun. We have this wonderful platform with all these wonderful languages...and they all do things differently. We have a first stab at official "Scripting" support in JSR-223, but it only defines the integration point from Java to the language in question...only partially defining the integration points for those languages calling back to Java (a tricky problem) and defining *nothing* related to making those languages interoperate with each other. Again, Java becomes the integration point, and the bottleneck (as in "how do you pass a dynamic-typed object from one language to another by sending it through a statically typed language?"). The very exercise of making Grails (or any other framework) work equally well across multiple user-facing language interfaces will require better interoperability and more clearly-defined integration points for dynamic and static languages on the JVM.
• The Phobos guys at Sun are already tackling many of these same problems while expanding their JavaScript-mostly Rails-inspired web framework to other JSR-223 languages (notably, Ruby). So anyone taking on the Grails effort would certainly not be alone, and there are some damn smart engineers at Sun interested in tackling these very same problems.

And I guess in the end it's a question of how visionary we all can be. Can the Grails team envision a future for Grails that doesn't require the use of Groovy? Are you (and am I) as a non-Groovy developer willing to help them get there? And in the end, isn't this what thought leaders on a multi-lingual platform are supposed to be doing?

--

So, enough of the inflammatory bull-baiting. I dare you to tell me why I'm wrong here. Even better, I dare you to look into it yourself. 75kloc of code, only a third of it Groovy. Established libraries you're sure to have used at some point. Clean integration, following the principals inspired and proven by Rails.

But fronted with your language of choice.

Make it so.

Behind The Scenes: JRuby 0.9.8 Released!

Things have been moving quickly in JRuby-land, and we've just kicked out a major release in JRuby 0.9.8. Among the big features:

• Ruby classes can extend concrete/abstract Java classes and override methods
• New Java primitive array syntax
• Reimplementation of String, Numeric classes, and Array to be more correct and performant
• Significant bottlenecks have been identified. In some cases IO is 6.5x faster than previous releases. Java included classes are significantly faster than in the past.
• 220 Jira issues resolved since last release

Now these are certainly delicious. Performance improvements are always great, and we're been improving Java integration little by little. But there's another bullet I omitted that is worth a little more attention:

• Ruby on Rails support

Now you may feel free to read that as you wish. I read it as "we expect pure-Ruby Rails apps to work." By "pure Ruby", I mean no unsupported C-language extensions required. By "work" I mean "work."

It was our original plan over two months ago that by Februrary's end we'd be passing enough of Rails' own test cases to call it "supported". And in a relative rarity for an open source project, we hit that milestone. It was actually early last week we started saying we officially support Rails, which at the time meant better than 95% of Rails test cases passing. By the time we cleaned up the remaining release items, we had passed 98%. And we're still climbing.

So what does this mean for a practical Rails user? Well, it means that the dream of deploying Rails apps on any Java-based server in any Java-based organization is another big step closer to coming true. I've actually been demoing Rails apps running in a WAR file on GlassFish for all my recent talks, and it's pretty solid. I don't imagine it's perfect, of course, but it's looking really very good. It will be "ready" by the time we hit 1.0.

A few folks have been wondering about the jump from JRuby 0.9.2 to JRuby 0.9.8. The simple answer is this: we're two releases away from 1.0. We wanted to make it obvious how much work has gone into 0.9.8 (as many SVN revisions in this release as the last three combined) and show our commitment to getting a 1.0 release out very soon. The 0.9.8 release will be followed by 0.9.9, which will be followed by...well, you get the picture. We're in the home stretch now.

The other core team members have reported on the release, so I won't cover all that. Here's a list of links:

Tom Enebo
Ola Bini
Nick Sieger

In related news, there's also a new release of ActiveRecord-JDBC, the gem-installable module enabling JDBC databases to be used by ActiveRecord. This is also very welcome, since there have been a number of bug fixes in trunk that weren't easily installable the past couple months. Now you should be able to run both Rails 1.2.2 and 1.1.6 well with JRuby, just using released code. It helps complete the puzzle.

So what's next for JRuby? Well, with the 0.9.8 release we've reached a really big milestone as far as Ruby compatibility, so we're likely to turn our attentions toward areas not directly related to Ruby 1.8. Specifically:

• Work will continue on the bytecode compiler, and I'm hoping to have the JIT permanently enabled within the next week
• Java integration will get a heavier focus, to start bringing us more in line with languages like Groovy for ease-of-integration. In general things are looking pretty solid right now, but we know there's a lot of work to do here. I'd like for us to come as close as possible to Groovy's level of tight integration as possible.
• We will also start working on native Unicode support, by exposing a Chars class like the one provided in Rails' MultiByte library. This will allow us to maintain compatibility with Ruby and provide solid Unicode support without introducing an incompatible set of features.
• Both Tom and Ola have expressed an interest in continuing work on our Ruby 2.0 bytecode engine. Ola would like to continue improving and expanding the bytecode support, while Tom would like to play with a second parser that goes straight to bytecode. Both of those will help feed into a potentially faster interpreted mode in the future, giving us many of the perf gains shown in Ruby 1.9 benchmarks.

For me personally, performance is going to be the big item. The recent Ruby Shootout showed that we have some work to do to bring out performance up to MRI standards, but it didn't tell the whole story. The truth of the matter is more like this:

• JRuby interpreted mode is roughly 3-4x as slow as Ruby 1.8
• JRuby compiled mode is roughly 1.5-2x faster than Ruby 1.8
• KRI (Ruby 1.9, "Koichi's Ruby Implementation") is many times faster than MRI for the posted benchmarks
• KRI is 2x or less faster than MRI for the general case

And this is without us doing any optimization of the compiler or adding in any special-case optimizations like KRI provides for fixnums. We're still looking great for future performance, and I want to start tackling this area heavily over the next couple months.

As always, we welcome contributors, bug reports, patches, and user anecdotes for JRuby. We love to hear about bugs, we want to know your opinion, and we respect and honor our community whenever we get the chance. Remember, JRuby is your project, and can only be successful with your help.

Visit www.jruby.org to learn more about the project, download the release, and get involved!

On to the next release!

Another Step Toward Rails WAR Files

Ashish Sahni has posted instructions for packaging a Rails app as a WAR on his blog, based on the work of a number of JRuby community members. I ran through his instructions, and have only two modifications to make:

1. When installing Rails, you still probably want to pass --no-ri --no-rdoc since rdoc generation is still far too slow.
2. Because JRuby has a classloading bug (not loading from context classloader) you'll need to put the rails-integration JAR in glassfish/lib.

I was only able to get the "properties" page to display, at rails/info/properties. A separate page that used ActiveRecord didn't successfully execute...but it was close enough to taste.

So then, there's two important points you should get out of this:

Rails in a WAR file will happen...soon

This has always been our goal, with every improvement and fix we made to JRuby. The holy grail of Rails deployment would be to "zip it up and deploy it" like Java app developers can do with WAR files. And that goal is just around the corner, thanks to the JRuby community. That leads me to the second point:

The JRuby community is growing fast and doing great things

This is especially exciting to me. Tom and I are only two folks, even with full-time license to work on JRuby and related projects. The only way Ruby can be successful on the JVM is with community cooperation, and the work toward WAR deployment exemplifies what can happen.

I'd like to thank the folks on the JRuby-Extras project who have been working on this: Robert Egglestone, Chris Nelson, Fausto Lelli, and also Ashish Sahni at Sun who put together a great walkthrough.

Excellent work.

Advanced Rails Deployment with JRuby

There's a lot of work going on right now focusing on various mechanisms for deploying JRuby-based apps. This article will summarize some of the work happening and why it's really, really important for the Ruby world.

Ruby-in-a-JAR

First, a little sideline into general Ruby embeddability work.

Over the past few days I've made modifications to enable running a Ruby app completely out of a single JRuby JAR file (Java ARchive). The major changes required were:

1. Add jarjar to the project to combine all dependency jars into a single archive, including jline (readline support), asm (compiler, other stuff in the future), bsf (scripting API), jvyaml and plaincharset (Ola's JvYAML library and supporting charset lib).
2. Add an Ant task to build the "complete" jar and include all Ruby standard libraries in the same archive.
3. Somewhat unrelated, a small patch for irb/init.rb to allow it to fail gracefully if it can't load locale-specific files from the filesystem.

So by running "ant jar-complete" you get out a single JAR file that contains a complete, working Ruby interpreter plus stdlib. For example:

~ $ java -jar jruby-complete.jar -e "puts 'Hello, Ruby-in-a-JAR!'"
Hello, Ruby-in-a-JAR!
~ $ java -jar jruby-complete.jar -rirb -e "IRB.start"
irb(main):001:0>

Of course, you don't have to take my word for it. I've uploaded a copy of this jar for you to try yourself. The full archive is about 2900kb. That's suitable for embedding in just about any application, and in fact I used a stripped-down 1600kb version for the JRuby Applet. Note: this is JRuby trunk code and mostly experimental...but that's what makes it so fun :)

Ruby-in-a-JAR - The ultimate in Ruby interpreter portability

A Better Deployment for Rails

Now the main course: several folks have been working on several exciting deployment scenarios for JRuby on Rails apps.

The current "best option" for deploying Rails apps into production generally involves the HTTP front-end Mongrel. Although Mongrel is largely written in Ruby, it is very fast, largely because of its native C component for HTTP request parsing. It's also considerably more secure than CGI-based options, largely because of creator Zed Shaw's attention to detail. The typical Rails app will be deployed as a "pack of Mongrels", where the number of desired concurrent requests is multiplied by the number of independent Rails apps to determine a total number of processes. These processes must be managed, monitored, and respawned as appropriate, but the result is a fairly stable and scalable deployment model.

However with JRuby, there will soon be a better option. I previously reported about TAKAI Naoto's efforts to deploy Rails behind an AsyncWeb front-end, showing tremendous performance improvements over a WEBrick-based deployment. Naoto-san has now taken things to the next level: Rails deployment under GlassFish.

The potential here should be obvious. GlassFish, like other Java EE application servers, is extremely good at scaling up many concurrent requests across many independent applications; so good that many organizations deploy only a single appserver-per-machine and stuff it full of applications to serve. That means a single server, a single process to manage. GlassFish also supports clustering, which means you'll be able to hit the deploy button once and have your n-server cluster instantly start serving up Rails. But there's one last area that trumps all the rest:

That single app server can handle as many concurrent requests across as many independent Rails apps as you desire, scaling them across all the CPU cores you can throw at it.

That's right...no more N * M process management, no more zombie processes, no more immature tooling to manage all those servers and all those deployments. One tool, one server process, no headaches.

That's an extremely bright future, and we're almost there.

What Next?

Naoto-san is not the only one working on JRuby on Rails deployment options. There are a number of folks in the JRuby community approaching the same goal from different directions, using innovative techniques like servlets implemented in Ruby and Spring-based service wiring. The JRuby community sees the potential here and things are moving very quickly.

My work on Ruby-in-a-JAR will also play directly into this. Currently most deployment scenarios require Rails app files to remain "loose" on the filesystem, as with the current standard deployment model. However it won't be long before you can zip up your Rails app into a WAR file (Web ARchive) and deploy it lock, stock, and barrel to as many servers as you want.

These efforts combined will create, in my opinion, the most manageable, scalable, powerful Rails deployment model yet available...and it's just around the corner.

We've also launched into Rails compatibility work in earnest. I've created a wiki page on JRuby support for Rails that details the results of running Rails' own test cases. Long story short: we're looking pretty damn good.

JRuby on Rails is in the home stretch. And we're covering ground very quickly.