Sunday, August 31, 2008

A Duby Update

I haven't forgotten about my promise to post on FFI and MVM APIs, but I've been taking occasional breaks from JRuby (heaven forbid!) to get some time on on Duby.

What Is Duby?

Duby, for those who have not heard of it, is my little toy language. It's basically a Ruby-like static-typed language with local type inference and not a whole lot of bells and whistles. The goal for Duby (which is most definitely a working will probably change), is to provide the all the best parts of Ruby syntax people are familiar with, but add to it:
  • Written all in Ruby (and obviously the eventual plan would be to port it to Duby)
  • Backend-agnostic (JVM is obviously my focus, but nothing stops someone from building an LLVM or CLR typer+compiler)
  • Minimally-intrusive static typing (Duby infers types from arguments and calls, like Scala)
  • Features missing from Java (Duby treats module inclusion and class reopening like defining extension methods in C#)
  • A very pluggable type inference engine (Duby's "Java" typer is currently all of about 20 lines of code that plugs into the engine)
  • A pluggable compiler (Duby will allow adding compiler plugins to turn str1 + str2 into concatenation or StringBuffer calls, for example)
  • Absolutely no runtime dependencies (I want compiled output from Duby to be *done*, so there's no runtime library to lug along so it works; once compiled, there are no dependencies on Duby)
The primary motivation for Duby was originally to have a Ruby-like language we could use to implement parts of JRuby. The JVM, it's type system, and its bytecode are all actually really, really nice. There's a huge collection of libraries, fast primitives (that get optimized into faster native code), and a bytecode specification that's pretty easy for almost anyone to grok. But there's a problem: Java.

Now don't get me wrong, Java is a great language, but it's become a victim of its own success. While other languages have been adding niceities like local type inference, structural typing, closures and extension methods, Java's stayed pretty much the same. There have been no major language changes to Java since Java 5's additions of generics, enums, annotations, varargs, and a few other miscellaneous odds and ends. Meanwhile, I live in a torturous world between Ruby and Java, where I'd love to write everything in Ruby (too slow, too inexact for "stable layer" code), but must instead write everything in Java (with associated syntactic baggage and 20th-century language design). And so necessity dictates taking a new approach.
def fib(a => :fixnum)
if a < 2
fib(a - 1) + fib(a - 2)

puts fib(45)
So here we see an example I've shown in previous posts, but with a twist. First off, it's almost exactly the same as the equivalent Ruby code except for the argument type declaration => :fixnum. The rest of the script is all vanilla Ruby, even down to the puts call at the bottom.

But all is not as it seems. This is not Ruby code.

The type declaration in the method def looks natural, but it's not actually parseable Ruby. I had Tom Enebo hack a change in to JRuby's parser (off by default) to allow that syntax. Duby originally had a syntax something like this, so it could be parsed by any Ruby impl:

def fib(a)
{a => :fixnum}
But it's obviously a lot uglier.

New Type Inference Engine

Ignoring Java for a moment we can focus on the type inference happening here. Originally Duby only worked with explicit Java types, which obviously meant it would only ever be useful as a JVM language. The use of those types was also rather ugly, especially in cases where you just want something "Fixnum-like". So even though I had a working Duby compiler several months ago, I took a step back to rewrite it. The rewrite involved two major changes:
  1. Rather than build Duby directly on top of JRuby's AST I introduced a transformation phase, where the Ruby AST goes in and a Duby AST comes out. This allowed me to build up a structure that more accurately represented Duby, and also has the added bonus that transformers could be built from any Ruby parse output (like that of ruby_parser).
  2. Instead of being inextricably tied to the JVM's types and type system, I rewrote the inference engine to be type-system independent. Basically it uses all symbolic and string-based type identifiers, and allows wiring in any number of typing plugins, passing unresolved nodes to them in turn. Two great example plugins exist now: a Math plugin that knows how to handle mathematical and boolean operators against numeric types like :fixnum (it knows :fixnum < :fixnum produces a :boolean, for example), and a Java plugin that knows how to reach out into Java's classes and methods to infer return types for calls out of Duby-space.
The result of this is that up to the point of compilation, there's no explicit dependency on any named set of types, any type system, or any backend. Here's the output from the type inference engine running against that fib script above:
* [Simple] Learned local type under MethodDefinition(fib) : a = Type(fixnum)
* [Simple] Retrieved local type in MethodDefinition(fib) : a = Type(fixnum)
* [AST] [Fixnum] resolved!
* [Simple] Method type for "<" Type(fixnum) on Type(fixnum) not found.
* [Simple] Invoking plugin: #<Duby::Typer::MathTyper:0xcc5002>
* [Math] Method type for "<" Type(fixnum) on Type(fixnum) = Type(boolean)
* [AST] [Call] resolved!
* [AST] [Condition] resolved!
* [Simple] Retrieved local type in MethodDefinition(fib) : a = Type(fixnum)
* [Simple] Retrieved local type in MethodDefinition(fib) : a = Type(fixnum)
* [AST] [Fixnum] resolved!
* [Simple] Method type for "-" Type(fixnum) on Type(fixnum) not found.
* [Simple] Invoking plugin: #<Duby::Typer::MathTyper:0xcc5002>
* [Math] Method type for "-" Type(fixnum) on Type(fixnum) = Type(fixnum)
* [AST] [Call] resolved!
* [Simple] Method type for "fib" Type(fixnum) on Type(script) not found.
* [Simple] Invoking plugin: #<Duby::Typer::MathTyper:0xcc5002>
* [Math] Method type for "fib" Type(fixnum) on Type(script) not found
* [Simple] Invoking plugin: #<Duby::Typer::JavaTyper:0x1635aad>
* [Java] Failed to infer Java types for method "fib" Type(fixnum) on Type(script)
* [Simple] Deferring inference for FunctionalCall(fib)
* [Simple] Retrieved local type in MethodDefinition(fib) : a = Type(fixnum)
* [AST] [Fixnum] resolved!
* [Simple] Method type for "-" Type(fixnum) on Type(fixnum) not found.
* [Simple] Invoking plugin: #<Duby::Typer::MathTyper:0xcc5002>
* [Math] Method type for "-" Type(fixnum) on Type(fixnum) = Type(fixnum)
* [AST] [Call] resolved!
* [Simple] Method type for "fib" Type(fixnum) on Type(script) not found.
* [Simple] Invoking plugin: #<Duby::Typer::MathTyper:0xcc5002>
* [Math] Method type for "fib" Type(fixnum) on Type(script) not found
* [Simple] Invoking plugin: #<Duby::Typer::JavaTyper:0x1635aad>
* [Java] Failed to infer Java types for method "fib" Type(fixnum) on Type(script)
* [Simple] Deferring inference for FunctionalCall(fib)
* [Simple] Method type for "+" on not found.
* [Simple] Invoking plugin: #<Duby::Typer::MathTyper:0xcc5002>
* [Math] Method type for "+" on not found
* [Simple] Invoking plugin: #<Duby::Typer::JavaTyper:0x1635aad>
* [Java] Failed to infer Java types for method "+" on
* [Simple] Deferring inference for Call(+)
* [Simple] Deferring inference for If
* [Simple] Learned method fib (Type(fixnum)) on Type(script) = Type(fixnum)
* [AST] [Fixnum] resolved!
* [Simple] Method type for "fib" Type(fixnum) on Type(script) = Type(fixnum)
* [AST] [FunctionalCall] resolved!
* [AST] [PrintLine] resolved!
* [Simple] Entering type inference cycle
* [Simple] Method type for "fib" Type(fixnum) on Type(script) = Type(fixnum)
* [AST] [FunctionalCall] resolved!
* [Simple] [Cycle 0]: Inferred type for FunctionalCall(fib): Type(fixnum)
* [Simple] Method type for "fib" Type(fixnum) on Type(script) = Type(fixnum)
* [AST] [FunctionalCall] resolved!
* [Simple] [Cycle 0]: Inferred type for FunctionalCall(fib): Type(fixnum)
* [Simple] Method type for "+" Type(fixnum) on Type(fixnum) not found.
* [Simple] Invoking plugin: #<Duby::Typer::MathTyper:0xcc5002>
* [Math] Method type for "+" Type(fixnum) on Type(fixnum) = Type(fixnum)
* [AST] [Call] resolved!
* [Simple] [Cycle 0]: Inferred type for Call(+): Type(fixnum)
* [AST] [If] resolved!
* [Simple] [Cycle 0]: Inferred type for If: Type(fixnum)
* [Simple] Inference cycle 0 resolved all types, exiting
There's a lot going on here. You can see the MathTyper and JavaTyper both getting involved here. Since there's no explicit Java calls it's mostly the MathTyper doing all the heavy lifting. The inference stage progresses as follows:
  1. Make a first pass over all AST nodes, performing trivial inferences (declared arguments, literals, etc).
  2. Add each unresolvable node encountered to an unresolved list.
  3. Cycle over that list repeatedly until either all nodes have resolved or the list's contents do not change from one cycle to the next.
It's a fairly brute-force inference mechanism, certainly not on the scale of a full Hindley-Milner inference. Honestly I find the type declaration in the argument list to be far more helpful than harmful, though, and I'm not smart enough to write my own H/M engine at the moment.

Include Java

Anyway, back to Duby. Here's a more complicated example that makes calls out to Java classes:
import "System", "java.lang.System"

def foo
home = System.getProperty "java.home"
System.setProperty "", "something"
hello = System.getProperty ""

puts home
puts hello

puts "Hello world!"
Here we see a few new concepts introduced.

First off, there's an import. Unlike in Java however, import knows nothing about Java types; it's simply associating a short name with a long name. The syntax (and even the name "import") is up for debate...I just wired this in quickly so I could call Java code.

Second, we're actually making calls that leave the known Duby universe. System.getProperty and setProperty are calls to the Java type java.lang.System. Now the Java typer gets involved. Here's a snippit of the inference output for this code:
* [Simple] Method type for "getProperty" Type(string) on Type(java.lang.System meta) not found.
* [Simple] Invoking plugin: #<Duby::Typer::MathTyper:0xaf17c7>
* [Math] Method type for "getProperty" Type(string) on Type(java.lang.System meta) not found
* [Simple] Invoking plugin: #<Duby::Typer::JavaTyper:0x1eb717e>
* [Java] Method type for "getProperty" Type(string) on Type(java.lang.System meta) = Type(java.lang.String)
* [AST] [Call] resolved!
The Java typer is fairly simple at the moment. When asked to infer the return type for a call, it takes the following path:
  1. Attempt to instantiate known Java types for the target and arguments. It makes use of the list of "known types" in the typing engine, augmented by import statements. If those types successfully resolve to Java types...
  2. It uses Java reflection APIs (through JRuby) to look up a method of that name with those arguments on the target type. From this method, then, we have a return type. The return type is reduced to a symbolic name (since again, the rest of the type inference engine knows nothing of Java types) and we consider it a successful inference. If the method does not exist, we temporarily fail to resolve; it may be that additional methods are defined layer that will support this name and argument list.
So in this case, the "System" type has been associated with the "java.lang.System" class (the "meta" in the type reference means it's a class reference rather than an instance reference), and the argument type "string" resolves to "java.lang.String". So java.lang.System.getProperty(java.lang.String) resolves as returning java.lang.String, and we have successfully resolved the call.

Next Steps

I see getting the JVM backend and typer working as two major milestones. Duby already can learn about Java types anywhere in the system and can compile calls to them. But mostly what works right now is what you see above. There's no support for array types, instantiating objects, or hierarchy-aware type inference. There's no logic in place to define new types, static methods, or to define or access fields. All this will come in time, and probably will move very quickly now that the basic plumbing is installed.

I'm hoping to get a lot done on Duby this month while I take a "pseudo-vacation" from constant JRuby slavery. I also have another exciting project on my plate: wiring JRuby into the now-functional "invokedynamic" support in John Rose's MLVM. So I'll probably split my time between those. But I'm very interested in feedback on Duby. This is real, and I'm going to continue moving it forward. I hope to be able to use this as my primary language some day soon.

Update: A few folks asked me to post performance numbers for that fib script above. So here's the comparison between Java and Duby for fib(45).

Java source:
public class FibJava {
public static int fib(int a) {
if (a < 2) {
return a;
} else {
return fib(a - 1) + fib(a - 2);

public static void main(String[] args) {
Java time:
➔ time java -cp . FibJava

real 0m13.368s
user 0m12.684s
sys 0m0.154s
Duby source:
def fib(a => :fixnum)
if a < 2
fib(a - 1) + fib(a - 2)

puts fib(45)
Duby time:
➔ time java -cp . fib

real 0m12.971s
user 0m12.687s
sys 0m0.112s
So the performance is basically identical. But I prefer the Duby version. How about you?

Sunday, August 24, 2008

Zero to Production in 15 Minutes

There still seems to be confusion about the relative simplicity or difficulty of deploying a Rails app using JRuby. Many folks still look around for the old tools and the old ways (Mongrel, generally), assuming that "all that app server stuff" is too complicated. I figured I'd post a quick walkthrough to show how easy it actually is, along with links to everything to get you started.

Here's the full session, in all its glory, for those who just want the commands:
~/work ➔ java -Xmx256M -jar ~/Downloads/glassfish-installer-v2ur2-b04-darwin.jar
~/work ➔ cd glassfish
~/work/glassfish ➔ chmod a+x lib/ant/bin/*
~/work/glassfish ➔ lib/ant/bin/ant -f setup.xml
~/work/glassfish ➔ bin/asadmin start-domain
~/work/glassfish ➔ cd ~/work/testapp
~/work/testapp ➔ jruby -S gem install warbler
~/work/testapp ➔ jruby -S gem install activerecord-jdbcmysql-adapter
~/work/testapp ➔ vi config/database.yml
~/work/testapp ➔ warble
~/work/testapp ➔ ../glassfish/bin/asadmin deploy --contextroot / testapp.war
Now, on to the full walkthrough!


JRuby 1.1.3 or higher

None of the steps in the main walkthrough require JRuby, since Warbler works fine under other Ruby implementations. But if you want to install and test against the JDBC ActiveRecord adapters, JRuby's the way. And in general, if you're deploying on JRuby, you should probably test and build on JRuby as well. Go to under "Download!" and grab the latest 1.1.x "bin" distribution. I link here JRuby 1.1.3 tarball and JRuby 1.1.3 zip for your convenience. Download, unpack, put in PATH. That's all there is to it.

Java 5 or higher

Most systems already have a JVM installed. Here's some links to OpenJDK downloads for various platforms, in case you don't already have one.
  • Windows, Linux, Solaris: Download the JDK directly from Sun's Java SE Downloads page. I typically download the JDK (Java Development Kit) because I find it convenient to have Java sources, compilers, and debugging tools available, but this walkthrough should work with the JRE (Java Runtime Environment) as well. Linux and Solaris users should also be able to use their packaging system of choice to install a JDK.
  • OS X: The 32-bit Intel macs can't run the Apple Java 6, so you'll want to look at the Soylatte Java 6 build for OS X to get the best performance. A small doesn't have Cocoa-based UI components, so it will use X11 if you start up a GUI app.
  • BSDs: FreeBSD users should check the FreeBSD Java downloads page. I believe there's a port for FreeBSD and package/port for OpenBSD but I couldn't dig up the details. We have had users on both platforms, though, so I know they work fine.
  • Others: There's basically a JDK for just about every platform, so if you're not on one of these just do a little digging. All you need to know is that it needs to be a full Java 5 or higher implementation.
Rails 2.0 or higher

Hopefully by now most of you are on a 2.x version of Rails. This walkthrough will assume you've got Rails 2.0+ installed. If you're using JRuby, it's a simple "jruby -S gem install rails" or if you've got JRuby's bin in PATH, "gem install rails" should do the trick. Note that the Warbler (described later) should work in any Ruby implementation, since it's just a packager and it includes JRuby.

Step One: The App Server

The words "Application Server" are terrifying to most Rubyists, to the point that they'll refuse to even try this deployment model. Of course, the ones that try it usually agree it's a much cleaner way to deploy apps, and generally they don't want to go back to any of the alternatives.

Much of the teeth-gnashing seems to surround the perceived complexity of setting a server up. That was definitely the case 5 years ago, but today's servers have been vastly simplified. For this walkthrough, I'll use GlassFish, since it's FOSS, fast, and easy to install.

I'm using GlassFish V2 UR2 (that's Version 2, Update Release 2) since it's very stable and by most accounts the best app server available, FOSS or otherwise. Not that I'm biased or anything. At any rate, it's hard to argue with the install process.

1. Download from the GlassFish V2 UR2 download page. The download links start about halfway down the page and are range from 53MB (English localization) to 93MB (Multilanguage) in size.

2. Run the GlassFish installer. The .jar file downloaded is an executable jar containing the installer for GlassFish as well as GlassFish itself. The -Xmx specified here increases the memory cap for the JVM from its default 64MB to 256MB, since the archive gets unpacked in memory.
~/work ➔ java -Xmx256M -jar ~/Downloads/glassfish-installer-v2ur2-b04-darwin.jar
installation complete
~/work ➔
Before the unpack begins, the installer will pop up a GUI asking you to accept the GlassFish license.
Read the license or's up to you. But to accept, you need to at least pretend you read it and scroll the license to the bottom.
The installer will proceed to unpack all the files for GlassFish into ./glassfish.

3. Run the GlassFish setup script. In the unpacked glassfish directory, there are two .xml files: setup.xml and setup-cluster.xml. Most users will just want to use setup.xml here, but if you're interested in clustering several GlassFish instances across machine, you'll want to look into the clustered setup. I won't go into it here.

The unpacked glassfish dir also contains Apache's Ant build tool, so you don't need to download it. If you already have it available, your copy should work fine, and the chmod command below--which sets the provided Ant's bin scripts executable--would be unnecessary. If you're on Windows, the bin scripts are bat files, so they'll work fine as-is.

Two items to note: you should probably move the glassfish dir where you want it to live in production, and you should run the installer with the version of Java you'd like GlassFish to run under. Both can be changed later, but it's better to just get it right the first time.
~/work ➔ cd glassfish
~/work/glassfish ➔ chmod a+x lib/ant/bin/*
~/work/glassfish ➔ lib/ant/bin/ant -f setup.xml
Buildfile: setup.xml

[mkdir] Created dir: /Users/headius/work/glassfish/bin

[unpack200] Unpacking with Unpack200
[unpack200] Source File :/Users/headius/work/glassfish/lib/appserv-cmp.jar.pack.gz
[unpack200] Dest. File :/Users/headius/work/glassfish/lib/appserv-cmp.jar
[delete] Deleting: /Users/headius/work/glassfish/lib/appserv-cmp.jar.pack.gz
[copy] Copying 1 file to /Users/headius/work/glassfish/config
[copy] Copying 1 file to /Users/headius/work/glassfish/bin
[copy] Copying 1 file to /Users/headius/work/glassfish/bin
[exec] Using port 4848 for Admin.
[exec] Using port 8080 for HTTP Instance.
[exec] Using port 7676 for JMS.
Total time: 29 seconds
~/work/glassfish ➔
4. Start up your GlassFish server. It's as simple as one command now.
~/work/glassfish ➔ bin/asadmin start-domain
Starting Domain domain1, please wait.
Log redirected to /Users/headius/work/glassfish/domains/domain1/logs/server.log.
Redirecting output to /Users/headius/work/glassfish/domains/domain1/logs/server.log
Domain domain1 is ready to receive client requests. Additional services are being started in background.
Domain [domain1] is running [Sun Java System Application Server 9.1_02 (build b04-fcs)] with its configuration and logs at: [/Users/headius/work/glassfish/domains].
Admin Console is available at [http://localhost:4848].
Use the same port [4848] for "asadmin" commands.
User web applications are available at these URLs:
[http://localhost:8080 https://localhost:8181 ].
Following web-contexts are available:
[/web1 /__wstx-services ].
Standard JMX Clients (like JConsole) can connect to JMXServiceURL:
[service:jmx:rmi:///jndi/rmi://charles-nutters-computer.local:8686/jmxrmi] for domain management purposes.
Domain listens on at least following ports for connections:
[8080 8181 4848 3700 3820 3920 8686 ].
Domain does not support application server clusters and other standalone instances.

~/work/glassfish ➔
Congratulations! You have installed GlassFish. Simple, eh?

A few tips for using your new server:
  • There's a web-based admin page at http://localhost:4848 where the admin login is admin/adminadmin by default. You'll want to change that password. Select "Application Server" on the left and then "Administrator Password" along the top.
  • Poke around the admin console to get a feel for the services provided. You won't need any of them for the rest of this walkthrough, but you might want to dabble some day. And if you want to set up a connection pool later on (which ActiveRecord-JDBC supports) this is where you'll do it.
  • Most folks will probably want to set up init scripts to ensure GlassFish is launched at server startup. That's outside the scope of this walkthrough, but it's pretty simple. I'll update this page (and it's equivalent on the JRuby Wiki) once I know more.
  • GlassFish works just fine as a standalone server, but many users will want to proxy it through Apache or another web server. Again, this is outside the scope of this walkthrough, but it should be as simple as configuring a virtual host or a set of matching URLs to hit the GlassFish server at port 8080 (which is the default port for web applications). For apps I'm running, however, I just use GlassFish.
Step 2: Package your App

This step is made super-trivial by Nick Sieger's Warbler. It includes JRuby itself and provides a simple set of commands to package up your app, add a packaging config file, and more. In this case, I'll just be packaging up a simple Rails app.

Note that Warbler works just fine under non-JRuby Ruby implementations, since it's all Ruby code. But again, if you're deploying with JRuby, it's probably a good idea to test and build with JRuby as well.
~/work ➔ jruby -S gem install warbler
Successfully installed warbler-0.9.10
1 gem installed
Installing ri documentation for warbler-0.9.10...
Installing RDoc documentation for warbler-0.9.10...
~/work ➔ cd testapp
~/work/testapp ➔ ls .
README Rakefile app config db doc lib log public script test tmp vendor
~/work/testapp ➔ warble
jar cf testapp.war -C tmp/war .
~/work/testapp ➔
And that's essentially all there is to it. You will get a .war file containing your app, JRuby, Rails, and the Ruby standard library. This one file is now a deployable Rails applications, suitable for any app server, any OS, and any platform without a recompile. The target server doesn't even have to have JRuby or Rails installed.

Step 3: Deploy your Application

There's two ways you can deploy. You can either go to the Admin Console web page, select "Web Applications" from the "Applications" category on the left, and deploy the file there, or you can just use GlassFish's command-line interface. I will demonstrate the latter, and I'm providing the optional contextroot flag to deploy my app at the root context.
~/work/testapp ➔ ../glassfish/bin/asadmin deploy --contextroot / testapp.war
Command deploy executed successfully.
~/work/testapp ➔
That's it? Yep, that's it! If we now hit the server at port 8080, we can see the app is deployed.
Step 4: Tweaking

There's a few things you can do to tweak your deployment a bit. The first would be to generate a warble.rb config file and adjust settings to suit your application.
~/work/testapp ➔ warble config
~/work/testapp ➔ head config/warble.rb
# Warbler web application assembly configuration file do |config|
# Temporary directory where the application is staged
# config.staging_dir = "tmp/war"

# Application directories to be included in the webapp.
config.dirs = %w(app config lib log vendor tmp)
In this file you can set the min/max number of Rails instances you need, additional files and directories to include, additional gems and libraries to include, and so on. The file is heavily commented, so you should have no trouble figuring it out, but otherwise the Warbler page on the JRuby Wiki is probably your best source of information. And since a lot of people ask how many instances they should use, I'll provide a definitive answer: it depends. Try the defaults, and scale up or down as appropriate. Hopefully with Rails 2.2 this will no longer be needed, as is the case for Merb and friends.

The other tweak you'll probably want to look into is using the JDBC-based ActiveRecord adapters instead of the pure Ruby versions (or the C-based versions, if you're migrating from the C-based Ruby impls). This is generally pretty simple too. Install the JDBC adapter for your database, and tweak your database.yml. Here's the commands on my system:
~/work/testapp ➔ jruby -S gem install activerecord-jdbcmysql-adapter
Successfully installed jdbc-mysql-5.0.4
Successfully installed activerecord-jdbcmysql-adapter-0.8.2
2 gems installed
Installing ri documentation for jdbc-mysql-5.0.4...
Installing ri documentation for activerecord-jdbcmysql-adapter-0.8.2...
Installing RDoc documentation for jdbc-mysql-5.0.4...
Installing RDoc documentation for activerecord-jdbcmysql-adapter-0.8.2...
~/work/testapp ➔ vi config/database.yml
~/work/testapp ➔
And the modified database.yml file:
# This was changed from "adapter: mysql"
adapter: jdbcmysql
encoding: utf8
Now repackage ("warble" command again), redeploy, and you're done!


Hopefully this walkthrough clears up some confusion around JRuby on Rails deployment to an app server. It's really a simple process, despite the not-so-simple history surrounding Enterprise Application Servers, and GlassFish almost makes it fun :)

Sunday, August 17, 2008

Q/A: What Thread-safe Rails Means

There's been a little bit of buzz about David Heinemeier Hansson's announcement that Josh Peek has joined Rails core and is about to wrap up his GSoC project making Rails finally be thread-safe. To be honest, there probably hasn't been enough buzz, and there's been several misunderstandings about what it means for Rails users in general.

So I figured I'd do a really short Q/A about what effect Rails thread-safety would have on the Rails world, and especially the JRuby world. Naturally there's some of my opinions reflected here, but most of this should be factually correct. I trust you will offer corrections in the comments.

Q: What does it mean to make Rails thread-safe?

A: I'm sure Josh or Michael Koziarski, his GSoC mentor, can explain in more detail what the work involved, but basically it means removing the single coarse-grained lock around every incoming request and replacing it with finer-grained locks around only those resources that need to be shared across threads. So for example, data structures within the logging subsystem have either been modified so they are not shared across threads, or locked appropriately to make sure two threads don't interfere with each other or render those data structures invalid or corrupt. Instead of a single database connection for a given Rails instance, there will be a pool of connections, allowing N database connections to be used by the M requests executing concurrently. It also means allowing requests to potentially execute without consuming a connection, so the number of live, active connections usually will be lower than the number of requests you can handle concurrently.

Q: Why is this important? Don't we have true concurrency already with Rails' shared-nothing architecture and multiple processes?

A: Yes, processes and shared-nothing do give us full concurrency, at the cost of having multiple processes to manage. For many applications, this is "good enough" concurrency. But there's a down side to requiring as many processes as concurrent requests: inefficient use of shared resources. In a typical Mongrel setup, handling 10 concurrent requests means you have to have 10 copies of Rails loaded, 10 copies of your application loaded, 10 in-memory data caches, 10 database connections...everything has to be scaled in lock step for every additional request you want to handle concurrently. Multiply the N copies of everything times M different applications, and you're eating many, many times more memory than you should.

Of course there are partial solutions to this that don't require thread safety. Since much of the loaded code and some of the data may be the same across all instances, deployment solutions like Passenger from Phusion can use forking and memory-model improvements in Phusion's Ruby Enterprise Edition to allow all instances to share the portion of memory that's the same. So you reduce the memory load by about the amount of code and data in memory that each instance can safely hold in common, which would usually include Rails itself, your static application code, and to some extent the other libraries loaded by Rails and your app. But you still pay the duplication cost for database connections, application code, and in-memory data that are loaded or created after startup. And you still have "no better" concurrency than the coarse-grained locking since Ruby Enterprise Edition is is just as green-threaded as normal Ruby.

Q: So for green-threaded implementations like Ruby, Ruby EE, and Rubinius, native threading offers no benefit?

A: That's not quite true. Thread-safe Rails will mean that an individual instance, even with green threads, can handle multiple requests at the same time. By "at the same time" I don't mean threads will never allow two requests to actually run concurrently or to utilize multiple cores. What I mean is that if a given request ends up blocking on IO, which happens in almost all requests (due to REST hits, DB hits, filesystem hits and so on), Ruby will now have the option of scheduling another request to execute. Put another way, removing the coarse-grained lock will at least improve concurrency up to the "best" that green-threaded implementations can do, which isn't too bad.

The practical implication of this is that rather than having to run a Rails instance for every process you want to handle at the same time, you will only have to run a certain constant number of instances for each core in your system. Some people use N + 1 or 2N + 1 as their metric to map from cores (N) to the number of instances you would need to effectively utilize those cores. And this means that you'd probably never need more than a couple Rails instances on a one-core system. Of course you'll need to try it yourself and see what metric works best for your app, but ultimately even on green-threaded implementations you should be able to reduce the number of instances you need.

Q. Ok, what about native-threaded implementations like JRuby?

A. On JRuby, the situation improves much more than on the green-threaded implementations. Because JRuby implements Ruby threads as native kernel-level threads, a Rails application would only need one instance to handle all concurrent requests across all cores. And by one instance, I mean "nearly one instance" since there might be specific cases where a given application bottlenecks on some shared resource, and you might want to have two or three to reduce that bottleneck. In general, though, I expect those cases will be extremely rare, and most would be JRuby or Rails bugs we should fix.

This means what it sounds like: Rails deployments on JRuby will use 1/Nth the amount of memory they use now, where N is the number of thread-unsafe Rails instances currently required to handle concurrent requests. Even compared to green-threaded implementations running thread-safe Rails, it willl likely use 1/Mth the memory where M is the number of cores, since it can parallelize happily across cores with only "one" instance.

Q: Isn't that a huge deal?

A: Yes, that's a huge deal. I know existing JRuby on Rails users are going to be absolutely thrilled about it. And hopefully more folks will consider using JRuby on Rails in production as a result.

And it doesn't end at resource utilization in JRuby's case. With a single Rails instance, JRuby will be able to "warm up" much more quickly, since code we compile and optimize at runtime will immediately be applicable to all incoming requests. The "throttling" we've had to do for some optimizations (to reduce overall memory consumption) may no longer even be needed. Existing JDBC connection pooling support will be more reliable and more efficient, even allowing connection sharing from application to application as well as across instances. And it will put Rails on JRuby on par with other frameworks that have always been (probably) thread-safe like Merb, Groovy on Grails, and all the Java-based frameworks.

Naturally, I'm thrilled. :)

Thursday, August 7, 2008

'Twas Brillig

It's been a long time since I posted last. I figured it was time to get at least a basic update out to folks.

JRuby's been clicking along really well. Perhaps a little too well. Our bug tracker counts 476 open bugs and climbing, even while we're trying to periodically sweep through them. We're slipping behind, but there's a silver lining: every other bug is filed by someone new. So there's happiness in slavery.

Since my last post, a lot has happened. We pushed out JRuby 1.1.3 around the middle of July, which boasted a whole bunch of new awesome. Vladimir Sizikov posted a really nice rundown of the changes.

My favorite item has to be the improved interpreter performance, which now is clearly faster than MRI's interpreter. So whether you're running interpreted or compiled in JRuby, you should be getting pretty solid straight-line performance.

There were also dozens upon dozens of compatibility fixes, an upgrade to RubyGems 1.2, and lots of other miscellaneous changes to make JRuby a little friendlier and easier to use. It's nice to be able to focus on that kind of stuff now that compatibility is mostly a done deal (ok, high 90th percentile...but pretty darn good).

So then...what's on the slate for the next JRuby release?

Making the Leap?

We had originally been talking about JRuby 1.1.3 being the last planned release in the 1.1 line. It had reached a really solid level of stability and performance, people were putting it in production for all sorts of apps, and we were generally very happy with it. "Last planned release" doesn't mean we wouldn't continue to do maintenance releases as just means we weren't going to do day-to-day development against the 1.1 line.

The primary reason for this is a number of large-scale projects we'd been putting off. For example, the "Java Integration" (JI) subsystem of JRuby has long been a serious thorn in our sides. Based off probably a dozen different people's contributions over nearly eight years, it had become an impenetrable maze of code, half in Java, half in Ruby, but all clearly in the "suck" column. And I'm not saying anything negative about the contributors who wrote all that suckage (which includes me)...they all made valiant attempts to improve the situation. But a dozen 5% solutions had led us down an ever-more-terrifying rabbit hole. We faced the ultimate decision: fix or rewrite?

We were all set to rewrite. I'd already started an experiment I called "MiniJava", a mostly code-generated replacement for JI that boasted dynamic dispatch speed no slower than Ruby to Ruby and static dispatch speed (from Java into Ruby) only about two times slower than Java calling Java. There were many levels of awesome there, but one serious problem: no tests.

JRuby's JI layer has evolved over a long period of time, and has been worked on by a mix of TDD fans and non-fans. There have also been numerous attempts to rewrite portions of the system, usually ending far short of intentions and with no new tests for functionality added along the way. As a result, any attempt to replace the JI layer would be an exercise in pain: even *we* didn't know how it all works. It's probably not exaggerating to say that existing test cases covered less than half of JI functionality, and that the current JRuby team probably understood less than half of JI's implementation. Not a great place to start from.

Faced with the certainty of pain and knowing, workaholic that I am, that a large portion of the rewrite would probably fall on my shoulders, I decided to give it one more go. I decided to attempt an in-place refactoring of JRuby's Java Integration layer, tens of thousands of lines of spaghetti Java and Ruby code.


I've had false starts before. The compiler had at least two partial attempts that failed to go anywhere. I've rewritten the JRuby interpreter several times. Hell, I can't count the number of times I tried to refactor JRuby's IO subsystem before finally succeeding. But, that is some seriously heinous code. So I had to start small. How about something I was already intimately familiar with: method dispatch.
puts "Measure bytelist appends (via Java integration)"
5.times {
puts Benchmark.measure {
sb =
foo = org.jruby.util.ByteList.plain("foo")
1000000.times {

puts "Measure string appends (via normal Ruby)"
5.times {
puts Benchmark.measure {
str = ""
foo = "foo"
1000000.times {
str << foo

Here's one of our JI benchmarks. The idea here is that since JRuby's String type is backed by a ByteList, appending to a ByteList through Java integration should be roughly equivalent in execution cost to appending to a String. Or at least, that would be the ideal situations. It was not, however, the case in JRuby 1.1.3.
JRuby 1.1.3

Measure bytelist appends (via Java integration)
3.580000 0.000000 3.580000 ( 3.579149)
2.551000 0.000000 2.551000 ( 2.551251)
2.615000 0.000000 2.615000 ( 2.614461)
2.505000 0.000000 2.505000 ( 2.505265)
2.715000 0.000000 2.715000 ( 2.715380)
Measure string appends (via normal Ruby)
0.290000 0.000000 0.290000 ( 0.289629)
0.247000 0.000000 0.247000 ( 0.246666)
0.259000 0.000000 0.259000 ( 0.259266)
0.250000 0.000000 0.250000 ( 0.250354)
0.253000 0.000000 0.253000 ( 0.253113)

Ouch. Ten times worse? Sure, I could see it being a couple times worse; after all, going from Ruby code into a Ruby type and back out to Ruby code should be faster than Ruby to Java to Ruby, since we're talking about leaving the controlled world of JRuby core and coming back. But not ten times worse.

Armed with this and a few other benchmarks, I started attacking the dispatch path for Java calls. Largely the optimizations made were the same ones we'd done over the past year for Ruby code: avoid boxing arguments in arrays when possible; clean up and simplify overloaded method selection; as much as possible eliminate constructing any objects not directly used in the eventual reflected call. And early returns started to look really good. Once the basics of the new call logic were in place, performance started to look a lot better.
JRuby trunk

Measure bytelist appends (via Java integration)
1.490000 0.000000 1.490000 ( 1.489711)
0.658000 0.000000 0.658000 ( 0.657639)
0.646000 0.000000 0.646000 ( 0.645717)
0.638000 0.000000 0.638000 ( 0.637849)
0.603000 0.000000 0.603000 ( 0.602439)
Measure string appends (via normal Ruby)
0.312000 0.000000 0.312000 ( 0.312046)
0.232000 0.000000 0.232000 ( 0.231591)
0.243000 0.000000 0.243000 ( 0.242401)
0.235000 0.000000 0.235000 ( 0.235499)
0.232000 0.000000 0.232000 ( 0.231688)

That's more like it...less than three times slower than one of our fastest Ruby-based calls. Other benchmarks showed similar improvement.
JRuby 1.1.3

Measure Integer.valueOf, overloaded call with a primitive
2.384000 0.000000 2.384000 ( 2.383484)
2.167000 0.000000 2.167000 ( 2.167232)
2.191000 0.000000 2.191000 ( 2.191187)
2.212000 0.000000 2.212000 ( 2.211901)
2.202000 0.000000 2.202000 ( 2.202460)

JRuby trunk

Measure Integer.valueOf, overloaded call with a primitive
0.635000 0.000000 0.635000 ( 0.635259)
0.470000 0.000000 0.470000 ( 0.470315)
0.471000 0.000000 0.471000 ( 0.471354)
0.468000 0.000000 0.468000 ( 0.467723)
0.468000 0.000000 0.468000 ( 0.467589)

Here's an example of where improving overload selection made a huge difference. Previously, every time Ruby code called an overloaded Java method, we built up an ArrayList of all argument types, used that to get an aggregate hashcode, used the hashcode to see whether there was a cached previous match, and otherwise went through a brute-force search to match incoming types to outgoing Java signatures.

Wait a second. We constructed an ArrayList just to get a hashcode based on its contents? That dog won't hunt, Monsignor.

So I replaced that logic with a five-line method that aggregates the type's hashcodes into an uber-hashcode, which is then used as the cache key. Combine that with specific-arity searches (avoiding argument boxing in arrays) and search logic that understands Ruby objects (avoiding pre-coercing each argument to potentially the wrong type), and hey, we're starting to see the light.

Keeping it Real

Once it became apparent that a refactoring was most definitely possible, even if it took some hard work and a lot of dedication, we decided to put out a 1.1.4 release, focusing primarily on Java integration, but as always including peripheral bug fixes and performance improvements. Having made that decision, the JI job suddenly became a lot more important. I vowed that 1.1.3 would be the last release to contain a JI layer we were silently afraid of.

Then there was the matter of existing users. Since the beginning of the year, more and more folks have been branching out from the core Rails world that had been JRuby's bread and butter into "Java scripting" sorts of applications. Probably the most prominent ones are the team at Happy Camper Studios, who not only built a real-world-practical Swing framework for JRuby (see MonkeyBars) and a library for packaging up JRuby-based Ruby apps as single-file executables (see Rawr), but who also were pushing the boundaries of what Ruby and JRuby are capable of (see RailGun, and probably see a lot more in the near future). And to top it off, they were releasing MonkeyBars-based apps commercially...making their living depending on JRuby's JI layer.

Would it be a good decision to leave them with the old code for six months while we do a rewrite? I'll let you think about that for a bit.

More Results

Another area that needed serious work was juggling Ruby and Java arrays. The logic for coercing a Ruby array into a Java array was implemented half in Ruby and half in Java, neither half being very efficient in themselves. Add to that the constant back and forth, and you have a recipe for disaster. With another multi-day effort, I managed to wrestle the code to one side of the fence, refactor it, and improve performance over twenty times.

require 'java'
require 'benchmark'

TIMES = (ARGV[0] || 5).to_i

TIMES.times do do |bm|"control") {a = [1,2,3,4]; 100_000.times {a}}"ary.to_java") {a = [1,2,3,4]; 100_000.times {a.to_java}}"ary.to_java :object") {a = [1,2,3,4]; 100_000.times {a.to_java :object}}"ary.to_java :string") {a = [1,2,3,4]; 100_000.times {a.to_java :string}}

JRuby 1.1.3
user system total real
control 0.013000 0.000000 0.013000 ( 0.013130)
ary.to_java 7.523000 0.000000 7.523000 ( 7.522787)
ary.to_java :object 7.794000 0.000000 7.794000 ( 7.794777)
ary.to_java :string 9.905000 0.000000 9.905000 ( 9.905805)

JRuby trunk
user system total real
control 0.009000 0.000000 0.009000 ( 0.009548)
ary.to_java 0.240000 0.000000 0.240000 ( 0.239946)
ary.to_java :object 0.248000 0.000000 0.248000 ( 0.247385)
ary.to_java :string 0.418000 0.000000 0.418000 ( 0.418230)

You can find similar improvements on trunk for object construction, interface implementation, and several other areas. Work continues, but performance is starting to look way better.

Don't You Think About Anything But Performance?

Of course a side effect of simplifying the code for performance reasons is that fixing bugs and adding features becomes a lot easier. Check out a tiny bit of new logic that works now on JRuby trunk:
# "closure conversion" was only supported for instance methods before
thread = { puts 'Wahoo!' }
# output: 'Wahoo!'

Of course this is a somewhat contrived example, but there are some obviously useful examples too:
javax.swing.SwingUtilities.invoke_later { puts "Yay, Swing!" }
# script terminates, but the Swing event thread keeps running until it's fired our block

The ability to pass a block to "any method" that accepted an interface as its last parameter was only functional for instance methods in all previous releases of JRuby. After the refactoring, expanding it to constructors and static methods took about 5 minutes. With tests. We'll come back to that in a moment.

The simplification of the code also means we'll probably be able to fix a bunch of Java Integration bugs we've punted on for months. For example, it's been a long-standing bug that you can't implement a Java interface in Ruby by defining the underscored versions of that interface's method names. But now, with newly rewritten JI interface-implementation code, it should be a snap to add that feature. We're planning to do a JI bug audit for 1.1.4 and knock down as many long-standing issues as we can.

Proving It Works

So back to the tests for a moment.

Some months ago we managed to get a baseline suite of RSpec specs into the JRuby development process thanks to Nick Sieger. Initially, the tests were very sparse, only a few specific things Nick had a chance to put together. But as interested community members started sending in patches, we started to grow a nice little suite.

As I've been working on the refactoring, I've been trying to "test along" as I learn how bits of JRuby's JI layer function. And so I've added a number of new specs for type coercion, interface implementation, method dispatch and overload selection, and others. Ola Bini came out of hiding to contribute a pretty comprehensive set of Array specs, which were a great help during the slaying of the Array-coercion dragon. So we're finally getting that suite we'd always needed, and once the refactoring is done we'll be in a far better position to start taking JI to the next level.

What's Next?

At the moment the refactoring is maybe 25% along. I'm the only one working on it, and it's a crapload of code, so it's going to take a little time. But most key performance bottlenecks have been remedied at this point.

There are a few areas that remain to be tackled:
  • Refactoring the extensive logic governing how Ruby classes can extend abstract or concrete Java classes. Kresten Krab Thorup contributed this well over a year ago, and it's kinda been an island unto itself. It will take a considerable effort to rework.

  • Moving the remaining core JI logic from Ruby into Java. I know, turtles and all that...but I've seen enough projects try to do large-scale refactorings in Ruby to know that the tools and techniques simply aren't there yet. By moving this logic into Java, where it belongs, we'll probably be able to delete 90% of it. That means more room for performance and features, and a much higher likelihood that future enhancements can safely live in Ruby without incurring a severe performance penalty.

  • Eliminating the old "lower level" and "higher level" Java integration layers. Originally, JRuby's JI was implemented as a set of reflection-like Ruby classes wrapping Java's reflection classes (which comprised the "lower level") and a substantial amount of Ruby code that juggled these reflected bits to represent Java types and make Java calls (the so-called "higher level"). While conceptually this makes some sense, in practice it meant that any call from Ruby to Java actually ended up as dozens, maybe hundreds of Ruby invocations before the target Java method could be invoked. Performance improvements over the past four years were able to improve matters, but largely the only substantial gains have come from, and will continue to come from, eliminating the two separate levels entirely.

I intend for all of this to be in place for JRuby 1.1.4, which we want to release this month.

Next Time on Headius: Tune in for my next post, hopefully soon, about two Rubinius APIs we've added for 1.1.4: Multiple VMs (MVM) and Foreign Function Interface (FFI).

Sunday, August 3, 2008

libdl _dl_debug_initialize problem solved

I'm posting this to make sure it gets out there, so nobody else spends a couple days trying to fix it.

Recently, after some upgrade, I started getting the following message during JRuby's run of RubySpecs:
Inconsistency detected by dl-open.c: 623: _dl_open: Assertion
`_dl_debug_initialize (0, args.nsid)->r_state == RT_CONSISTENT' failed!

I narrowed it down to a call to getgrnam in the 'etc' library, which we provide using JNA. Calling that function through any means caused this error.

My debugging skills are pretty poor when it comes to C libraries, so I mostly started poking around trying to upgrade or downgrade stuff. Most other posts about this error seemed to do the same thing, but weren't very helpful about which libraries or applications were changed. Eventually I got back to looking at libc6 versions, to see if I could downgrade and hopefully eliminate the problem, when I saw this text in the libc6-i686 package:
 WARNING: Some third-party binaries may not work well with these
libraries. Most notably, IBM's JDK. If you experience problems with
such applications, you will need to remove this package.

Nothing else seemed to depend on libc6-i686, so I removed it. And voila, the problem went away.

This was on a pretty vanilla install of Ubuntu 7.10 desktop with nothing special done to it. I'm not sure where libc6-i686 came from.

On a side note, sorry to my regular readers for not blogging the past month. I've been hard at work on a bunch of new JRuby stuff for an upcoming 1.1.4 release. I'll tell you all about it soon.