JavaOne Wrap-up

It was a great JavaOne for JRuby! Now that Tom and I are back home and I've recuperated from a long and busy week, here's an update on the great JRuby JavaOne Adventure.

The Press Conference

Tim Bray continues to support and promote the JRuby project, along with other alternative JVM languages. He set up a press conference on Tuesday the 16th for us along with Sam Heisz from Caucho (presenting Quercus, Caucho's GPLed PHP implementation for the JVM) and Roberto Chinnici from Sun (presenting Phobos, a scripting-language-based web framework for JSR223-compatible languages). Sam presented first, then us, then Roberto.

My rough count puts the number of journalists and interested parties at 15-20 (some came in late or left early).

We were given ten minutes to make our case for JRuby. Tom kicked it off with a quick overview of what Ruby and JRuby are, and I made a quick statement about Ruby's potential on the JVM. Then I demoed building a tiny Swing app through IRB in under two minutes before handing it back to Tom for his JRuby on Rails demo.

I think both demos were well-received. With the press Ruby and Rails have been getting, JRuby tends to sell itself.

The inevitable questions about performance and business cases were mostly deflected by Tim, and for that I thank him. Comparing performance is somewhat pointless at this point in JRuby's development, since we've only recently begun to explore optimizations. The business case for JRuby is also simply the business case for Ruby; if the language suits your problem, JRuby is a good option.

There were also questions asking us to compare JRuby to the other languages and implementations out there. These we mostly punted on, avoiding a fight. Folks seem to really want the alternative languages implementors to fight, and this became a recurring theme during the week.

At the end of the press conference, a number of Sun folks talked to us about building a community around JRuby, and possibly having a presence on java.net. We're very enthusiastic about growing the JRuby community and will weigh the various options for some minimal java.net presence.

Presentation for InfoQ.com

InfoQ.com is a new online tech magazine currently launching. We were invited by Floyd Marinescu and Obie Fernandez to participate in the launch by presenting our JRuby talk, sans live demos and JavaOne branding, to be videotaped and published on the InfoQ site. This talk we gave on Wednesday the 17th at noon.

Floyd is one of the original founders of TheServerSide, and we were happy to help him kick off this new venture--especially given its broad acceptance of alternative development and web technologies like Ruby and Rails.

The presentation took place in the upstairs theater of the Thirsty Bear, a typical conference-goer's hangout near the Moscone convention center. The room was set up with rows of chairs, a projector, a podium, and a video camera. Upon starting, we had only a few attendees, but several more trickled in during our presentation. According to the InfoQ guys, we had the best turnout of any talks so far that morning.

A number of good questions came up, which Tom and I answered on camera. It was a great dry-run for our "real" JavaOne presentation later that day.

Look for the final presentation to be posted on the InfoQ site within the next week.

The Main Event

Some time before the InfoQ presentation I stopped practicing. I'm not one for scripted presentations; I do better with a general presentation outline and a couple practice rehearsals. I approached my RubyConf presentation this way and it worked out well, so I followed the same pattern for JavaOne.

In total, I think I rehearsed my part of the presentation only 4 times. I ran through the IRB demo many times, however, to make sure I had the sequence of commands memorized.

All that said, I spent the early afternoon relaxing. I hung out in the pavilion for a while, relaxed in the gaming area, and then attended the Groovy talk around 2:45. The basics of the Groovy presentation seemed much like in past years. The slides were mostly the same, and the content seemed to vary only slightly. There was an extensive demo using ActiveX to control an Excel document; I was not quite sure whether this was intended to be a Groovy demo or an ActiveX demo, since the same operations could be done by Java or any other scripting languages just as easily. At any rate, it was a good prep for our presentation which came immediately after.

I was obviously there early, and Tom showed up well ahead of time as well. We got mic'ed up, talked a bit on how ready we were, and then relaxed while folks streamed into the auditorium. I estimate it was perhaps 70% full, for around 700 people, but we'll get an official count sometime soon.

We decided on the following order for the presentation:

Introductions and opening words: Me
Agenda, intro to JRuby and Ruby: Tom
IRB demo: Me
Rails demo: Tom
JRuby in the Wild, JRuby's Future, and Conclusion: Me
Q/A: Tom and Me

Up until Wednesday morning, I had no plans for the introduction. I knew I wanted to ask a few introductory questions, and eventually settled on the following two:

- How many of you have heard of Ruby or Ruby on Rails? (Perhaps 75-80% of the attendees raised their hands
- How many of you heard of them for the first time in the past year? (Almost all the same hands)

"That's momentum. That's the momentum behind Ruby and Rails right now".

I proceeded to hold up the current Dr Dobbs Journal, with a cover story "Ruby on Rails: Java's Successor?". Most of the audience chuckled at the title, and I agreed that perhaps it wasn't the right question. I mentioned that the article described Rails as a potential tipping point in web development patterns and called out the fact that several new frameworks had sprung up in the Java world aping Rail's design. Then I handed off to Tom.

Tom gave a very quick and clean introduction to Ruby, JRuby, and features of both. I've read on several blog postings about the presentation that this intro was welcomed by attendees; though many had heard about Ruby, most had no experience with the code. The JRuby features also called out Java integration scenarios, which are the obvious reason why you'd want to run JRuby in the first place. Tom did a great job on these slides, which you can imagine were not terribly exciting for us to present. Then it was my turn.

The IRB demo was our first really cool thing to show off. I started off demonstrating some of the basic of Ruby and IRB, entering basic statements, defining a method, defining a class. Then I went straightaway into the fun stuff: building that little Swing app. Tom commented after the press conference and again after this presentation how impressive he thought it was, and there were certainly murmurs, gasps, and nods from the attendees. It really provided a good feel for how easy it is to call Java code from Ruby.

Tom came next, providing the "big surprise" for JavaOne attendees: a fully working Rails app, running under JRuby and using JDBC for database access. The wow factor was not as apparent here, since it just looked like a web application; however, several folks in the audience with more than a basic knowledge of Rails were notably impressed. Through follow-up questions and later blog posts, the gravity of this milestone has begun to be felt.

I gave a quick overview of RDT, RadRails, JEdit, and DataVision, four apps that make use of JRuby internals. Then I ran through the various plans and future milestones for JRuby, adjusting them based on recent developments (since our original slides were submitted in March, and mountains have moved since then).

We finished with about fifteen minutes to go, which I think was refreshing for most of the attendees. We covered everything we wanted to cover, and we knew people were having difficulty moving from session to session. We also wanted plenty of time for Q/A.

Questions ranged from performance to rails to "can I run normal Ruby code" to "can I use Java objects in Ruby scripts". We patiently answered all questions as well as possible; there was a lot of interest in using JRuby for its Java integration abilities.

Both Tom and I thought the presentation went extremely well. We were very happy with the turnout and we're excited to take the next steps with JRuby.

JRuby Meet-up, San Francisco, May 19 4:00PM

UPDATE: The correct date is May 19th

After a crazy week at JavaOne, Tom and I will be settling in at the Chieftain in San Francisco to unwind. We'd love to have any JRuby enthusiasts stop by and chat a bit. Tom's blog has all the details. RSVP if you like, so we we'll know to look for you, but show up regardless!

And They Said JRuby was Dead...

Things are looking very good for JavaOne.

The past several months have been a period of furious development on the JRuby project, with milestone after milestone flying by. Four months ago we couldn't run most basic Ruby apps and were still wrestling with basic 1.8 compatibility issues. Since that time we've gotten IRB working, Rake running, RubyGems almost working, and then managed to process a simple Rails request. All the while, our existing libraries have grown in stability and completeness, and we've attracted many new contributors to the project. This stone is gathering no moss.

JavaOne is a going to be a big event for us, but we wanted something more. What would the week before a major conference be without some buzz-worthy announcements to make?

Hold on to your hats.

Where We've Been

JRuby has been around for around five years. It was originally based on Ruby 1.6 C code, and was pretty much a direct port from C to Java. Over those years, up until Fall of 2005, the focus was mainly on basic compability issues, updating libraries and language semantics to 1.8 levels, and generally just making things work well enough for use. All told, it was a pretty good port, but there was a long way to go. There were a few Java apps that used it for scripting, but the tide was turning away from JRuby with the release of ever-more-complex Ruby applications that were increasingly difficult to run on a partial implementation.

Starting this past Fall, and really gaining steam since the start of 2006, our focus shifted from simple 1.8 compatibilities and interpreter tweaks to much more ambitious goals. We began to tackle the killer apps for Ruby one by one, fixing what broke and getting a much more intimate knowledge of both Ruby and JRuby internals than we had before. First came early runs of Rake. They worked, but required changes to the way external scripts launched. Then something a bit more lofty: I disappeared into my office for a week to get IRB working...and work it eventually did. Then we realized that RubyGems would be a great target; while it's not yet working 100%, it's now really close.

Then came Rails.

The Big Ticket

There is little doubt that Rails is the app that's selling Ruby today. Ruby has become, in many circles, synonymous with Ruby on Rails. While there's a discussion to be had about whether a single killer app should hold so much sway, it's impossible to argue that Ruby would be where it is today without Rails dragging it into the spotlight. Rails has, more than any other app, brought Ruby to the world.

We just had to get a bite of that apple.

Getting Rails to run in JRuby is a very challenging task. For those of you that know what Rails does, imagine the code that's required to do that. Think of all the magic necessary to get things so slick and transparent. Look at the size of the Rails codebase which, while far smaller than a beastly Java library, is positively enormous compared to the typical Ruby app. If you've ever seen the code, you know what kinds of tricks and traps there are in the Rails code, and how many dark alleys you might venture down when debugging a problem deep within its guts.

Now imagine running it on an interpreter that breaks in weird and indeterminate ways.

We have fixed hundreds of issues on the road to getting Rails working. Every step of the way, we have had to pore through the Rails code, digging for that one irreducible test case that demonstrates a bug, sometimes sifting through tens of files, thousands of lines of code. It has been an extreme challenge. However...

What Would You Do With JRuby on Rails?

Today, we can announce that a simple end-to-end Rails application has successfully run under JRuby.

As reported on Tom's blog, a little CRUDified cookbook application can now deliver a form and process submissions correctly. Using an ActiveRecord-to-JDBC adapter (yes, you read that right) donated by Nick Sieger, the app is able to generate a form, display a table of results, and process creates and deletes. All this magic, all working on JRuby. We're ecstatic.

Can It Truly Be?

See for yourself. It's even wrapped in a little servlet that invokes Rails FastCGI-style, keeping it loaded in memory for additional requests. It's the culmination of all our experiments, bug fixes, and hard work over the past several months. And it sure feels good.



Onward and Upward

Of course this demo is slightly canned. We coded to the test, and the average Rails app still isn't going to work out of the box. There's plenty of little things missing, like reliable session management, more ActiveRecord magic, and various rendering flaws, to list a few we know about. But as a technology preview, running Rails essentially unmodified--even on a simple end-to-end app--is our biggest milestone yet.

With the recent successes, we plan to target the end of summer for full, general-purpose Rails support. With more contributors and more time to work, that timetable could move up, but there's also the compiler work that has begun in earnest. It's going to be a great summer for coding, and a great year for dynamic(-typed) lanaguages on the JVM.

Oh, and one last note for the perennial skeptics and naysayers out there: Rails on the JVM is coming sooner than you think.

A Few Other Updates

Here's another couple items to hold you off until the next round of milestones:

IRB glitches resolved
A remaining IRB bug that caused declared classes to be scoped under Kernel (as in class A would be scoped as Kernel::A) has been resolved. I had to get this fixed to demo IRB at JavaOne

RubyGems works, sorta
Tom discovered that if a known good RubyGems install (from a C Ruby installation, for example) was copied into JRuby's dirs, gems would require and load correctly. So although our RubyGems and gem install code isn't quite there yet, actually using gems seems to be working. Huzzah!

Improved codegen DSL
John Lam, of the RubyCLR project, pointed me to a nice little eval trick that has helped simplify my code-generation DSL a bit. By eliminating most of the block params, it's a bit less verbose and certainly prettier:

class_bytes = ClassBuilder.def_class :public, "FooClass" do
  def_constructor :public do
    call_super
    return_void
  end

  def_method :public, :string, "myMethod", [:void], [:exception] do
    call_this GenUtils.array_cls(:string), "getStringArr"
    call_this :string, "getMessage"
    return_top :ref
  end
  
  def_method :private, :string, "getMessage", [:void] do
    construct_obj :stringbuffer, [:string] do
      constant "Now I will say: "
    end
    
    call_method :stringbuffer, "append", :string, :stringbuffer do
      constant "Hello CodeGen!"
    end
    call_method :string, "toString", :void, :stringbuffer
    return_top :ref
  end
  
  def_method :public, GenUtils.array_cls(:string), "getStringArr" do
    construct_array :string, 5 do |i|
      constant "string \##{i}"
    end
    
    array_set 2, :string do
      constant "replacement at index 2"
    end
    
    return_top :ref
  end
end

I'll be going full-speed-ahead on compilation after JavaOne, you can count on that.

YAML parser improved?
One of our favorite contributors, Ola Bini, now has a working version of a YAML 1.1-compliant parser which is able to load fairly large files. There's still some work to do, but if all goes well we may be able to migrate to it in the near term.

Dynamic languages press conference
The JRuby team (Tom and I) will be part of a JavaOne Day-1 press conference on the future of dynamic languages on the JVM. Along with the Quercus guys from Caucho (PHP for the JVM) and Roberto Chinnici from Sun (doing cool stuff with JSR223), we'll have our own ten minutes to show the press how cool JRuby really is. It oughta be a fun time, and should help get JRuby a bit wider exposure.

This is just too much fun.

Scripting and services top Sun software chief's list

Scripting and services top Sun software chief's list:

"Support for scripting languages in Java, squeezing revenue from a set of long-promised - but strangely MIA - services, and more open source initiatives are among top priorities for Sun Microsystems' newly appointed software chief.…"

Seems like a good time to be working on an open-source "scripting" language implementation, no?

A DSL for Bytecode Generation

Having discovered the power and magic of bytecode generation, it occurred to me that none of the existing libraries have the subtle elegance that most code generation tasks really deserve. I believe there's a simple reason for this: they're written in Java. Some of them really try to make things easier, and perhaps come close to succeeding, but they're all still cumbersome, clunky, and very, very verbose.

I have been writing JRuby's compiler in pure ruby by calling out to a Java-based bytecode generation library. My initial attempts were fairly straightforward calls: push this, call that, pop the other. Very linear, very boring, very verbose, and not a great deal simpler than the equivalent Java code. It seemed such a shame to waste an expressive language like Ruby on such a menial task, I've decided to build a domain-specific language for Java bytecode generation.

A short sample of what works today (basically the operations I needed for my test compiler):

class_bytes = ClassBuilder.def_class :public, "FooClass" do |c|
  c.def_constructor :public do |con|
    con.call_super
    con.return_void
  end

  c.def_method :public, :string, "myMethod", [:void], [:exception] do |m|
    m.call_this GenUtils.array_cls(:string), "getStringArr"
    m.call_this :string, "getMessage"
    m.return_top :ref
  end
  
  c.def_method :private, :string, "getMessage", [:void] do |m|
    m.construct_obj :stringbuffer, [:string] do |p|
      p.constant "Now I will say: "
    end
    
    m.call_method :stringbuffer, "append", :string, :stringbuffer do |p|
      p.constant "Hello CodeGen!"
    end
    m.call_method :string, "toString", :void, :stringbuffer
    m.return_top :ref
  end
  
  c.def_method :public, GenUtils.array_cls(:string), "getStringArr" do |m|
    m.construct_array :string, 5 do |p,i|
      p.constant "string \##{i}"
    end
    
    m.array_set 2, :string do |p|
      p.constant "replacement at index 2"
    end
    
    m.return_top :ref
  end
end

This approach has a number of advantages over others:

• The structure of the generator is very similar to that of the generated code
• Method parameters and array initializers (or the code to make them available) are logically associated with the eventual call or array they'll apply to
• The builders maintain some internal state, and will be able to count stack depth, validate typing, automatically attempt casts, and automatically return the correct types
• It's far easier to read
• It's far more fun

Some notes on the code above:

• The param-building blocks (with |p| params) are in all cases optional. If omitted, method calls will assume you have prepared all params, array creation will create an empty array, and array sets will assume the value is already present on the stack.
• The core bytecode operations (dup, etc) are still present and callable on the MethodBuilder m. This allows you to fall back to linear-style when necessary.
• Various classes (perhaps eventually all classes) from java.lang are aliased as symbols like :string and :object. At the ClassBuilder level, it is also possible to "import" classes, as in c.import "javax.swing.JFrame", :jframe and use the aliased symbol throughout this generation (much like import in a .java file)
• I'm looking for a better way to handle arrays. GenUtils is only used internally except for array types, and I'd like to hide it completely.
  

I'm tossing this working snippit out to the world for comments and critique...and perhaps as a little teaser of things to come. I'm planning to add a few more operations and port the early v1 compiler over to this soon...then both will develop together. I see this DSL/library as having huge potential for other projects that want a simple, elegant way to do bytecode generation.

Thoughts? Comments?

My JavaOne Calendar

I've entered all the sessions I'm planning to attend into Google Calendar, available at the following URLS:

- RSS
- ICal
- You can search for "Headius at JavaOne" in GCal too.

These are negotiable if there's folks out there that want to grab a lunch, dinner, or 30 minutes and a cup of coffee. Do let me know: email

I'm arriving late Monday and leaving late Saturday...the rest is on the calendar.