Isaac Gouy has updated the JRuby numbers, and modified the default comparison to be with Ruby 1.8.6 rather than with Groovy as it was before. And true to form, JRuby is faster than Ruby on 14 out of 18 benchmarks.
There are reasons for all four benchmarks that are slower:
- pidigits is simply too short for JRuby to hit its full stride. Alioth runs it with n = 2500, which on my system doing a simple "time" results in JRuby taking 11 seconds and Ruby taking 5. If I bump that up to 5000, JRuby takes 27 seconds to Ruby's 31.
- regex-dna and recursive-complement are both hitting the Regexp performance problem we have in JRuby 1.0.x and in the 1.1 beta. We expect to have that resolved for 1.1 final, and Ola Bini and Marcin Mielczynski are each developing separate Regexp engines to that end.
- startup, beyond being a touch unfair for a JVM-based language right now, is actually about half our fault and half the JVM's. The JVM half is the unpleasantly high cost of classloading, and specifically the cost of generating many small temporary classes into their own classloaders, as we have to do in JRuby to avoid leaking classes as we JIT and bind methods at runtime. The JRuby half is the fact that we're loading and generating so many classes, most of them too far ahead of time or that will never be used. So there's blame to go around, but we'll never have Ruby's time for this.
"startup, beyond being a touch unfair for a JVM-based language right now ..."
ReplyDeleteI have to disagree with that ;-)
It might be irrelevant if we only ever invoke the programs on a long running JRuby server, but that might not be what we're doing.
We're talking about JRuby startup ~25x slower than Java startup, ~10x slower than Rhino startup.
If you think startup will still be this slow beyond the beta then I think you need to loudly set expectations.
"pidigits is simply too short for JRuby to hit its full stride"
Which is a most euphemistic way of saying JRuby startup is so slow that it swamps the calculation time :-)
Long startup time is an explanation why a program is slow - it doesn't make the program any faster.
That's great news, but I have to agree with Isaac. Startup times shouldn't be ignored - especially given the scripting language aspect of Ruby. If you want to use JRuby for short scripts, you'd want it to startup pretty much immediately.
ReplyDeletePutting Ruby on the JVM brings some amazing benefits, but there's no getting away from the fact it's Java. That's the price of progress I suppose :-/ Is there any way that JRuby could be launched in a 'light' mode that sacrifices long term performance for a quicker start up through dynamic loading, or is this entirely down to JVM startup times?
Good Work!
Stu