We need diversity in the JVM Languages group, and it's been brought to my attention that some popular/key/interesting languages may not have representation. So we need to change that.
If you are interested in the future of non-Java languages on the JVM, you should be on this list. Yes, we talk about a lot of JVM language implementation challenges, we discuss compilers and stack frames and call-site optimizations, but we also talk about features peripheral to language implementation like package indexing and retrofitting Java 5+ code. We need your help.
Once you've joined, or if you're already member, you have a second task
I respectfully request that each of you search out one individual you think would be interested in the list and try to get them involved. Toss them a quick email, invite them to describe their project or language or implementation to us, and promise them they're joining a very interesting and entertaining community. History will thank you, and so will I.
Sunday, August 5, 2007
A Business Case for Supporting Jython
The facts:
Put simply: if you want to become an OSS rockstar tomorrow, get Mercurial running on Jython.
(credit for this idea goes to OpenJDK ambassador Tom Marble...I'm just trying to needle the OSS world into making it happen)
- Sun and many other organizations have started considering moves to Mercurial. In Sun's case, it's a mandate for all Sun-managed OSS projects (OpenSolaris, OpenJDK, etc).
- Moving projects to Mercurial frequently (usually?) requires IDE/tool support.
- Sun's IDE/tools and those of many other organizations are Java-based (NetBeans, Eclipse, and so on).
- Mercurial is written in Python.
- Jython is an implementation of Python for the JVM.
Put simply: if you want to become an OSS rockstar tomorrow, get Mercurial running on Jython.
(credit for this idea goes to OpenJDK ambassador Tom Marble...I'm just trying to needle the OSS world into making it happen)
Thursday, July 19, 2007
Alioth Numbers for JRuby 1.0
Someone pointed out to me the other day that the Alioth "Compuer Language Benchmarks Game" (as they call it now) had started to include JRuby 1.0 in the results. And how do we fare? We're slower than Ruby 1.8.6! Hooray!
But not by much. Depending on your definition of "on par", I'd say we're safely in the range of being on par with Ruby 1.8.6 performance, at least for these too-short, too-small measurements.
Alioth benchmarks are regularly panned by language enthusiasts...or at least enthusiasts on the "slow" end of the benchmarks. In the case of JVM-based language implementations, the problem is the same old excuse: not enough time for the JVM JIT to work its magic. This case isn't all that different, but it's fair to say that Alioth isn't testing straight-line performance in a hot application, but getting from point A to point B without a running start. And in that domain, it provides a reasonable window into implementation performance.
So then, the JRuby 1.0 numbers. You can click the link to see them, but they break down about like this:
Performance:
So I guess that's all I've got to point out. We've been working on performance since 1.0, and there's a number of major improvements planned for the 1.1 release. And considering that "beating Ruby performance" wasn't a primary goal for JRuby 1.0, I think our "roughly on par" numbers here are pretty damn good. Granted, Ruby 1.8.x isn't the fastest implementation in the world, but we're pretty happy to have improved performance by an order of magnitude in the past year.
Now, onward to the future!
But not by much. Depending on your definition of "on par", I'd say we're safely in the range of being on par with Ruby 1.8.6 performance, at least for these too-short, too-small measurements.
Alioth benchmarks are regularly panned by language enthusiasts...or at least enthusiasts on the "slow" end of the benchmarks. In the case of JVM-based language implementations, the problem is the same old excuse: not enough time for the JVM JIT to work its magic. This case isn't all that different, but it's fair to say that Alioth isn't testing straight-line performance in a hot application, but getting from point A to point B without a running start. And in that domain, it provides a reasonable window into implementation performance.
So then, the JRuby 1.0 numbers. You can click the link to see them, but they break down about like this:
Performance:
- Four tests are equal or faster in JRuby--usually not more than 2x faster, but "4.8x" faster in one case. That fastest one involves "concurrency", and I haven't studied it enough to know whether it's meaningful or not.
- Eight tests are less than 2x slower in JRuby.
- The remaining four tests are greater than 2x slower in JRuby.
- Startup time is considerably worse; it's listed as 305x slower for JRuby (!!!) but it's not a particularly useful ratio. We take a couple seconds to get going compared to Ruby's hundredths. That's life.
- All except one worse in JRuby
- Most more than 2x worse in JRuby
- Several more then 10x worse in JRuby
- Does anyone care?
So I guess that's all I've got to point out. We've been working on performance since 1.0, and there's a number of major improvements planned for the 1.1 release. And considering that "beating Ruby performance" wasn't a primary goal for JRuby 1.0, I think our "roughly on par" numbers here are pretty damn good. Granted, Ruby 1.8.x isn't the fastest implementation in the world, but we're pretty happy to have improved performance by an order of magnitude in the past year.
Now, onward to the future!
Wednesday, July 18, 2007
Groovy and JRuby Cooperating?
Can it be true? Of course it can!
Graeme Rocher blogs about a new feature in upcoming Groovy 1.1 beta 3: support for Ruby-style "method missing" hooks!
In heavily metaprogrammed applications, it's very common to define methods on the fly in response to calls made. For example, in Rails ActiveRecord, no "find by" methods are defined out of the box, but you may still call "find_by_x_and_y_and_z" to run your specific query. In order to reduce the overhead of handling these "missing methods" (usually through parsing the called name), it's typical to generate new methods on demand, binding them to the called class. So if you call find_by_name and it doesn't exist, two things happen: the query is generated and run and a new find_by_name method is dynamically created for future queries.
After Graeme blogged about the typical way to dynamically create methods in Groovy, I tapped him on IM and we started to talk about how Ruby handles the same use case. We agreed that the overhead of overriding invokeMethod (Groovy's entry point for all dynamic calls) was problematic...it's already got enough to do without adding the method-generation logic. As it turns out, "method missing" was a perfect remedy, and pretty easy to add into Groovy. Graeme's already started work to modify Grails to take advantage of methodMissing and I'm sure it will catch on like wildfire in the rest of Groovydom. And as a result you, the JVM dynamic language community, gain the benefit of a little Groovy/JRuby cross-pollination.
So for all you haters out there who just love to pit language teams against one another I say this:
Nyah.
Graeme Rocher blogs about a new feature in upcoming Groovy 1.1 beta 3: support for Ruby-style "method missing" hooks!
In heavily metaprogrammed applications, it's very common to define methods on the fly in response to calls made. For example, in Rails ActiveRecord, no "find by" methods are defined out of the box, but you may still call "find_by_x_and_y_and_z" to run your specific query. In order to reduce the overhead of handling these "missing methods" (usually through parsing the called name), it's typical to generate new methods on demand, binding them to the called class. So if you call find_by_name and it doesn't exist, two things happen: the query is generated and run and a new find_by_name method is dynamically created for future queries.
After Graeme blogged about the typical way to dynamically create methods in Groovy, I tapped him on IM and we started to talk about how Ruby handles the same use case. We agreed that the overhead of overriding invokeMethod (Groovy's entry point for all dynamic calls) was problematic...it's already got enough to do without adding the method-generation logic. As it turns out, "method missing" was a perfect remedy, and pretty easy to add into Groovy. Graeme's already started work to modify Grails to take advantage of methodMissing and I'm sure it will catch on like wildfire in the rest of Groovydom. And as a result you, the JVM dynamic language community, gain the benefit of a little Groovy/JRuby cross-pollination.
So for all you haters out there who just love to pit language teams against one another I say this:
Nyah.
Sunday, July 15, 2007
The Ever-Evolving JVM
John Rose, lead of the "invokedynamic" effort (Java Specification Request 292), has posted some exciting articles about the future of the JVM and a number of changes potentially for the next Java version. Among these is, of course, the dynamic invocation efforts, but these entries include information on non-local returns from closures, tail calls, and tuple support. I'm excited to have John as a co-worker and to be helping out the invokedynamic effort in my own small way by working on JRuby's compiler. John has also provided guidance on how to make a dynamic language fast on current JVMs, which has informed much of my compiler's design.
Check out his articles:
Longumps Considered Inexpensive
tail calls in the VM
tuples in the VM
It's going to be a fun year for language developers on the JVM!
Check out his articles:
Longumps Considered Inexpensive
tail calls in the VM
tuples in the VM
It's going to be a fun year for language developers on the JVM!
More Compiler Strategy: Call Adapters and Stack-based Methods
Compilers are hard. But not so hard as people would have you believe.
I've committed an update that installs a CallAdapter for every compiled call site. CallAdapter is basically a small object that stores the following:
The end result is that while compiled class init is a bit larger (needs to load adapters for all call sites), compiled method size has dropped substantially; in compiling bench_method_dispatch.rb, the two main tests went from 4000 and 3500 bytes of code down to 1500 and 1000 bytes (roughly). And simpler code means HotSpot has a better chance to optimize.
Here's the latest numbers for the bench_method_dispatch_only test, which just measures time to call a Ruby-implemented method a bunch of times:
I have also made a lot of progress on adapting the compiler to create stack-based methods when possible. Basically, this involved inspecting the code for anything that would require access to local variables outside the body of the call. Things like eval, closures, etc. At the moment it works well and passes all tests, but I know methods similar to gsub which modify $~ or $_ are not working right. It's disabled at the moment, pending more work, but here's the method dispatch numbers with stack-based method compilation enabled:
Oh, and for those who always need a fib fix, here's fib with both optimizations turned on:
I've committed an update that installs a CallAdapter for every compiled call site. CallAdapter is basically a small object that stores the following:
- method name
- method index
- call type (normal, functional, variable)
The end result is that while compiled class init is a bit larger (needs to load adapters for all call sites), compiled method size has dropped substantially; in compiling bench_method_dispatch.rb, the two main tests went from 4000 and 3500 bytes of code down to 1500 and 1000 bytes (roughly). And simpler code means HotSpot has a better chance to optimize.
Here's the latest numbers for the bench_method_dispatch_only test, which just measures time to call a Ruby-implemented method a bunch of times:
Test interpreted: 100k loops calling self's foo 100 timesAnd Ruby 1.8.6 for reference:
2.383000 0.000000 2.383000 ( 2.383000)
2.691000 0.000000 2.691000 ( 2.691000)
1.775000 0.000000 1.775000 ( 1.775000)
1.812000 0.000000 1.812000 ( 1.812000)
1.789000 0.000000 1.789000 ( 1.789000)
1.776000 0.000000 1.776000 ( 1.777000)
1.809000 0.000000 1.809000 ( 1.809000)
1.779000 0.000000 1.779000 ( 1.781000)
1.784000 0.000000 1.784000 ( 1.784000)
1.830000 0.000000 1.830000 ( 1.830000)
Test interpreted: 100k loops calling self's foo 100 timesNote that these are JIT numbers rather than fully precompiled numbers, so this is 100% real-world safe. Fully precompiled is just a bit faster, since there's no interpreted step or DefaultMethod wrapper to go through.
2.160000 0.000000 2.160000 ( 2.188087)
2.220000 0.010000 2.230000 ( 2.237414)
2.230000 0.010000 2.240000 ( 2.248185)
2.180000 0.010000 2.190000 ( 2.218540)
2.240000 0.010000 2.250000 ( 2.259535)
2.220000 0.010000 2.230000 ( 2.241170)
2.150000 0.010000 2.160000 ( 2.178414)
2.240000 0.010000 2.250000 ( 2.259772)
2.260000 0.000000 2.260000 ( 2.285141)
2.230000 0.010000 2.240000 ( 2.252396)
I have also made a lot of progress on adapting the compiler to create stack-based methods when possible. Basically, this involved inspecting the code for anything that would require access to local variables outside the body of the call. Things like eval, closures, etc. At the moment it works well and passes all tests, but I know methods similar to gsub which modify $~ or $_ are not working right. It's disabled at the moment, pending more work, but here's the method dispatch numbers with stack-based method compilation enabled:
Test interpreted: 100k loops calling self's foo 100 timesIt seems very promising work. I hope I'll be able to turn it on soon.
1.735000 0.000000 1.735000 ( 1.738000)
1.902000 0.000000 1.902000 ( 1.902000)
1.078000 0.000000 1.078000 ( 1.078000)
1.076000 0.000000 1.076000 ( 1.076000)
1.077000 0.000000 1.077000 ( 1.077000)
1.086000 0.000000 1.086000 ( 1.086000)
1.077000 0.000000 1.077000 ( 1.077000)
1.084000 0.000000 1.084000 ( 1.084000)
1.090000 0.000000 1.090000 ( 1.090000)
1.083000 0.000000 1.083000 ( 1.083000)
Oh, and for those who always need a fib fix, here's fib with both optimizations turned on:
~ $ jruby -J-server bench_fib_recursive.rbAnd MRI:
1.258000 0.000000 1.258000 ( 1.258000)
0.990000 0.000000 0.990000 ( 0.989000)
0.925000 0.000000 0.925000 ( 0.926000)
0.927000 0.000000 0.927000 ( 0.928000)
0.924000 0.000000 0.924000 ( 0.925000)
0.923000 0.000000 0.923000 ( 0.923000)
0.927000 0.000000 0.927000 ( 0.926000)
0.928000 0.000000 0.928000 ( 0.929000)
~ $ ruby bench_fib_recursive.rbThese numbers went down a bit because the call adapter is currently just generic code, and generic code that calls lots of different methods causes HotSpot to stumble a bit. The next step for the compiler is to generate custom call adapters for each call site that handle arity correctly (avoiding IRubyObject[] all the time) and call directly to the most-likely target methods.
1.760000 0.010000 1.770000 ( 1.775660)
1.760000 0.010000 1.770000 ( 1.776360)
1.760000 0.000000 1.760000 ( 1.778413)
1.760000 0.010000 1.770000 ( 1.776767)
1.760000 0.010000 1.770000 ( 1.777361)
1.760000 0.000000 1.760000 ( 1.782798)
1.770000 0.010000 1.780000 ( 1.794562)
1.760000 0.010000 1.770000 ( 1.777396)
Thursday, July 12, 2007
To Keyword Or Not To Keyword
One of the most attractive aspects of Ruby is the fact that it has relatively few sacred keywords. In most cases, things you'd expect to be keywords are actually methods, and you can wrap or hook their behavior and create amazing potential.
One perfect example of this is require. Because require is just a method, you can define your own version that wraps its behavior. This is exactly how RubyGems does its magic...rather than immediately calling the default require, it can modify load paths based on your installed gems, allowing for a dynamically-expanding load path and the pluggability we've all come to know and love.
But all such keyword-like methods are not so well behaved. Many methods make runtime changes that are otherwise impossible to do from normal Ruby code. Most of these are on Kernel. I propose that several of these methods should actually be keywords.
Update: Evan Phoenix of Rubinius (EngineYard), Wayne Kelly of Ruby.NET (Queensland University), and John Lam of IronRuby (Microsoft) have voiced their agreement on this interesting ruby-core mailing list thread. Have you shared your thoughts?
Justifying Keywords
There's a number of (in my opinion, very strong) justifications for this:
What do we gain from doing all this?
I can at least give a JRuby perspective. I expect others can give their perspectives.
In JRuby, we could greatly optimize method invocations if, for example, we knew we could just use Java's local variables (on Java's stack) rather than always heap-allocating a scoping structure. We could also avoid allocating a frame or binding when they are not needed, just allowing Java's call frame to be "enough" for us. We can already detect if there are closures in a given context, which helps us learn that a heap-allocated scope will be necessary, but we can't safely detect eval, block_given?, etc. As a result of these methods-that-would-be-keywords, we're forced to set up and tear down every method in the most expensive manner.
Other implementations&emdash;including Ruby 1.9/2.0 and Rubinius&emdash;would probably be able to make similar optimizations if we could calculate ahead of time whether these keyword operations would occur.
For what it's worth, I may go ahead and implement JRuby's compiler to treat these methods as keywords, only falling back on the "method" behavior when we detect in the rest of the system that the keyword has been overridden. But that situation is far from ideal...we'd like to see all implementations adopt this behavior and so benefit equally.
As an example, here's an early demonstration of the performance change in our old friend fib() when we can know ahead of time if any of these keywords are called (fib calls none of them). This example shows the performance today and the performance when we can safely just use Java local variables and scoping constructs. We could additionally omit heap-allocated frames for each call, giving a further boost.
I've included Ruby 1.8.6 to provide a reference value.
Current JRuby:
What do you think? Is it worth it?
One perfect example of this is require. Because require is just a method, you can define your own version that wraps its behavior. This is exactly how RubyGems does its magic...rather than immediately calling the default require, it can modify load paths based on your installed gems, allowing for a dynamically-expanding load path and the pluggability we've all come to know and love.
But all such keyword-like methods are not so well behaved. Many methods make runtime changes that are otherwise impossible to do from normal Ruby code. Most of these are on Kernel. I propose that several of these methods should actually be keywords.
Update: Evan Phoenix of Rubinius (EngineYard), Wayne Kelly of Ruby.NET (Queensland University), and John Lam of IronRuby (Microsoft) have voiced their agreement on this interesting ruby-core mailing list thread. Have you shared your thoughts?
Justifying Keywords
There's a number of (in my opinion, very strong) justifications for this:
- Many Kernel methods manipulate runtime state in ways no other methods can. For example: local_variables requires access to the caller's variable scope; block_given? requires access to the block/iter stacks (in MRI code); eval requires access to just about everything having to do with a call; and there are others, see below.
- Because many of these methods manipulate normally-inaccessible runtime state, it is not possible to implement them in Ruby code. Therefore, even if someone wanted to override them (the primary reason for them to be methods) they could not duplicate their behavior in the overridden version. Overriding only destroys their utility.
- These methods are exactly the ones that complicate optimizing Ruby in all implementations, including Ruby 1.9, Rubinius, JRuby, Ruby.NET, and others. They confound a compiler's efforts to optimize calls by always leaving open questions about the behavior of a method. Will it need access to a heap-allocated scope? Will it save off a binding or the current call frame? No way to know for sure, since they're methods.
- *eval - requires implicit access to the caller's binding
- block_given?/iterator? - requires access to block/iter information
- local_variables - requires access to caller's scope
- public/private/protected - requires access to current frame's visibility
- It manipulates runtime state in ways impossible from user-created code
- It can't be implemented in user-created code, and therefore could not reasonably be overridden or hooked to provide additional behavior
What do we gain from doing all this?
I can at least give a JRuby perspective. I expect others can give their perspectives.
In JRuby, we could greatly optimize method invocations if, for example, we knew we could just use Java's local variables (on Java's stack) rather than always heap-allocating a scoping structure. We could also avoid allocating a frame or binding when they are not needed, just allowing Java's call frame to be "enough" for us. We can already detect if there are closures in a given context, which helps us learn that a heap-allocated scope will be necessary, but we can't safely detect eval, block_given?, etc. As a result of these methods-that-would-be-keywords, we're forced to set up and tear down every method in the most expensive manner.
Other implementations&emdash;including Ruby 1.9/2.0 and Rubinius&emdash;would probably be able to make similar optimizations if we could calculate ahead of time whether these keyword operations would occur.
For what it's worth, I may go ahead and implement JRuby's compiler to treat these methods as keywords, only falling back on the "method" behavior when we detect in the rest of the system that the keyword has been overridden. But that situation is far from ideal...we'd like to see all implementations adopt this behavior and so benefit equally.
As an example, here's an early demonstration of the performance change in our old friend fib() when we can know ahead of time if any of these keywords are called (fib calls none of them). This example shows the performance today and the performance when we can safely just use Java local variables and scoping constructs. We could additionally omit heap-allocated frames for each call, giving a further boost.
I've included Ruby 1.8.6 to provide a reference value.
Current JRuby:
~ $ jruby -J-server bench_fib_recursive.rbUsing Java's local variables instead of a heap-allocated scope:
1.323000 0.000000 1.323000 ( 1.323000)
1.118000 0.000000 1.118000 ( 1.119000)
1.055000 0.000000 1.055000 ( 1.056000)
1.054000 0.000000 1.054000 ( 1.054000)
1.055000 0.000000 1.055000 ( 1.054000)
1.055000 0.000000 1.055000 ( 1.055000)
1.055000 0.000000 1.055000 ( 1.055000)
1.049000 0.000000 1.049000 ( 1.049000)
~ $ jruby -J-server bench_method_dispatch_only.rb
Test interpreted: 100k loops calling self's foo 100 times
3.901000 0.000000 3.901000 ( 3.901000)
4.468000 0.000000 4.468000 ( 4.468000)
2.446000 0.000000 2.446000 ( 2.446000)
2.400000 0.000000 2.400000 ( 2.400000)
2.423000 0.000000 2.423000 ( 2.423000)
2.397000 0.000000 2.397000 ( 2.397000)
2.399000 0.000000 2.399000 ( 2.399000)
2.401000 0.000000 2.401000 ( 2.401000)
2.427000 0.000000 2.427000 ( 2.428000)
2.403000 0.000000 2.403000 ( 2.403000)
~ $ jruby -J-server bench_fib_recursive.rbRuby 1.8.6:
2.360000 0.000000 2.360000 ( 2.360000)
0.818000 0.000000 0.818000 ( 0.818000)
0.775000 0.000000 0.775000 ( 0.775000)
0.773000 0.000000 0.773000 ( 0.773000)
0.799000 0.000000 0.799000 ( 0.799000)
0.771000 0.000000 0.771000 ( 0.771000)
0.776000 0.000000 0.776000 ( 0.776000)
0.770000 0.000000 0.770000 ( 0.769000)
~ $ jruby -J-server bench_method_dispatch_only.rb
Test interpreted: 100k loops calling self's foo 100 times
3.100000 0.000000 3.100000 ( 3.100000)
3.487000 0.000000 3.487000 ( 3.487000)
1.705000 0.000000 1.705000 ( 1.706000)
1.684000 0.000000 1.684000 ( 1.684000)
1.678000 0.000000 1.678000 ( 1.678000)
1.683000 0.000000 1.683000 ( 1.683000)
1.679000 0.000000 1.679000 ( 1.679000)
1.679000 0.000000 1.679000 ( 1.679000)
1.681000 0.000000 1.681000 ( 1.681000)
1.679000 0.000000 1.679000 ( 1.679000)
~ $ ruby bench_fib_recursive.rb
1.760000 0.010000 1.770000 ( 1.775304)
1.750000 0.000000 1.750000 ( 1.770101)
1.760000 0.010000 1.770000 ( 1.768833)
1.750000 0.010000 1.760000 ( 1.782908)
1.750000 0.010000 1.760000 ( 1.774193)
1.750000 0.000000 1.750000 ( 1.766951)
1.750000 0.010000 1.760000 ( 1.777814)
1.750000 0.010000 1.760000 ( 1.782449)
~ $ ruby bench_method_dispatch_only.rb
Test interpreted: 100k loops calling self's foo 100 times
2.240000 0.000000 2.240000 ( 2.268611)
2.160000 0.010000 2.170000 ( 2.187729)
2.280000 0.010000 2.290000 ( 2.292342)
2.210000 0.010000 2.220000 ( 2.250331)
2.190000 0.010000 2.200000 ( 2.210965)
2.230000 0.000000 2.230000 ( 2.260737)
2.240000 0.010000 2.250000 ( 2.256210)
2.150000 0.010000 2.160000 ( 2.173298)
2.250000 0.010000 2.260000 ( 2.271438)
2.160000 0.000000 2.160000 ( 2.183670)
What do you think? Is it worth it?
Subscribe to:
Posts (Atom)