Saturday, August 1, 2009

Return of Ruboto!

It's been a while since I was able to work on JRuby's Android support, but tonight I managed to finally circle back. And I've got something much more impressive working today: a real IRB application!



(And yes, this works just fine on the phone too)

It turned out to be incredibly easy to get this working. I'm not using any stinking plugin because they all seemed to just get in my way. So I generated a dummy application using the "android" tool, dropped the jruby jar in "libs", wrote up a quick interactive console, built and signed it, and that's all there was to it.

JRuby turns out to work very well for this sort of thing because we have an interpreter, so we can parse and execute code dynamically. Hooray for interpreted support!

I had to tweak a couple things to work around shortcomings in Android:
  • I had to edit the dx tool to allow up to a 1024M heap, since JRuby's jar has a ton of stuff in it
  • I had to catch and swallow an ArrayIndexOutOfBounds exception coming out of Dalvik's enum support. Bug!
This is of course a proof-of-concept. Writing full applications in Ruby isn't far behind, but we'll need a couple adjustments to JRuby to support it well:
  • Ability to run 100% precompiled with no runtime code generation
  • Strip out parser, interpreter, compiler, and bytecode-generation bits to shrink the jar
  • Tidy up the AOT compiler and wire it into the app build process
  • Generate some Ruby stub logic for the Android APIs, so they'll work well from Ruby
  • Strip down the weirder and wilder Ruby features (eval, etc) to allow fastest-possible execution
I know how to do all of this.

I've pushed ruboto-irb to Github so you can check it out and play with it. I welcome contributors :)

Ruboto lives!

Update: Good news, everyone!

First, the two bugs I've encountered have both been previously reported and are due to be fixed in an upcoming Android release. They are the enum bug and the reflection bug.

Second, someone going by the handle of "Psycho" reports in the comments that the next version of the Android Scripting Environment (ASE) will include JRuby support! Of course I'm interested in more than just scripting applications with JRuby...I'd like to be able to write applications using only Ruby code, so I'll continue working on this. But JRuby support seems to be coming in from all directions.

21 comments:

  1. This is great news! I could have used this today, when I was at a coffee shop without my laptop computer and wanted to try something out. I used ConnectBot to connect to my slice so I could run irb there, but the connection was slow and I wound up just giving up. From now on I won't have to worry about connectivity just to process a string.

    ReplyDelete
  2. Ben: Yeah, once I work out the logistics of doing so I'll release a "JRuby IRB" app to the market for anyone to download and install. The other milestones will come in time.

    ReplyDelete
  3. Have you looked at the Android Scripting Environment? It might have some reusable scaffolding (for the UI or adapter code, for example).

    Where was the bug in enum? Mea culpa.

    ReplyDelete
  4. This is awesome!

    You should definitely charge a few bucks for this app. Just releasing it is a huge enough service that most of us would feel guilty using it for free.

    ReplyDelete
  5. Paying a bit for your great job on getting Ruby on Android, sure. But that would mean cutting out a large part of us Europeans.

    Anyways, good job!

    ReplyDelete
  6. swankjesse, Bob, Elliott: I think I found the existing bug report about enums, and it matches what I'm seeing. So at least that one is reported. I'll see about reporting the reflection bug if it's not already there.

    ReplyDelete
  7. Psycho: I'm glad to hear ASE will support JRuby soon. I am more interested in being able to write applications entirely in Ruby than just being able to script applications. Is there any interest in this on the Android team?

    I would love to hear more about your work, so hopefully you'll post something to the JRuby dev list and we can talk about how to do even more with JRuby on Android.

    ReplyDelete
  8. Awesome! I really need to get myself one of these Android phones.

    ReplyDelete
  9. Very nice! I've followed the instructions and almost there I think.

    Last command is currently failing:

    adb -s HT93BKV00445 install -r bin/IRB-unsigned.apk
    635 KB/s (3453738 bytes in 5.305s)
    pkg: /data/local/tmp/IRB-unsigned.apk
    Failure [INSTALL_FAILED_DEXOPT]

    Any ideas? I've googled and it suggests a validation error or not enough storage. I've got over 1GB free on the card so that shouldn't be a problem?

    ReplyDelete
  10. Martin Sadler: Ahh interesting! Since you say there should be plenty of space, then it's likely the other alternative: failure to verify. Maybe you should contact me directly (or on JRuby mailing lists) but check:

    * That your device is running Android 1.5
    * That your device has enough RAM...not too many apps running when you try to install

    Barring that we could try a clean build of the jruby jar and then try some debugging flags to get more information about why it's failing to validate.

    ReplyDelete
  11. Sussed out the INSTALL_FAILED_DEXOPT ... turns out although I had plenty of SD memory I didn't have enough internal memory free.. so I uninstalled a few apps and all works just fine now :)

    Thanks Charles!

    ReplyDelete
  12. James, Martin: I've basically done no work to strip out stuff from jruby.jar that won't work on the phone, like native libraries, so I know we can shrink it quite a bit. And if we are able to move toward 100% interpreted or 100% precompiled execution modes, there's additional code we can rip out. We also need to look into how Anroid supports installable libraries, so that there could potentially be a single install of JRuby that several applications use. Then the apps can be as small as the ruby code they ship.

    ReplyDelete
  13. Cutting out the cruft would be a major first step. The app right now is like 9MB. Another thought was to have other apps use services (Intents?) provided by a main JRuby meta-app, so that not every app would need the jruby jar. I have no actual idea how that would happen (aside from some sort of drb/eval process, which feels unrobust).

    That this works at all, though, is quite slick.

    ReplyDelete
  14. James: wow, 9MB? Maybe you're doing it with the complete jar? With the normal jruby jar in my build the .apk is only 3.4MB.

    I think a good MB or more of that is probably native libs, and there's other stuff we can rip out that won't work on the device (or doesn't make sense on the device) like the Jay yacc debugger, readline, unix sockets, etc. If we could build a stub generator for Java classes you want to call from Ruby, we could eliminate the use of reflection. And of course if we simplify Java integration it would probably be half as much code. There's a lot of room to improve...

    ReplyDelete
  15. "James: wow, 9MB? Maybe you're doing it with the complete jar? With the normal jruby jar in my build the .apk is only 3.4MB."

    On local disk it shows as 3.4, but when I look on my G1, under Manage Applications, it shows it as 9.24. The next largest app there is the browser, at 4.4MB.

    So it expands. :)

    ReplyDelete
  16. when you say "Ability to run 100% precompiled with no runtime code generation" you don't mean uninterpreted, I assume?

    ReplyDelete
  17. Roger Pack: I mean being able to precompile everything so it all runs full speed. It's not possible (or at least not easy) to generate bytecode on the device itself, so JRuby's normal JIT operation does not work. Precompiling will allow compiling everything in advance.

    ReplyDelete
  18. Thanks for sharing such nice piece here. I would like to buy it as if I will have no laptop at outside so I can use this one. It will also entertain me. So Thanks for sharing its nice information here. It inspires me to buy it.

    ReplyDelete
  19. Is it possible to bundle a gem or ruby files with this app? If so, how can it be done?

    Regards,
    Oliver

    ReplyDelete
  20. okeiss: Yes, it's certainly possible! When packaging gems in any JRuby app, you generally want to unpack or bundle them, probably using the new "bundler" library. RubyGems itself does not take well to running inside an archive. Bundling the files should make them "just work" without issue.

    ReplyDelete