Sunday, September 2, 2007

Java Native Access + JRuby = True POSIX

I know, I know. It's got native code in it, and that's bad. It's using JNI, and that's bad. But damn, for the life of me I couldn't see enough negatives to using Java Native Access that would outweigh the incredible positives. For example, the new POSIX interface I just committed:

import com.sun.jna.Library;
public interface POSIX extends Library {
public int chmod(String filename, int mode);
public int chown(String filename, int owner, int group);
}

Given this library, it takes a whopping *one line of code* to wire up the standard C library on a given platform:

import com.sun.jna.Native
POSIX posix = (POSIX)Native.loadLibrary("c", POSIX.class);

That's. It.

So the idea behind JNA (jna.dev.java.net) is to use a foreign function interface library (libffi in this case) to wire up C libraries programmatically. This allows loading a library, inspecting its contents, and providing interfaces for those functions at runtime. They get bound to the appropriate places in the Java interface implementation, and JNA does some magic under the covers to handle converting types around. And so far, it appears to do an outstanding job.

With this code in JRuby, we now have fully complete chmod and chown functionality. Before, we had to either shell out for chmod or use Java 6 APIs that wouldn't allow setting group bits, and we always had to shell out for chown because there's no Java API for it. Now, both work *flawlessly*.

The potential of this is enormous. Any C function we haven't been able to emulate, we can just use directly. Symlinks, fcntl, file stats and tests, process spawning and control, on and on and on. And how's this for a wild idea: with a bit of trickery, we could actually wire up Ruby C extensions, just by providing JNI/JRuby-aware implementations of the Ruby C API functions.

So there it is. I went ahead and committed a trunk build of JNA, and I'm working with the JNA guys to get a release out. We'll have to figure out the platform-specific bits, probably by just shipping a bunch of pre-built libraries (not a big deal; the JNI portion is about 26k), but the result will be true POSIX functionality in JRuby...the "last mile" of compatibility will largely be solved.

Many thanks to Timothy Wall, who's been nursemaiding me through getting JNA working well, and who appears to be the primary driver of the JNA project right now. If you're interest in this stuff, check out JNA, start using it, make it popular. As far as I'm concerned this is how native access is supposed to be.

Update: My mistake, Wayne Meissner has also been pouring a metric buttload of effort into JNA. Credit where credit's due!

9 comments:

  1. What about Windows equivalents of chmod, chown, and so on?

    ReplyDelete
  2. JNA will allow any functions in any shared library/DLL to be called, chmod/chown/whatever functionality can be implemented for Windows by calling the functions in the Windows-specific DLLs liker user32.dll and kernel32.dll.

    ReplyDelete
  3. Very good indeed. That's moon landing! :-)

    ReplyDelete
  4. Wayne Meissner has been the other major driver on JNA, while working on gstreamer for Java. He did all the hard work of integrating libffi and getting a number of additional platforms working.

    ReplyDelete
  5. "And how's this for a wild idea: with a bit of trickery, we could actually wire up Ruby C extensions, just by providing JNI/JRuby-aware implementations of the Ruby C API functions."

    Intriguing, very intriguing.

    ReplyDelete
  6. What wicked idea :-) Very good indeed. Once Ruby C extension is interfaced that way we go on step closer to full Ruby compliance. We gain time to use the C extensions while someone is creating a pure Java alternative. We don't have to wait for the whole shebang to get ready. There are tons of good extensions that would take hundreds of man-hours to be flawlessly ported over to Java.

    Congratulations again!! Hope to see news about this soon.

    ReplyDelete
  7. Slava,

    MSVCRT includes the chmod, chown, etc functions already, so those won't be a problem.

    A more difficult task will be the Process module methods.

    ReplyDelete
  8. Excellent Job!

    As a financial analyst, I use Ruby for small scripting tasks, i.e. get some data from a website and plug it into an excel spreadsheet.
    JRuby looks really good. I am just not able to duplicate the win32ole functionality on JVM. I know it is possible, but not with my current skill level.
    Is there an easy way to use JNA to duplicate the win32ole?
    -thanks

    ReplyDelete
  9. Actually, the best way to implement Win32OLE would be to use a library like JACOB, which is a Java/COM bridge. That's what other projects use for OLE invocation from Java. Largely, all that would be needed is wiring up the basics of the win32ole Ruby library to JACOB, and you're done.

    ReplyDelete