299263 - Implement Java component loader

Status: RESOLVED WONTFIX

(Core Graveyard :: Java to XPCOM Bridge, enhancement)

Description

[jhp (no longer active)]

CC'ing some folks who might be interested.  Patch coming up.

Comment 1

[jhp (no longer active)]

Attachment: work in progress

The biggest concern when working on this was doing it in such a way that the
JVM would only get loaded  (for the most part) when the component was actually
created.  I had several ideas, but in the end I decided to define the module
and components in a simple XML file.  This would be easiest for a component
developer.

You can take a look at the XML file in the patch
(extensions/java/xpcom/tests/loader/contents.xml).  This just covers the
required elements.  The full 'spec' would look something like this:

<module>
  <name> Name of Module </name>
  <ctor> (optional) Module constructor method </ctor>
  <dtor> (optional) Module destructor method </dtor>
  <component>
    <description> Descriptive name for component </description>
    <cid> Class ID ("xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx") </cid>
    <contractid> Contract ID (i.e. "@foo.org/bar;1") </contractid>
    <ctor> Component contructor method (format: <package>.<class>#<method>,
i.e. "org.foo.bar.Bar#barCtor") </ctor>
    <regproc> (optional) Component registration method </regproc>
    <unregproc> (optional) Unregistration method </unregproc>
    <factorydtor> (optional) Factory destructor method </factorydtor>
    <ifacesproc> (optional) GetInterfaces method </ifacesproc>
    <langhelpproc> (optional) Language helper method </langhelpproc>
    <classinfo> (optional) ???? </classinfo>
    <flags> (optional) Class info flags </flags>
  </component>
  <component>
    ...
  </component>
  ...
</module>

I then load the XML file in |nsGenericJavaModule::LoadComponentManifest()|,
using some DOM document APIs.  I essentially enumerate through all the children
and copy in the information.  Not sure if I am doing this correctly, though.

What do you guys think of this approach?

Some other notes:
I made |nsGenericJavaFactory| inherit from |nsGenericFactory|, so I wouldn't
need to implement all the methods, many of which are the same between the two. 
To that end, I had to make the destructor and member variable for
|nsGenericFactory| be "protected" instead of "private".  Is that an issue?

The test in this patch works fine on Linux, but only if I have already set
LD_LIBRARY_PATH to "$JAVA_HOME/jre/lib/i386:$JAVA_HOME/jre/lib/i386/client". 
This is needed to that it will find 'libjvm.so' and some of the libraries it
links to.  I tried setting up the path from within the code, so that the user
would only need to set JAVA_HOME, but I couldn't get it to work.  I think I
read somewhere where bsmedberg mentioned that on Linux and Mac OS X you can't
change the library path after the process has already started.	Is there any
way around this limitation?

Comment 2

[jhp (no longer active)]

Another note: As a result of inheriting from |nsGenericFactory|, I also had to
link to 'xpcomglue_s'.  If I don't, then when loading the javaloader, I get an
unresolved symbol: 'nsGenericFactory::QueryInterface(nsID const&, void**)'.

Comment 3

[jhp (no longer active)]

Attachment: load JVM on registration

bsmedberg asked why I try to delay loading of JVM, since component will only
get registered when it is newly installed/upgraded, so it shouldn't be much of
a hit.	Well, here is a patch that does that.  I still needed a common entry
point (similar to |NSGetModule| C method), so I thought of making each Java
component have a class called |XPCOMModule|, which has a |getModule| method. 
Then in the code, I search the jar file for that class in order to get its
fully qualified name, and load it up.

Comment 4

[Darin Fisher]

It might be interesting to find out how blackconnect exposed NSGetModule.  Maybe
it has a different way that would be useful to mimic?

Comment 5

[jhp (no longer active)]

Blackconnect installed two files for a Java XPCOM component: a *.jar.info
information file, and a *.jar.comp file that contained the actual JAR file with
Java code.  The *.jar.info file (i.e.
http://lxr.mozilla.org/mozilla/source/java/xpcom/java/test/bcJavaSample.jar.info)
is just a simple text file that has some info about the XPCOM component.  So in
the end, it's very similar to my "context.xml" file from the first patch in this
bug.  I just didn't want to have two separate files in the 'components'
directory, so I put it inside the JAR file.  So basically, Blackconnect doesn't
start the JVM until someone tries to create an instance of the component.

Comment 6

[Darin Fisher]

Instead of XPCOMModule.getModule, you could also make it so that FooBar.jar
would be required to implement the following:

  public class FooBar {
    public static nsIModule NSGetModule(nsIComponentManager compMgr,
                                        nsIFile location) {
      ...
    }
  }

I suppose you could then search for "*/FooBar.class".  To me this is somewhat
nice because simple Java components could have only a single class (without
having to call the class XPCOMModule), and it reminds me of the way the "main"
function works for java programs.

Comment 7

[jhp (no longer active)]

But how would I know to search for "FooBar.class"?  Where would I get that text?
 That was the main reason I decided to make it a known class name such as
"XPCOMModule.class".  The best way I can think of is to make the name of the
component jar file specify the name of the main class;  therefore, "FooBar.jar"
component would contain "FooBar.class".

Comment 8

[Darin Fisher]

> ... therefore, "FooBar.jar" component would contain "FooBar.class".

Yup, that was exactly what I was suggesting.  I should have clarified that.

Comment 9

[jhp (no longer active)]

Attachment: patch w/ custom class loader, extension

* Created a custom class loader that extends |URLClassLoader|, which loads all of the Java components.  Looks for a class called "<JAR name>Module", and then calls the |NSGetModule| method on it.
* This patch still uses the old way of doing loaders, since I am looking to release a FF 1.5/2 extension.  I'll update to the new loader code once this code is working well on all 3 major platforms.
* Added some code in rules.mk for automating the Java compilation process.
* Since this loader is an extension, I had to add a hack to go back over some components directories in order to register any Java components there.  See |nsJavaComponentLoader::Observe|.
* On Windows and Linux, this code depends on the "JAVA_HOME" environment variable to find the installed JVM.  On Windows, I assume the best way to find the JVM is to look in the registry.  Not sure what to do on Linux.  There isn't really a set location that JVMs are installed, and I don't think I can rely on "JAVA_HOME" always being set.  Any ideas (for both platforms)?

Comment 10

[jhp (no longer active)]

This also brings up the issue of dynamically generating Java interfaces.  Since any extension can provide their own interface(s), JavaXPCOM won't know about them, since they are not in the MozillaInterfaces.jar file.  So I need a way to generate them on the fly.  That way, it would be possible (for example) to only ship the core frozen interfaces in a jar file, and every other interface would be generated on the fly.

Comment 11

[Mike Shaver (:shaver -- probably not reading bugmail closely)]

Why not generate all the interfaces dynamically, to save the download footprint of the MozillaInterfaces.jar?  That's my plan with XPCOM.NET, such as it is.  (It also makes it less likely that the Java interface gets out of sync with the XPT when we update.)

Comment 12

[jhp (no longer active)]

Yeah, I could generate all of the interfaces.  Now I just need to find out what the best way to do that is.

Comment 13

[Benjamin Smedberg]

Does Java have a varargs calling function? You could theoretically represent any XPCOM interface function call like so

interface nsISupports
{
  callMethod(string Name, /*arguments*/...);
}

Which could be used to represent arbitrary or even unknown interfaces... and the type conversion would be performed at runtime using nsIInterfaceInfo.

Comment 14

[Benjamin Smedberg]

I'm kinda assuming that it's hard or impossible to dynamically generate Java types, because I couldn't find any mention of it on the web. Perhaps I'm not looking hard enough.

Comment 15

[jhp (no longer active)]

varargs style arguments were added in Java 1.5, but I don't think Java is flexible enough to do what you are suggesting.  I'll see what I can find.

Comment 16

[jhp (no longer active)]

As far as I can tell, Rhino's ClassFileWriter class doesn't create interfaces.  I would have to add some code in order for it to handle interfaces.

Several people on the java.sun.com forums have mentioned using com.sun.tools.javac.Main.compile() to actually take a source file and compile it on the fly, but this method is only available in the tools.jar package, and that is only provided in the JDK, not the JRE.  So this isn't really an option.

Comment 17

[jhp (no longer active)]

In bug 279649, biesi suggested http://cglib.sourceforge.net/ for dynamically creating Java proxies for XPCOM objects.  I took a look at this project last night, and it can dynamically create both interfaces and classes.  However, it does more than we need.

One of the libraries it depends on, though, is ASM (http://asm.objectweb.org), which can be used to create interfaces and classes.  Plus, it's small, fast, and easy to use.  But it has a different license (http://asm.objectweb.org/license.html), which basically says 'do as you like, but let others know you are using this code'.  Not sure what Mozilla's policy is on using libraries with other licenses.

Comment 18

[Benjamin Smedberg]

You can email gerv to make sure, but all kinds of MIT/BSD licenses are compatible with the MPL tri-license.

Comment 19

[Antonio López-Cerón Vivo]

(In reply to comment #17)
> In bug 279649, biesi suggested http://cglib.sourceforge.net/ for dynamically
> creating Java proxies for XPCOM objects.  I took a look at this project last
> night, and it can dynamically create both interfaces and classes.  However, it
> does more than we need.
> 
> One of the libraries it depends on, though, is ASM (http://asm.objectweb.org),
> which can be used to create interfaces and classes.  Plus, it's small, fast,
> and easy to use.  But it has a different license
> (http://asm.objectweb.org/license.html), which basically says 'do as you like,
> but let others know you are using this code'.  Not sure what Mozilla's policy
> is on using libraries with other licenses.
> 

Another very interesting code that might be usefull is http://www.janino.net/
and its license could be suitable one. (http://www.janino.net/licensing.html)

Comment 20

[jhp (no longer active)]

Attachment: patch w/ dynamic interface generation

* This patch uses the asm library to dynamically generate Mozilla interfaces on the fly.  In order to do so, I had to create a chain of class loaders (see diagram at top of JavaComponentLoader.java).  That means that the bulk of MozillaInterfaces.jar doesn't need to ship with the Java Component Loader extension (only Mozilla.java and associated classes are necessary).  Also, the component loader can now make use of interfaces that weren't available at build time, such as custom interfaces that are shipped with extensions.
* This patch depends on the patches from bug 328901 and bug 333618.

Comment 21

[jhp (no longer active)]

Actually, this latest patch isn't ideal.  The Java component loader library makes use of some internal JavaXPCOM methods and globals (InitializeJavaGlobals(), gJavaKeywords, shortClass, etc), as part of the implementation of the InterfaceInfoManager and related classes..  This is OK for now, since the Java component loader library is linked against the JavaXPCOM library.  However, in the future, when FIrefox will be XULRunner based, the only way for this patch to work would be to expose those internal JavaXPCOM symbols, which isn't what we want.

Perhaps the best solution would be to move InterfaceInfoManager.java and related classes to JavaXPCOM.  They could be made part of the MozillaGlue.jar library (see bug 328901).  Or we could create a separate library called MozillaUtils.jar.  That way, MozillaGlue.jar would contain classes and interfaces associated with embedding and initialization, whereas MozillaUtils.jar would contain helper classes and classes that don't belong in MozillaGlue.jar.  How's that sound?

Comment 22

[Brian King [:kinger]]

Any status update on this?

Comment 23

[jhp (no longer active)]

I've started to update the code to account for all the changes that have taken place in JavaXPCOM.  Unfortunately, I can only work on this as time permits, so I can't say when it will be done.  But I want to get it out as soon as possible.

Comment 24

[Derrick Rice (:drice)]

Is this bug/upcoming feature the reason that new interfaces cannot be implemented in Java?

http://groups.google.com/group/mozilla.dev.tech.java/browse_thread/thread/1fc57ba670c887a0#

Comment 25

[Mike Kaply [:mkaply]]

JavaXPCOM is effectively dead. Should this bug be closed?

Comment 26

[Mike Hommey [:glandium]]

JavaXPCOM was removed in bug 648593.