Closed Bug 1865048 Opened 8 months ago Closed 7 months ago

Java/Kotlin cross-language support

Categories

(Webtools :: Searchfox, enhancement)

enhancement

Tracking

(firefox123 fixed)

RESOLVED FIXED
Tracking Status
firefox123 --- fixed

People

(Reporter: nicolas.guichard, Assigned: nicolas.guichard)

References

Details

Attachments

(4 files)

Steps to reproduce:

Now that Searchfox begins to have support for Java/Kotlin, it would be interesting to have some sort of support for the binding layer between C++ code and JVM code.

The Java/C++ bindings happen in a number of ways.

  • Calling C++ from Java:
    On the Java side we have a method prototype marked with the native modifier in every case.

On the C side by default Java will look for a function named Java_<escaped_class_path>_<escaped_method_name>[__<escaped_argument_specs>]. This default behavior is used a couple of times in Mozglue. (case A1)

This behavior can be overridden by calling a JNIEnv::RegisterNatives method.
This is done manually a couple of times in dom/media/systemservices/android_video_capture/video_capture_android.cc for instance. (case A2)

Otherwise the Java code uses a @WrapForJNI annotation that generates a template C++ class mozilla::java::<ClassName>::Natives<T> with wrapper functions and the right RegisterNatives call. The C++ class with the actual implementations instantiates the template to link the wrapper functions to the implementations and may live anywhere. (case A3)

  • Calling Java from C++:
    This requires calling a JNIEnv::Call[Static]<...>Method method. This is done manually a couple of times, for instance in dom/media/systemservices/android_video_capture/device_info_android.cc. (case B1)

The same @WrapForJNI annotation also generates wrappers around those JNIEnv calls in the C++ class mozilla::java::<ClassName>. (case B2)

For instance the EventDispatcher class uses @WrapForJNI both ways.
The Java class is https://dev.searchfox.org/mozilla-central/source/mobile/android/geckoview/src/main/java/org/mozilla/gecko/EventDispatcher.java

The hasGeckoListener method is marked native and @WrapForJNI. https://dev.searchfox.org/mozilla-central/source/mobile/android/geckoview/src/main/java/org/mozilla/gecko/EventDispatcher.java#207
A mozilla::java::EventDispatcher::Natives<Impl> template class is generated, with a wrapper function and the right structure for a RegisterNatives call. https://dev.searchfox.org/mozilla-central/source/__GENERATED__/widget/android/GeneratedJNI/EventDispatcherNatives.h#36
The mozilla::widget::EventDispatcher instanciates mozilla::java::EventDispatcher::Natives<mozilla::widget::EventDispatcher> and derives from it. https://dev.searchfox.org/mozilla-central/source/widget/android/EventDispatcher.h#29
Calling hasGeckoListener on the Java object calls hasGeckoListener on mozilla::widget::EventDispatcher.

The hasListener method is @WrapForJNI but implemented in Java. https://dev.searchfox.org/mozilla-central/source/mobile/android/geckoview/src/main/java/org/mozilla/gecko/EventDispatcher.java#473
A mozilla::java::EventDispatcher::hasListener wrapper function is generated. https://dev.searchfox.org/mozilla-central/source/__GENERATED__/widget/android/GeneratedJNI/EventDispatcherWrappers.h#170

Expected results:

There are a few possible ways to extract that data:

For case A1 (Java calling C based on function name) we could identify functions that match the Java_... name template (which are often annotated with JNICALL) and add the matching Java identifier to bindingSlots.
There are two issues for this:

  • the function name provides the Java identifier, as in some_package.SomeClass.SomeInnerClass.methodName, which lacks some information to be converted to the SCIP symbol some_package/SomeClass#SomeInnerClass#methodName().
  • overloads are disambiguated using the Java argument notation, eg some_package.SomeClass.overloadedMethod(I) vs some_package.SomeClass.overloadedMethod(Ljava.lang.String;) which again doesn't really map to SCIP overload resolution which yields something like some_package/SomeClass#overloadedMethod(). vs some_package/SomeClass#overloadedMethod(+1).

For case A2 (Java calling C from manual registration) there is no general solution since the registration happens at runtime. We could have some heuristics but it would be pretty fragile IMO.
The connection has to be made manually, either by using an external config file (like the ontology-mapping.toml described in https://bugzilla.mozilla.org/show_bug.cgi?id=1727789#c3) or internally by tagging the revelant functions. The mozsearch clang plugin could read attributes like [[mozsearch::bound_as("java", "some_package/SomeClass#SomeInnerClass#methodName().")]] for instance.

For case A3 (Java calling C++ through a @WrapForJNI-generated wrapper) we should be able to provide an automatic solution:

  • The @WrapForJNI annotation could add some annotation like [[mozsearch::binding(Impl::hasGeckoListener, "java", "org/mozilla/gecko/EventDispatcher#hasGeckoListener().")]] to the generated C++ code that gets picked up by the clang plugin.
  • Or without adding markers to the code we “parse” back what @WrapForJNI generated: for each class T in the namespace mozilla::java we look at the instanciation of T::Natives<Impl> (there should be only one) and mark Impl as bound the Java class described in T::name, then bind inner items as appropriate.

For case B1 (C/C++ calling Java manually) there may be no actual wrapper function on the C side to mark as a “binding”. In some cases we could parse strings that describe a class/method and provide a Go To action on those. For instance in device_info_android.cc:114 there is this call: mozilla::jni::GetClassRef(jni, "org/webrtc/videoengine/VideoCaptureDeviceInfoAndroid"); "org/webrtc/videoengine/VideoCaptureDeviceInfoAndroid" could propose going to the class definition.
Functions that are actual bindings could declare themselves with a [[mozsearch::binding_to(...)]] attribute or be listed manually in ontology-mapping.toml.

Case B2 (C++ calling Java though a @WrapForJNI wrapper) is like a simpler version of case A3, either:

  • @WrapForJNI annotates the generated C++ classes/methods with [[mozsearch::binding_to(...)]]; or
  • we could have a rule that binds each class T in the namespace mozilla::java to the Java class pointed to by T::name, and similar for inner classes/methods.

I think adding annotations into the C++ code itself (whether manually or automatically with @WrapForJNI) would generally be more robust, but working with rules that “reverse” the @WrapForJNI logic and marking the few other cases manually in a toml file shouldn't require any changes in the Firefox codebase.

Depends on: 1490144
See Also: → 1727789
Assignee: nobody → nicolas.guichard
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true

A first version is available for testing on dev.searchfox.org.
This aims to handle case A3 and B2 (everything @WrapForJNI-related) and provides a mean to handle case A2 and B1 by manually marking C++ items with __attributes__((annotate("[binding_to|bound_as]", "java", "[class|method|setter|getter|const]", "S_jvm_[SCIP symbol]"))).

A few examples:
Case B2 (C++ calling Java) for classes:

Case B2 (C++ calling Java) for methods:

Case B2 (C++ calling Java) for fields:

Case B2 (C++ calling Java) for consts:

Case A3 (Java calling C++) for classes:

Case A3 (Java calling C++) for methods:

The version on dev.searchfox.org now additionally supports case B1 (Java calling C++ through the Java_... naming scheme), for instance:

This is intended for Searchfox, to analyse and display the bindings
between Java and C++ code.

When the @WrapForJNI annotation is set on a method marked as native,
the code generator creates 3 files, for instance for EventDispatcher:

  • GeneratedJNIEventDispatcherWrappers.cpp
  • GeneratedJNI/EventDispatcherWrappers.h
  • GeneratedJNI/EventDispatcherNatives.h

The class that implements the member function bound from Java inherits
from EventDispatcher::Natives. That member function is defined in yet
another implementation file.

The mozsearch clang plugin only sees one single translation unit at a
time, so when it sees the definition of the actual bound method (and
when it emits the related data), it doesn't see the EventDispatcher
wrappers implementation anymore. At the moment it misses the name of
the wrapped Java class qualified class name.

This moves the name initialization from the wrappers impl file to the
header so that the mozsearch clang plugin can see the whole picture at
once when analysing the actual member functions' implementations.

The Java class members names were already initialized in the header so
this also makes things a bit more consistent in that regard.

This is a squash of commits 978bdb2d..549ce9bf from github:mozsearch/mozsearch.
Previously reviewed on https://github.com/mozsearch/mozsearch/pull/673.

Depends on D196795

Pushed by bugmail@asutherland.org:
https://hg.mozilla.org/integration/autoland/rev/c415fd20e342
Move the wrapped class name initialization to the header of JNI wrappers. r=owlish,geckoview-reviewers
https://hg.mozilla.org/integration/autoland/rev/b1abf39e79b3
Mozsearch indexer: add bindingSlots/slotOwner to C++ code bound from/to Java code r=asuth
Status: ASSIGNED → RESOLVED
Closed: 7 months ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: