1197286 - EngineeringMode abstraction for custom APIs

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: GonkIntegration, defect)

Description

[Julian Hector [:tedd] [:jhector]]

Modifications inside Gecko to implement a custom API for testing purpose etc. such as the JRDExtension[1] prevent Gecko from being updated.

:huseby and I have been working on the abstraction of the nsIEngineeringMode interface to allow the implementation of a custom API.

nsIEngineeringMode serves as a plugin host and loads plugins (shared libraries, provided by partners) from a given directory (using dlopen()) and calls functions from that library (stable API). Each plugin can then implement their own custom API.

So when Gecko is updated, the interface implementation will be updated as well, but the API between the interface and the plugins is stable which allows the plugins to continue working without updating them.

An app with the 'engineering-mode' permission can then use the 'getValue' and 'setValue' functions to access the custom API.

For example:
navigator.engineeringMode.getValue("JrdSrv:Common:BatteryTemp");

will call the getValue function from the nsIEngineeringMode interface, which in return uses the supplied string ("JrdSrv:Common:BatteryTemp") to route the call to the plugin that registered for the 'JrdSrv' namespace.

When the appropriate plugin is called, it will be given the string to allow further routing inside the plugin. In the above example, a plugin would read a file from the file system and return the content (/sys/class/power_supply/battery/temp)

In case of the JRDExtension, 80% of the functionality can probably be implemented with the current state, because most of the functionality doesn't rely on Gecko (in the sense of using interfaces of Gecko)

This is an Idea/PoC/WIP on how the problem could be solved.

[1] https://github.com/t2m-foxfone/quic_lf_b2g_mozilla_gecko/tree/foxfone-one/dom/jrd

Comment 1

[Julian Hector [:tedd] [:jhector]]

Attachment: nsIEngineeringMode implementation

Current nsIEngineeringMode implementation, serving as a plugin host.

Comment 2

[Julian Hector [:tedd] [:jhector]]

Attachment: PoC EngineeringMode plugin

Comment 3

[Julian Hector [:tedd] [:jhector]]

Attachment: Patch for gonk-misc to preserve plugins directory

Comment 4

[Julian Hector [:tedd] [:jhector]]

Attachment: PoC app that uses EngineeringMode

Comment 5

[Julian Hector [:tedd] [:jhector]]

The code for everything so far can be found on my github:

Gecko: https://github.com/jhector/gecko-dev/tree/engineeringmode-abstraction
Gaia: https://github.com/jhector/gaia/tree/engineeringmode
Gonk-misc: https://github.com/jhector/gonk-misc/tree/engineeringmode
Sample plugin: https://github.com/jhector/platform_external_jrdplugin

Comment 6

[Thomas Zimmermann [:tzimmermann] [:tdz]]

Comment on attachment 8651147 [details]
PoC EngineeringMode plugin

Thanks Julien,

I left some comments in the Github tree. Some more general notes:

  * The header file for the engineering mode should live in a separate repository that is shared between Gecko and each plugin. I'm in the process of setting up |libhardware_moz| where all our system headers can be stored. (Bug 1196148)

  * There should be a few comments on the API in the header.

Comment 7

[Thomas Zimmermann [:tzimmermann] [:tdz]]

  * I wonder if we could move the plugins out of Gecko's address space.

Comment 8

[Thomas Zimmermann [:tzimmermann] [:tdz]]

In bug 1207193, I created the repository for all kinds of hardware-related C/C++ interfaces; as mentioned in comment 6. The interfaces for engineering mode can liver there as well.

Comment 9

[Julian Hector [:tedd] [:jhector]]

Thanks Thomas, I addressed some of your comments, in regards to the plugin path, is the preferred way to set a define via command line of the compiler? (which in return can be set by the build system, I guess)

As for the separate repo for the interfaces, how can I include the header file in the XPCOM implementation part (inside Gecko)?

Comment 10

[Thomas Zimmermann [:tzimmermann] [:tdz]]

Hi

I meanwhile added 'platform_hardware_libhardware_moz' [1] to our Github repositories. It should be pulled into the checkouts as soon as bug 1207193 gets the r+. You can add the engineering-mode header there as well.

(In reply to Julian Hector [:tedd] [:jhector] from comment #9)
> Thanks Thomas, I addressed some of your comments, in regards to the plugin
> path, is the preferred way to set a define via command line of the compiler?
> (which in return can be set by the build system, I guess)
> 
> As for the separate repo for the interfaces, how can I include the header
> file in the XPCOM implementation part (inside Gecko)?

You can add the search path in the moz.build file in Gecko as shown in [2]. Theres also a DEFINE directive that allows you to define preprocessor constants.

[1] https://github.com/mozilla-b2g/platform_hardware_libhardware_moz
[2] https://dxr.mozilla.org/mozilla-central/source/dom/media/moz.build#281

Comment 11

[Thomas Zimmermann [:tzimmermann] [:tdz]]

Oh, wait! I just realized the LOCAL_INCLUDES might not work with directories *outside* of the Gecko tree. In this case, you can add the include search path as illustrated at [1].

[1] https://dxr.mozilla.org/mozilla-central/source/configure.in#220

Comment 12

[Thomas Zimmermann [:tzimmermann] [:tdz]]

Line 292 [1] is probably the right location.

[1] https://dxr.mozilla.org/mozilla-central/source/configure.in#292

Comment 13

[Julian Hector [:tedd] [:jhector]]

Ah I see, thanks a lot Thomas.

Comment 14

[Thomas Zimmermann [:tzimmermann] [:tdz]]

(In reply to Julian Hector [:tedd] [:jhector] from comment #9)

> As for the separate repo for the interfaces, how can I include the header
> file in the XPCOM implementation part (inside Gecko)?

But actually I think you shouldn't include this header from Gecko at all. This should be a separate system service from day 1 because

 * external code shouldn't run in B2G's address space if possible for security,

 * B2G won't offer the necessary privileges as soon as we don't run it as root or switch on SELinux, and

 * a separate service is a cleaner system design.

Comment 15

[Julian Hector [:tedd] [:jhector]]

(In reply to Thomas Zimmermann [:tzimmermann] [:tdz] from comment #14)
> (In reply to Julian Hector [:tedd] [:jhector] from comment #9)
> But actually I think you shouldn't include this header from Gecko at all.
> This should be a separate system service from day 1 because
> 
>  * external code shouldn't run in B2G's address space if possible for
> security,
> 
>  * B2G won't offer the necessary privileges as soon as we don't run it as
> root or switch on SELinux, and
> 
>  * a separate service is a cleaner system design.

I agree that we should separate most of the code, but for example, when the plugin is providing a pointer to the interface, then the plugin needs to know the struct layout as well as Gecko, otherwise Gecko wouldn't be able to call functions from the plugin or vise versa.

So the header file containing the struct definitions etc, needs to be included in both Gecko and the plugin.

Right now the system is designed such that a shared library is loaded and mapped into memory by the XPCOM interface (plugin host), which means that both are essentially in the same address space.

Now if we were to separate that we would have to create a plugin host which essentially is a daemon that uses IPC to communicate with (if we want to support multiple plugins, i.e. plugins supplied by the partners).

Or have Gecko supply a Unix socket (file on the file system) which can be used for the IPC by multiple daemons (technically the 'plugins' from the partners).

The reason why initially the library approach was chosen, was to allow the partners to implement their customization without having to touch Gecko code (as it was done with JRD[1])

[1] https://github.com/t2m-foxfone/quic_lf_b2g_mozilla_gecko/tree/foxfone-one/dom/jrd