1037190 - [e10s] IDM CC add-on tracking meta bug for e10s

Status: RESOLVED INCOMPLETE

(Firefox :: Extension Compatibility, defect)

Description

[Achwaq Khalid]

User Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:33.0) Gecko/20100101 Firefox/33.0 (Beta/Release)
Build ID: 20140707030202

Comment 1

[Chris Peterson [:cpeterson]]

hi Dave, if you have any questions about add-on support for multiprocess Firefox (e10s), just drop by the #e10s IRC channel on irc.mozilla.org. MDN also has a good introduction:

https://developer.mozilla.org/en-US/Add-ons/Working_with_multiprocess_Firefox

Comment 2

[Charles Jones]

Hi Chris,

Thank you for asking

We started to examine e10s and stopped at the very beginning. Please advise where to get the answer to the following question

We have a binary C++ component, which among other things adds itself to "content-policy" category. For multiprocess Firefox ShouldLoad method of our component is run only for requests like "about: ..." when opening tabs. We don't receive any following real http requests. How add-ons in multiprocess Firefox should work (register, and run) with "content-policy" category? Still I cannot adopt examples from https://developer.mozilla.org/en-US/Add-ons/Working_with_multiprocess_Firefox for this task 

thank you

regards

Charles Jones
Software engineer

Tonec Inc.

Comment 3

[Chris Peterson [:cpeterson]]

Ally: can you help Charles with his content policy questions? Content policy seems to be a problem for other add-ons. Is this something we can patch with a shim or CPOW?

Comment 4

[Allison Naaktgeboren :ally]

There is already a shim for content policy. However shims only work for js code. Since the addon is binary, there is nothing we can do from our end. The addon will need to be re-written.

Charles, binary addons are going to have more difficulty maintaining compatibility in the future. We would like to strongly encourage to rewrite the addon into javascript to save you many future headaches.

Comment 5

[Charles Jones]

Hi Ally

thank you for your quick reply

I started to re-write some parts of add-on on JS, and indeed, it (content
policy) seemed to work for js code.

But from the link below when I search for "shims"

https://developer.mozilla.org/en-US/Add-ons/Working_with_multiprocess_Firefox

I've noticed that "the shims are only a temporary measure, and will be
removed eventually"

If that is true, do you plan to delete the "content policy" shim soon?


(In reply to :ally Allison Naaktgeboren from comment #4)
> There is already a shim for content policy. However shims only work for js
> code. Since the addon is binary, there is nothing we can do from our end.
> The addon will need to be re-written.

Comment 6

[Allison Naaktgeboren :ally]

Charles,
We'll start thinking about removing shims once e10s is all the way on release and fully settled. So, not anytime soon.

Comment 7

[Charles Jones]

> We'll start thinking about removing shims once e10s is all the way on
> release and fully settled. So, not anytime soon.

Ally,

There is some kind of misunderstanding here.

I understand that you plan to get rid of shims not anytime soon, but the meaning of the question is like the following

Once you delete shims, will all extensions that use "content-policy" stop working for e10s?

I'm asking because you write that the extensions that use "content-policy" work only because you made a shim for it.

In other words, what is the sense for us to use (rely on) shims now when you plan to delete them "when e10s is all the way on release and fully settled"?

Comment 8

[Allison Naaktgeboren :ally]

Charles,
Shims are a way to ease the transition for addon authors to multiprocess. Shims were written so certain things 'would just magically work' in js addons. 

Yes, eventually the addons that rely on shims will have to update their code.

In your case, it might make more sense to write the new code so it doesn't rely on shims, and use the message manager to communicate between addon code in the content process and addon code in the chrome process about the content-policy.

Comment 9

[Charles Jones]

Ally,
Is it possible to turn off all shims in e10s? Maybe you have some option in about:config? Since all shims will go away in the future, we may need to test our new add-on without shims right away.

Comment 10

[Chris Peterson [:cpeterson]]

Charles, for now, there is a test feature (available in Firefox Nightly) to enable e10s without enabling the shims:

1. Disable e10s, either unchecking the "Preferences > Enable E10S (multi-process)" checkbox or resetting the about:config pref "browser.tabs.remote.autostart" to false.
2. Restart Nightly.
3. Select the "File > Open e10s Window" menu item.
4. This will open a new window that is in an e10s content process with no shims enabled. You can verify the window is running in e10s mode by checking for underlined tab titles.

We really should add an about:config pref to make this more obvious. :)

Comment 11

[Charles Jones]

May I ask another question?

Do you have any tutorial or recommendations on finding window handles for multiprocess Firefox?

On the page below 

https://developer.mozilla.org/en-US/Add-ons/Code_snippets/Finding_Window_Handles

There is a code that does not work in e10s (I tried running it in Chrome process)

var baseWindow = window.QueryInterface(Components.interfaces.nsIInterfaceRequestor) .getInterface(Components.interfaces.nsIWebNavigation) .QueryInterface(Components.interfaces.nsIDocShellTreeItem) .treeOwner .QueryInterface(Components.interfaces.nsIInterfaceRequestor) .getInterface(Components.interfaces.nsIXULWindow) .docShell .QueryInterface(Components.interfaces.nsIBaseWindow);

How can I get HWND of Firefox window?

Comment 12

[Chris Peterson [:cpeterson]]

Charles: what does your addon do with the HWND? Maybe there is high level API to refer to the window that doesn't require HWNDs?

Ally: can addons still fetch HWNDs in e10s?


We should probably update that Finding_Window_Handles MDN page..

Comment 13

[Allison Naaktgeboren :ally]

Chris: Yes, we should, given the deprecated api. Have you notified wbamberg about it?

Charles: Why are you trying to get an HWND into JS? 

I & the owner of windows specific code have tested your snippet and it works for us in the firefox browser console. What version of windows are you testing on?

Could you please re-test that snippet in the browser console in the latest Nightly? The parent/chrome process is expected to have access. The child process will not.

Comment 14

[Allison Naaktgeboren :ally]

The api may not be depreciated, however it is not listed here https://developer.mozilla.org/en-US/docs/Mozilla/Tech/XPCOM/Reference/Interface 

but is listed here https://developer.mozilla.org/en-US/docs/Interfaces ?

https://developer.mozilla.org/en-US/docs/XPCOM_Interface_Reference/nsIBaseWindow  404s 

Chris, I think you might really want to get the MDN folks on this. MDN contradicts itself about the apis existence and no longer have the page on the interface at all.

Comment 15

[Charles Jones]

Thank you for all your replies.

We need HWND for many cases, since we have common universal DLLs for all major browsers: IE, Firefox, Chrome, which receive HWND and other information

For example, when customers select a group of links to a specific site in a browser (according to user settings in IDM), we draw a button above the selection that allows downloading these links with IDM, etc.

I've tested that code several times. It does not work when I pass document.defaultView as input "window" parameter. This part of code was developed when you had different HWNDs for different tabs, and it still works in current stable version of Firefox (not multiprocess one)

At this time I made it in-dependable from the document (without input parameter).

var wm = Components.classes["@mozilla.org/appshell/window-mediator;1"].getService(Components.interfaces.nsIWindowMediator);
var recentWindow = wm.getMostRecentWindow("navigator:browser");
var baseWindow = recentWindow.QueryInterface(Components.interfaces.nsIInterfaceRequestor) .getInterface(Components.interfaces.nsIWebNavigation) .QueryInterface(Components.interfaces.nsIDocShellTreeItem) .treeOwner .QueryInterface(Components.interfaces.nsIInterfaceRequestor) .getInterface(Components.interfaces.nsIXULWindow) .docShell .QueryInterface(Components.interfaces.nsIBaseWindow);

And this code works. Thus, one problem is solved.

Comment 16

[Giorgio Maone [:ma1]]

[e10s tracking info]

Assignee: Giorgio Maone

Link to add-on: http://www.internetdownloadmanager.com/download.html

Contact info for add-on: [e-mail] (cc/ed in this bug)

Add-on ID: [e-mail]

How well does it work?: 100%

Steps to reproduce working features:
* Install the Internet Download Manager program from http://www.internetdownloadmanager.com/download.html
and restart Firefox, accepting the 3rd party add-on installation
* Right click on a link in a web page and select either "Download with IDM" or "Download all links with IDM": in both cases, the link(s) should be passed to Internet Download Manager to be downloaded

Any obvious performance problems? none

SDK-based: No, legacy XUL add-on, contains MessageManager-specific code to handle e10s

Chromium version: yes

Comment 17

[Charles Jones]

I'm afraid that our extension is not yet completely ready for multiprocess Firefox. For example, we have many "unsafe CPOW usage" warnings in Browser Console. Also, sometimes, Firefox shows a message that the extension slows down Firefox browser and suggests turning it off. We are working on the extension right now.

May you help us by answering the following question below?

Suppose (in mulitprocess Firefox) that some script in content process (not frame script) has created XPCOM component. Then a global object or variable is created in XPCOM component. How can we access this object/variable from frame-script (or from process script:  https://developer.mozilla.org/en-US/Firefox/Multiprocess_Firefox/Message_Manager/Process_scripts) ?

Comment 18

[Giorgio Maone [:ma1]]

(In reply to Charles Jones from comment #17)

> Suppose (in mulitprocess Firefox) that some script in content process (not
> frame script) has created XPCOM component. Then a global object or variable
> is created in XPCOM component. How can we access this object/variable from
> frame-script (or from process script: 
> https://developer.mozilla.org/en-US/Firefox/Multiprocess_Firefox/
> Message_Manager/Process_scripts) ?

If your XPCOM component is a service (i.e. a singleton) you just need to expose your variable as a property and grab the singleton instance, which will be unique for that process, as usual: 

var myService = Component.classes[classId].getService(interfaceId);
var myVar = service.myVariable;

Comment 19

[Charles Jones]

(In reply to Giorgio Maone from comment #18)
> 
> If your XPCOM component is a service (i.e. a singleton) you just need to
> expose your variable as a property and grab the singleton instance, which
> will be unique for that process, as usual: 
> 
> var myService = Component.classes[classId].getService(interfaceId);
> var myVar = service.myVariable;

Thank you for reply

We tried that in frame-script, but it did not work, and browser Console showed "testClassID is undefined" error message.

http://www.internetdownloadmanager.com/tmp/cc_error.png

testClassID is declared in extension manifest and the same code can run in chrome script without errors

Comment 20

[Giorgio Maone [:ma1]]

(In reply to Charles Jones from comment #19)


> testClassID is declared in extension manifest and the same code can run in
> chrome script without errors

When you wrote "some code has created a XPCOM component" I believed you were registering it at runtime.
XPCOM registration metadata from chrome.manifest is ignored outside the parent process.

If you need your component to be accessible in content processes as well, you must register it at runtime just like bootstrapped and SDK add-ons should, by using the nsIComponentRegistrar service like in the following example from Adblock Plus:
https://hg.adblockplus.org/adblockplus/file/0d76ad7eb80b/lib/contentPolicy.js#l365

Comment 21

[Giorgio Maone [:ma1]]

(In reply to Charles Jones from comment #19)

Charles, I've looked a bit more into your code, and I've noticed some problematic patterns there.

They can mostly be summarized as 1:1 mappings between DOM event handlers / observers / callbacks in the content process and synchronous messages exchanged with the parent process.

This is especially problematic with your content policy implementation in a frame script, which likely generates a lot of IPC traffic by using sendRpcMessage() in every single shouldLoad() callback, and which apparently throws a lot, filling the browser console with lines like this:

NS_NOINTERFACE: Component returned failure code: 0x80004002 (NS_NOINTERFACE) [nsIInterfaceRequestor.getInterface]1 frame-script.js:37:0

This is not better (probably worse, actually) than relying on the default content policy shim.

What I would suggest is at least filtering the shouldLoad() callbacks you're actually interested in on the content process side, or even better, if possible (depending on what you're trying to accomplish), replacing your content policy with a "http-on-start-request" observer in the parent process.

Comment 22

[Charles Jones]

Yes, thank you, Giorgio.

I understand what parts of code we need to move to content process side. And now we're re-developing the extension from scratch. That's why I wrote that the extension was not yet ready for multiprocess Firefox.

In previous e-mail you suggested registering XPCOM component at runtime.

Our interfaces are in .xpt files and they were registered via chrome.manifest. Can we register them at runtime? And how can it be done?

Comment 23

[Giorgio Maone [:ma1]]

(In reply to Charles Jones from comment #22)

> Our interfaces are in .xpt files and they were registered via
> chrome.manifest. Can we register them at runtime? And how can it be done?

If you're implementing your XPCOM components in JavaScript (if you're implementing them with native code, instead, notice that they will break very soon anyway) and you don't need your component's properties and methods to be accessed from native code, you don't need any interface.

Just expose a wrappedJSObject property, https://developer.mozilla.org/en/docs/wrappedJSObject#_Meet_wrappedJSObject_ 

Also, these days there are few compelling cases for implementing your own XPCOM component: either registering something in the CategoryManager (such as a nsIContentPolicy) or replacing some native component/service provided by the platform. 
Otherwise, if you just need to implement an interface to be passed around to platform's services, such as a nsIObserver or a nsIWebProgressListener, you're probably better off with modules,

https://developer.mozilla.org/en-US/docs/Mozilla/JavaScript_code_modules/Using

Notice that if you're rewriting your add-on using the Add-on SDK, it features its own module system.

Comment 24

[Giorgio Maone [:ma1]]

Charles, you may have noticed by now(In reply to Charles Jones from comment #22)

> now we're re-developing the extension from scratch. 

Please consider using the WebExtensions API:

https://wiki.mozilla.org/WebExtensions
https://wiki.mozilla.org/WebExtensions/FAQ

More specifically, since you've got a Chrome version of your extension, could you share with us its limitations, if any, compared to the Firefox versions and which APIs, among those still missing, we should prioritize to help you reuse your Chrome extensions' code as much as possible, rather then rewriting the whole?

Comment 25

[Charles Jones]

I do develop Firefox extensions mainly. I don't know much about Chrome extension programming because we have another developer for Chrome. Also he does not know much about Firefox. Since we have different developers, I'm afraid we don't know all details and problems of each API. But I can tell just one thing that Chrome developer told me. Chrome APIs have a drawback when we cannot read file data, only http headers, while Firefox has nsIStreamListener for this purpose.

Comment 26

[Andy McKay]

It looks like it calls:

chrome.webRequest.onBeforeSendHeaders (implemented)
chrome.runtime.sendMessage (implemented)

Comment 27

[Andy McKay]

Oh wait, this page says to ignore the extension on Chromes site.

http://www.internetdownloadmanager.com/register/new_faq/chrome_extension.html

So comment 26 was probably wrong.

Comment 28

[:shell escalante]

potentially related to flashgot webextension solution...re-engage after mid-March 18 after investigation moved forward as likely similar needs.  shell remind giorgio in few weeks.

Comment 29

[Firefox Bug Husbandry Bot]

With Firefox 57 only WebExtensions are permitted and are, by default, e10s compatible.