1299187 - Tracking: stop exporting symbols such that 3rd-party software cannot use XPCOM effectively

Status: RESOLVED WORKSFORME

(Core :: XPCOM, defect)

Description

[Benjamin Smedberg]

Tracking bug for work so that we can technically and effectively prevent 3rd-party software from using binary XPCOM.

Comment 1

[Chris Peterson [:cpeterson]]

Restricting binary XPCOM will help avoid binary compatibility problems for extensions using ctypes in 64-bit Firefox.

Comment 2

[Jerry Krinock]

I'm not sure if this is related, but when my native macOS app attempts to establish interapplication to my old Firefox extension, via Binary XPCOM, Firefox 53 or 54 crashes immediately.  It works fine in 51 and, if I recall correctly, 52.

I have been working on a WebExtensions replacement for my old extension, but it is still awaiting un-implemented features in WebExtensions.  I was hoping that XPCOM would continue to be supported until WebExtensions were completed.

Or at least, do something more graceful than crash Firefox.  More details are here:

https://discourse.mozilla-community.org/t/some-of-xul-xulrunner-removed-in-firefox-53/13551

Comment 3

[Chris Peterson [:cpeterson]]

(In reply to Jerry Krinock from comment #2)
> I'm not sure if this is related, but when my native macOS app attempts to
> establish interapplication to my old Firefox extension, via Binary XPCOM,
> Firefox 53 or 54 crashes immediately.  It works fine in 51 and, if I recall
> correctly, 52.

Jerry, can you share a link to the crash report from your Firefox's about:crashes page? It would be useful to see where Firefox is crashing to see if we can work around that. Is your extension available on addons.mozilla.org?

Comment 4

[Jerry Krinock]

Thank you, Chris.  Here is a typical crash…

https://crash-stats.mozilla.com/report/index/bp-53dd8a0d-d0cc-4f6e-8217-8d0f02170210

As I said in that forum posting, it looks pretty straightforward – call to _NS_GetServiceManager goes unanswered.

My extension is not in the store.  It is shipped in the package with my native apps.  The easiest way to demo the crash is:

• Get on a Mac.
• Download and install one of my apps: https://sheepsystems.com/bookmacster.BookMacster.zip
• Run Firefox 53 or 54 Nightly.
• Launch BookMacster.  Close or cancel any windows that open.
• Click in the menu: BookMacster > Manage Browser Extensions.  A window will open.
• In the top "Syncing" section, row for "Firefox", click button "Install".
• Do the dance with Firefox to get the extension installed.  It should be Version 330.
• Return to the Manage Browser Extensions window in BookMacster and click "Test" in the row for Firefox.  This sends a message to the .dylib in my extensions.  Firefox will crash immediately.

The extension is in the Contents/Resources subfolder the BookMacster.app package.  Although I have the source code for its Binary XPCOM .dylib, the last time I tried to build it, a couple years ago, the build failed and I gave up.  I never did understand much about how all of that XUL stuff worked, and recall having to `lipo` together some libraries from different Firefox versions in some crazy way.  I am definitely all in favor of WebExtensions.  The only good thing about XUL is that it was simpler than NPAPI :))

Comment 5

[Chris Peterson [:cpeterson]]

(In reply to Jerry Krinock from comment #4)
> Thank you, Chris.  Here is a typical crash…
> 
> https://crash-stats.mozilla.com/report/index/bp-53dd8a0d-d0cc-4f6e-8217-
> 8d0f02170210
> 
> As I said in that forum posting, it looks pretty straightforward – call to
> _NS_GetServiceManager goes unanswered.

Benjamin, Jerry's stack trace is not symbolicated so it's hard to see what's going on, but is there a way to avoid these crashes, such as adding stub functions? I'm worried that we'll see similar crashes in other binary extensions that are surprised when their XUL APIs suddenly disappear in Firefox 53.

Comment 6

[Marco Castelluccio [:marco]]

(In reply to Chris Peterson [:cpeterson] from comment #5)
> (In reply to Jerry Krinock from comment #4)
> > Thank you, Chris.  Here is a typical crash…
> > 
> > https://crash-stats.mozilla.com/report/index/bp-53dd8a0d-d0cc-4f6e-8217-
> > 8d0f02170210
> > 
> > As I said in that forum posting, it looks pretty straightforward – call to
> > _NS_GetServiceManager goes unanswered.
> 
> Benjamin, Jerry's stack trace is not symbolicated so it's hard to see what's
> going on, but is there a way to avoid these crashes, such as adding stub
> functions? I'm worried that we'll see similar crashes in other binary
> extensions that are surprised when their XUL APIs suddenly disappear in
> Firefox 53.

Similar case in bug 1333486. I'm not sure how frequent these extensions are.
Perhaps we should evaluate when this goes to Beta and add the extensions that are causing most crashes to the blocklist.

Comment 7

[Benjamin Smedberg]

We should add broken DLLs to the blocklist (we can start that now). I believe that function shims would actually make things worse, not better, because right now most DLLs fail to load at all which typically doesn't crash. It's only if they are using dynamic function lookup that you'd see this kind of crash.

Comment 8

[Chris Peterson [:cpeterson]]

(In reply to Benjamin Smedberg [:bsmedberg] from comment #7)
> We should add broken DLLs to the blocklist (we can start that now).

Benjamin, who is responsible for blocking these DLLs that break Firefox when XPCOM is not exported? I'm not tracking or driving this; I just don't want known crashes to ride the trains.

Comment 9

[Benjamin Smedberg]

They should be nominated for tracking as "normal". Jorge is handling that particular one through an addon block.

Comment 10

[Andrei Vaida [:avaida]]

[Test Plan]:
https://wiki.mozilla.org/Symbol_Exporting

Comment 11

[Jerry Krinock]

I have pretty much completed code in my native app which will check to see if my old Firefox extension (which will crash the current version beta of Firefox 53) is installed, and if so replace it with my new WebExtension.  I plan for most users, who have automatic updates switched on, to have this running before Firefox 53 hits on April 18.  However, not all users have automatic updates switched on :(

My understanding is that when this bug is FIXED, Firefox will do that removal for me.  Is that correct?
If so, is there any chance that this fix would make it into the production release of Firefox 53?
Is there something I need to do to get my old extension on a so-called "blocklist" for Firefox 53+?

Comment 12

[Benjamin Smedberg]

The major work of this bug is already landed in Firefox 53. It will typically prevent DLLs that depend on XPCOM from loading, but does not actively block addons.

Comment 13

[Jerry Krinock]

Thank you, Benjamin.  Unless this work landed in the last two days, I don't think so.  I just did the following test:

• Run Firefox 52 in macOS 10.12.
• Doubleclick the .xpi file of my old Firefox extension featuring an XPCOM dynamic library.  Firefox 52 shows a popover asking for my OK to install.
• Click buttons to affirm the install and relaunch Firefox.
• Send a test message from my native app to my Firefox extension, which invokes the .dylib.  My extension responds as expected.
• Quit Firefox 52.
• Launch Firefox 53.0b8 (64-bit) Build ID 20170330120906.  My extension loads as usual, eeek.
• Send a test message from my native app to my Firefox extension, which invokes the .dylib.

RESULT: Firefox 53.0b8 (64-bit) Build ID 20170330120906 crashes.

Comment 14

[Robert Kaiser]

(In reply to Benjamin Smedberg [:bsmedberg] from comment #12)
> The major work of this bug is already landed in Firefox 53.

As this is the second or third time I heard of this causing crashes, has the bug(s) that that landed this work been flagged to release management so they know about the risk? I'm not watching all that stuff any more, but I still hope those that do have the knowledge that something can cause issues there (of course, over the long run, this will reduce crashes, which is great, but there seems to definitely be a short-term risk here).

Comment 15

[Liz Henry (:lizzard) (relman/hg->git project)]

Tracking for 53. Benjamin, do you know what's going on in comment 13?

Comment 16

[Liz Henry (:lizzard) (relman/hg->git project)]

Cristian, can you help here?

Comment 17

[Benjamin Smedberg]

Liz and Jerry, this is a high-level tracking bug, still open, and not a useful spot to comment. Bug 1306329 was for the Firefox 53 work, has extensive discussion and has had multiple QA signoff emails go to the release-drivers list.

Jerry, if you have a specific crash you are welcome to file it separately and have it block bug 1306329. I'd want to see crash IDs.

Comment 18

[Jerry Krinock]

Thank you, Benjamin.

Probably because I am not a Mozillan, I do not know how to and I presume do not have authorization to "have it block" Bug 1306329.  So I added a comment to that bug, mostly repeating what I've said in here.

I also gave a link to another, newer crash report, and explained how Firefox crashes will begin for actual users on April 18.

Comment 19

[Benjamin Smedberg]

Untracking this metabug.

Comment 20

[Benjamin Smedberg]

I'm going to close this out as a metabug. I still would really like us to fix https://bugzilla.mozilla.org/show_bug.cgi?id=1299159 but the major problem and active tracking of it is done.