Closed Bug 1306838 Opened 8 years ago Closed 7 years ago

Shutdownhangs after "MEGA" addon updates

Categories

(Firefox :: Extension Compatibility, defect)

49 Branch
defect
Not set
critical

Tracking

()

RESOLVED WONTFIX
Tracking Status
firefox49 --- affected
firefox50 --- affected
firefox51 --- affected
firefox52 - wontfix

People

(Reporter: philipp, Unassigned)

Details

Crash Data

crash stats data shows a large spike in various shutdownhang signatures starting yesterday  2016-09-30. 
if you look into correlation data it becomes apparent that this is related to the addon firefox@mega.co.nz with version 3.6.0, so the sudden spike is most likely caused by a recent addon update by them:

  shutdownhang | NtUserPostMessage | PostMessageW|EXCEPTION_BREAKPOINT (337 crashes)
     99% (334/337) vs.  17% (13941/80573) firefox@mega.co.nz
          0% (0/337) vs.   0% (12/80573) 1.0.3.1-signed
          0% (0/337) vs.   0% (1/80573) 3.3.2
          0% (0/337) vs.   0% (1/80573) 3.3.7
          0% (0/337) vs.   0% (1/80573) 3.4.2
          0% (0/337) vs.   0% (1/80573) 3.4.20
          0% (0/337) vs.   0% (1/80573) 3.4.25
          0% (0/337) vs.   0% (1/80573) 3.4.27
          0% (0/337) vs.   0% (1/80573) 3.4.29
          0% (0/337) vs.   0% (1/80573) 3.4.34
          0% (0/337) vs.   0% (2/80573) 3.4.36
          0% (0/337) vs.   0% (1/80573) 3.4.40
          0% (0/337) vs.   0% (1/80573) 3.4.42
          0% (0/337) vs.   0% (1/80573) 3.4.43
          0% (0/337) vs.   0% (3/80573) 3.4.45
          0% (0/337) vs.   0% (1/80573) 3.4.48
          0% (0/337) vs.   0% (1/80573) 3.5.2
          0% (0/337) vs.   0% (3/80573) 3.5.3
          0% (0/337) vs.   0% (3/80573) 3.5.4
          0% (0/337) vs.   0% (386/80573) 3.5.5
         99% (334/337) vs.  17% (13519/80573) 3.6.0

it's hard to quantify the exact impact of this, since the issues seems spread across hundreds of different signatures but the rate of shutdownhang crashes on release nearly doubled since yesterday.
"nearly doubled" should actually be "more than doubled":
https://crash-analysis.mozilla.com/rkaiser/crash-report-tools/longtermgraph/?fxrel-bcat
Version 3.6.0 released on 2016-09-29 introduced the use of a process-script (which is used to setup an nsIContentPolicy) and `<em:multiprocessCompatible>` was added to install.rdf

I think the required question here is if these hangs are somewhat related to bug 1051238 or something of the extension itself.
the issue is declining on its own again now (without there being a new version of the extension). maybe this has been a one-time shutdownhang after the addon update, or something on the website has changed in this regard.
if the trend continues like this for the next days, i'll close this bug again.
(In reply to [:philipp] from comment #3)
> or something on the website has changed in this regard.

That can't be - the whole website is embed into the extension, that's what is used for.
Version 3.6.1 got released, are the hangs still declining?
thanks for the hint about version 3.6.1. those hangs (recorded as crashes) are rising now again, see the graph at https://crash-stats.mozilla.com/signature/?product=Firefox&version=49.0.1&date=%3E%3D2016-06-01&signature=shutdownhang%20|%20NtUserPostMessage%20|%20PostMessageW#graphs as an example on how those signatures are developing.

maybe it's really the extension update process that triggers the hang once per installation...
Summary: Shutdownhangs with MEGA 3.6.0 addon → Shutdownhangs after "MEGA" addon updates
spiking again now with the 3.6.2 update
another way to visualise the impact: the spikes in the shutdownhang graph at https://crash-analysis.mozilla.com/rkaiser/crash-report-tools/longtermgraph/?fxrel-bcat on 2016-09-30, 2016-10-05 and 2016-10-12 are due to this bug.
(In reply to Diego Casorran [:diegocr] from comment #2)
> Version 3.6.0 released on 2016-09-29 introduced the use of a process-script
> (which is used to setup an nsIContentPolicy) and
> `<em:multiprocessCompatible>` was added to install.rdf
> 
> I think the required question here is if these hangs are somewhat related to
> bug 1051238 or something of the extension itself.

I can't remember is addon.multiprocessCompatible set to false if the add-on is not in the cohort? The add-on is not in the cohort for Firefox 49.
Bug 1051238 is independent from e10s being on or the flag being set.

If using frame scripts is the problem, one possible solution would be to use an embedded WebExtension that handles the content script. Another would be to decrease the amount of stuff being loaded there. I'll contact the developers about this.
(In reply to Jorge Villalobos [:jorgev] from comment #10)
> Bug 1051238 is independent from e10s being on or the flag being set.

My reading of the add-on is that it's only loaded when it addon.multiprocessCompatible is true. Thanks for reaching out to the author, that's the best next step.
(In reply to Jorge Villalobos [:jorgev] from comment #10)
> (...)
> I'll contact the developers about this.

I'm right here :-)

Anyway, i don't think embedding a WE is the best solution here. Actually, the whole extension will turn into a WE eventually, see Bug 1219940

The next extension version could remove the install.rdf flag, just to see if these hangs still happens.
(In reply to Andy McKay [:andym] from comment #11)
> My reading of the add-on is that it's only loaded when it
> addon.multiprocessCompatible is true. Thanks for reaching out to the author,
> that's the best next step.

The extension does support Fx 24+ so every change is made on a compatible way.

In 3.6.0, the process-script was introduced in a way that it's only enabled if `addon.multiprocessCompatible` is set, so that older browser versions that knows nothing about the flag will use the bootstrap.js's nsIContentPolicy, basically.
Component: Untriaged → Extension Compatibility
Version 3.6.3 was released on 2016-10-19 and it did removed the <em:multiprocessCompatible> flag.

However, that version did had a bug that caused the bootstrap's startup function to not execute fully (Specifically, NS_ERROR_FACTORY_EXISTS being throw from a misplaced Ci.nsIComponentRegistrar.registerFactory on the `if(addon.multiprocessCompatible){}` block - I.e. that is only executed switching the flag)

Version 3.6.4 was released on 2016-10-20 fixing that bug.

as far as i can see, the hangs have ceased, which is interesting :-/
(In reply to Diego Casorran [:diegocr] from comment #14)
> as far as i can see, the hangs have ceased, which is interesting :-/
yeah, a couple of more updates were released without any noticeable further spikes in shutdown-hangs so i'll go ahead mark the bug as WFM :-)
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → WORKSFORME
If this bug is related to the multiprocess, it can't be considered resolved until that indicator is activated again and it works correctly. The latest versions have been marked as not compatible with multiprocess.
Is this a Firefox bug or extension bug?
For the record, version 3.7.38 was released today.

That version does (re-)enables the multiprocessCompatible flag (and hence the process-script) as required per bug1331684
thank you for the heads-up - and indeed the shutdownhang rate is going through the roof again after the update :-/
https://crash-analysis.mozilla.com/release-mgmt/crash-report-tools/longtermgraph/?fxrel-bcat
Status: RESOLVED → REOPENED
Resolution: WORKSFORME → ---
[Tracking Requested - why for this release]:
this is not a good user experience - especially as the mega addon receives updates quite often. 
looking at 51.0.1, the number of shutdownhang crash reports went from 12k on 2017-03-01 to 27k yesterday - that rise will be mostly attributable to this bug.
Based on the hang impact, I'd say disable the multiprocessCompatible flag (to keep users with Mega in non-e10s mode).  The bug that is referenced is known, but there is not a performance fix currently. We are rolling back the unmarked add-ons that had been added to e10s in 51, based on the impact of this shim.
Flags: needinfo?(dcasorran)
Version 3.7.40 got released today.

That version does disables the process-script (while keeping the flag enabled), and the crashes doesn't seem to be rising again, but might be too soon to judge. Let's wait a couple of days and see.
Flags: needinfo?(dcasorran)
yes, there's no apparent spike in shutdownhangs this time.
Too late for firefox 52, mass-wontfix.
MEGA is now a WebExtension.
Status: REOPENED → RESOLVED
Closed: 8 years ago7 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.