929849 - Shumway breaks browser initialization on platforms lacking profiler support

Status: RESOLVED FIXED

(Firefox Graveyard :: Shumway, defect)

Description

[Steve Singer (:stevensn)]

Tabs in firefox on recent mozilla-central builds don't close.

Clicking on the 'X' icon on a tab doesn't do anything.  
Selecting any of the tab related options when I right click on a tab "Close Other Tabs", "Close Tabs To the Right" also has no effect.

I first noticed this on a mozilla-central build from around October 10'th but it could have been introduced as early as two weeks before.

I have tried a new profile and safe-mode same results.

I am building Debian Linux ppc32.

Comment 1

[Steve Singer (:stevensn)]

Actually I first noticed this in a build based on mozilla-central from around Oct 17 but I don't think I had done a build in the previous week or two.

Comment 2

[mjh563]

Steve, if you could find a regression range that would be helpful. At least, the Nightly build that it first happened in.

Comment 3

[Steve Singer (:stevensn)]

423b9c30c73d is good (the tabs close)
4e7d1e2c93a6 is bad (the tabs don't close)

The would correspond to sometime between nightly  20131017030201 and 20131018030206

Comment 4

[mjh563]

This is the pushlog for that range
https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=423b9c30c73d&tochange=4e7d1e2c93a6

Could you try narrowing it down further to find the first bad changeset?

Comment 5

[Marco Perez [:mpk]]

The regression was likely introduced with changeset 151237:1374158dab4b (https://hg.mozilla.org/mozilla-central/rev/1374158dab4b). 151236:1cb3ba2ab9c1 still worked okay (see https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=ae5e6dc9bce9&tochange=4e7d1e2c93a6).

The text for "Undo Close Tabs" in the tabs' context menu also disappeared (only the icon is shown).

Comment 6

[Tim Taubert [:ttaubert] (inactive)]

Steve, can you please provide some steps to reproduce the issue? That would be very helpful, it seems that no one else has been able to reproduce it so far.

Comment 7

[Steve Singer (:stevensn)]

All I need to do is open a tab on ANY site and I can't close a tab.
In my effected builds.  I am building on a platform with JS_ION disabled.

I am working on narrowing the regression range, but at 530 minute build times for most builds it takes time.
so far I have found that

0792108113b0 is bad
b082b86b2cac is good

I am building 1374158dab4b now.

Comment 8

[Marco Perez [:mpk]]

I can reproduce it on a FreeBSD box (amd64). Fortunately my build times are around 20min.
If there are some build options or patches that might help, I'd be more than happy to try them out.
Or if there's anything else I should log just let me know.

Comment 9

[Steve Singer (:stevensn)]

I can now confirm that 1374158dab4b is causing this
1cb3ba2ab9c1bg works fine but 1374158dab4b has the issue.

Comment 10

[Tim Taubert [:ttaubert] (inactive)]

I'm not sure how to proceed here. We will need some more help to reproduce this. It doesn't seem to be a common issue, bug 900910 is on Aurora and we surely would have had a lot more reports then.

I see that it is reproducible on FreeBSD and Linux 32bit, right? You also said that it's still occurring in safe-mode and with a new profile. Is there something else that you're doing here? Can you still reproduce this in the latest Nightly? Can you reproduce this on any Nightly or is this specific to your custom build config?

Comment 11

[Marco Perez [:mpk]]

Yes, current trunk builds still suffer from this bug. My setup is quite simple, using clang v3.3 and adding just the options --enable-alsa and --enable-pulseaudio (it won't build otherwise).

Adding --disable-optimize and --enable-debug didn't help. Neither did disabling the additional CPU cores.

But the 64bit Linux builds from https://nightly.mozilla.org work just fine on the same machine (running Ubuntu 12.04 LTS).

Comment 12

[Steve Singer (:stevensn)]

I am building with gcc with the following options

--disable-updater --prefix=/usr/local/mozilla27  --disable-ion   --host powerpc-unknown-linux-gnu --target powerpc-unknown-linux-gnu --disable-gold

Comment 13

[Tim Taubert [:ttaubert] (inactive)]

I just tried with the default build options on my Ubuntu 13.10 and I can't reproduce it. Do you see any errors in the error console that could maybe tell what's failing when you try to close a tab?

Comment 14

[Marco Perez [:mpk]]

The browser console does in fact emit "uncaught exception: 2147942487" when trying to close a tab.
This also occurs when opening the tab context menu.

According to the entries in xpcom/base/ErrorList.h 0x80070057 translates to NS_ERROR_ILLEGAL_VALUE.

Comment 15

[Tim Taubert [:ttaubert] (inactive)]

This is probably caused by SessionStore throwing NS_ERROR_INVALID_ARG (which has the same value) because of something trying to call the SessionStore API by passing an untracked window.

Comment 16

[Tim Taubert [:ttaubert] (inactive)]

I double-checked our startup code path and I have no idea how a window could end up untracked with the current code. We add a "domwindowopened" observer when "final-ui-startup" is fired here:

http://mxr.mozilla.org/mozilla-central/source/toolkit/xre/nsAppRunner.cpp#3942

Shortly afterwards the command line handler is run which then opens the first browser window:

http://mxr.mozilla.org/mozilla-central/source/toolkit/xre/nsAppRunner.cpp#3949

Are there any custom patches you have applied or is it just a vanilla Firefox build you're doing for fun?

Comment 17

[Marco Perez [:mpk]]

The only patch I'm applying is https://bug780531.bugzilla.mozilla.org/attachment.cgi?id=763100.
But even without it this bug's still triggered.

Comment 18

[Steve Singer (:stevensn)]

I don't have any patches applied either.

I wonder if this is being triggered by some flag/switch set by configure (like what that audio-float patch does).
There are a bunch of flags different on FreeBSD and PowerPC builds than the stock x86/x64 Linux builds.

Comment 19

[Tim Taubert [:ttaubert] (inactive)]

Building on FreeBSD now, let's see...

Comment 20

[Marco Perez [:mpk]]

Attachment: debugger log from gdb (might show the underlying cause)

This is a log from a short session where I
- started Firefox
- opened a second tab
- tried to close it
- shut down Firefox
This is using safe-mode but with updates and some checks disabled in an earlier session. This build has webrtc and optimizations disabled and debug enabled. No patches or other options applied.

Comment 21

[Tim Taubert [:ttaubert] (inactive)]

Ok, great. FreeBSD up and running and my build finished. I can reproduce it but have no clue yet what the problem is.

Comment 22

[Tim Taubert [:ttaubert] (inactive)]

Ok, this was pretty easy with a machine at hand. The startup code in nsBrowserGlue calls ShumwayUtils.init() in Nightly builds:

http://mxr.mozilla.org/mozilla-central/source/browser/components/nsBrowserGlue.js#475

Which causes:

************************************************************
* Call to xpconnect wrapped JSObject produced this error:  *
[Exception... "'[JavaScript Error: "Cc['@mozilla.org/tools/profiler;1'] is undefined" {file: "resource://shumway/ShumwayStreamConverter.jsm" line: 52}]' when calling method: [nsIObserver::observe]"  nsresult: "0x80570021 (NS_ERROR_XPC_JAVASCRIPT_ERROR_WITH_DETAILS)"  location: "native frame :: <unknown filename> :: <TOP_LEVEL> :: line 0"  data: yes]
************************************************************

And thus aborts the rest of the startup routine initializing SessionStore etc. This means we should fix Shumway to handle not having a profiler around but also that only Nightly builds are broken.

Comment 23

[Tim Taubert [:ttaubert] (inactive)]

Has been fixed by:

https://github.com/mozilla/shumway/issues/848

Comment 24

[Tim Taubert [:ttaubert] (inactive)]

Yury, how often do you uplift Shumway's extension code to mozilla-central? We're particularly interested in the issue mentioned above.

Comment 25

[Yury Delendik (:yury)]

Attachment: Delays profiler initialization in Shumway

(In reply to Tim Taubert [:ttaubert] from comment #24)
> Yury, how often do you uplift Shumway's extension code to mozilla-central?
> We're particularly interested in the issue mentioned above.

At least one time per Firefox release cycle, when it's stable enough. Here is cherry-picked fix for this issue.

Comment 26

[Till Schneidereit [:till]]

Comment on attachment 823968 [details] [diff] [review]
Delays profiler initialization in Shumway

Review of attachment 823968 [details] [diff] [review]:
-----------------------------------------------------------------

yup

Comment 27

[Tim Taubert [:ttaubert] (inactive)]

Awesome, thanks to you both!

Comment 28

[Steve Singer (:stevensn)]

I can confirm that with this patch applied the tabs close for me.
Tim, thanks for tracking this down.

Comment 29

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/integration/fx-team/rev/3b3f094b2c22

Comment 30

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/3b3f094b2c22

Comment 31

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/3b3f094b2c22