749792 - Web runtime in-process-plugins fail on some sites

Status: RESOLVED WONTFIX

(Firefox Graveyard :: Webapp Runtime, defect)

Description

[Jason Smith [:jsmith]]

Steps:

1. Install https://marketplace.mozilla.org/en-US/app/multiplayer-piano/
2. Launch the application

Expected:

The piano should load, be able to play sound and allow interaction. See http://www.multiplayerpiano.com/ for how it actually works.

Actual:

The piano never loads in the runtime. No sound plays in the runtime. This is different behavior than desktop firefox and chrome.

Needs investigation. Looks like this application is using web sockets. The failure is happening during initial load of the piano. Need to reverse engineer why this is happening. This bug tracks this investigation and fixes required.

Comment 1

[Ed Lee :Mardak]

Seems like it's not an audio issue as Beatlab seems to play music just fine:
https://marketplace.mozilla.org/en-US/app/beatlab/

Comment 2

[Jason Smith [:jsmith]]

The problem in this bug happens on load of the page. I think it might be happening in this JavaScript file: http://www.multiplayerpiano.com/midibridge-0.5.3.min.js. The issue looks like, "I'm looking for this feature or thing to exist, but can't find it, so I will not load the app." Still need to figure out what that "thing or feature" is though.

Comment 3

[Ed Lee :Mardak]

This seems to work for me now. Probably bug 747451.

Comment 4

[Ed Lee :Mardak]

To be clear, the piano interface loads, sound works, and multiplayer chat/playing works.

Comment 5

[Jason Smith [:jsmith]]

Just re-tested this. This isn't working for me. Let me test this on a couple of other machines to see if this is happening on multiple machines.

Comment 6

[gbrandon]

Hello!  Site person here... let me know if I can help in any way.
In Firefox, the app is currently using Flash (SoundManager2) for audio.  I *have* observed audio failing in Gnash in Ubuntu, but working using the proprietary Flash plugin.

Comment 7

[gbrandon]

Btw, it's working for me (Prism 1.0b4), although CSS3 transitions and websockets are missing :)

Comment 8

[Jason Smith [:jsmith]]

(In reply to gbrandon from comment #7)
> Btw, it's working for me (Prism 1.0b4), although CSS3 transitions and
> websockets are missing :)

Hi. To to this in the web runtime with new firefox implementation, you will need to download a nightly build, install your app using navigator.mozApps.install("<your manifest URL here"), and then launch the application. Let me know what you see when you launch the application with the current nightly build.

Btw, could you drop a URL link in this bug to the web app manifest for multiplayer piano?

Comment 9

[Jason Smith [:jsmith]]

To do this*

Comment 10

[Jason Smith [:jsmith]]

I created an application that puts multiplayer piano in an iframe. I still get exactly the same behavior. See the screencast for a demo of the issue (http://screencast.com/t/5cJKSFYQb4).

Comment 11

[gbrandon]

(In reply to Jason Smith [:jsmith] from comment #8)
> Btw, could you drop a URL link in this bug to the web app manifest for
> multiplayer piano?
http://www.multiplayerpiano.com/piano.webapp

I will have to come back later to help test installation from Firefox nightly.  I actually didn't realize this existed until just now :)

Comment 12

[Jason Smith [:jsmith]]

Attachment: Multiplayer Piano FF Desktop 15 vs. WebRT

Comment 13

[Jason Smith [:jsmith]]

(In reply to gbrandon from comment #11)
> (In reply to Jason Smith [:jsmith] from comment #8)
> > Btw, could you drop a URL link in this bug to the web app manifest for
> > multiplayer piano?
> http://www.multiplayerpiano.com/piano.webapp
> 
> I will have to come back later to help test installation from Firefox
> nightly.  I actually didn't realize this existed until just now :)

gbrandon - Have you had the chance to try installing the multiplayer piano web application on Nightly? I'm still getting a reproduction of this issue on Win 7 64-bit. See attached screenshot.

My guess is this happening during the loading portion of the piano. gbrandon - Do you why the piano would remain permanently blacked out on load? Is there a feature you are checking for such that if it does not exist, the piano stays black?

Comment 14

[gbrandon]

(In reply to Jason Smith [:jsmith] from comment #13)
> (In reply to gbrandon from comment #11)
> > (In reply to Jason Smith [:jsmith] from comment #8)
> > > Btw, could you drop a URL link in this bug to the web app manifest for
> > > multiplayer piano?
> > http://www.multiplayerpiano.com/piano.webapp
> > 
> > I will have to come back later to help test installation from Firefox
> > nightly.  I actually didn't realize this existed until just now :)
> 
> gbrandon - Have you had the chance to try installing the multiplayer piano
> web application on Nightly? I'm still getting a reproduction of this issue
> on Win 7 64-bit. See attached screenshot.
> 
> My guess is this happening during the loading portion of the piano. gbrandon
> - Do you why the piano would remain permanently blacked out on load? Is
> there a feature you are checking for such that if it does not exist, the
> piano stays black?

Sorry, it totally slipped my mind.
Do you need me to test using Windows 7?  I could try in VirtualBox under Ubuntu.

I believe the only thing that would cause the keys to remain gray, should be the audio failing to load, most likely if Flash 10 isn't available.  SoundManager2 needed Flash >= 10 to allow one sound to play multiple times at once, so I required it.  Of course, I hope to write my own HTML5 audio engine to use in Firefox since that's not a buggy as it used to be.  Or maybe the "Audio Data API," if that turns out to be something I can use.

Comment 15

[Jason Smith [:jsmith]]

(In reply to gbrandon from comment #14)
> (In reply to Jason Smith [:jsmith] from comment #13)
> > (In reply to gbrandon from comment #11)
> > > (In reply to Jason Smith [:jsmith] from comment #8)
> > > > Btw, could you drop a URL link in this bug to the web app manifest for
> > > > multiplayer piano?
> > > http://www.multiplayerpiano.com/piano.webapp
> > > 
> > > I will have to come back later to help test installation from Firefox
> > > nightly.  I actually didn't realize this existed until just now :)
> > 
> > gbrandon - Have you had the chance to try installing the multiplayer piano
> > web application on Nightly? I'm still getting a reproduction of this issue
> > on Win 7 64-bit. See attached screenshot.
> > 
> > My guess is this happening during the loading portion of the piano. gbrandon
> > - Do you why the piano would remain permanently blacked out on load? Is
> > there a feature you are checking for such that if it does not exist, the
> > piano stays black?
> 
> Sorry, it totally slipped my mind.
> Do you need me to test using Windows 7?  I could try in VirtualBox under
> Ubuntu.

For now, you would have to test this either on Win XP+ or OS X 10.5+. But I know what the problem is now.

> 
> I believe the only thing that would cause the keys to remain gray, should be
> the audio failing to load, most likely if Flash 10 isn't available. 
> SoundManager2 needed Flash >= 10 to allow one sound to play multiple times
> at once, so I required it.  Of course, I hope to write my own HTML5 audio
> engine to use in Firefox since that's not a buggy as it used to be.  Or
> maybe the "Audio Data API," if that turns out to be something I can use.

Gotcha, now I know what the problem is. This sounds like a flash plugin bug (we've lots of problems with plugins right now in the Nightly build, so I'm not surprised this is the issue). The desktop web runtime isn't integrating with the installed version of flash in this application (it's installed on my machine and works with firefox).

Nominating for k9o - this falls in line with the P1 requirement for flash working in the web runtime.

Comment 16

[Jason Smith [:jsmith]]

Flash version on my win 7 64-bit machine: 11.2.202.235

Comment 17

[Jason Smith [:jsmith]]

Important note - this is windows specific. The issue does not reproduce on OS X. Don't know how to mark that is the OS above though (cause all isn't correct).

Also this issue I think may be happening with Bundlr as well with their vimeo videos.

Comment 18

[Jason Smith [:jsmith]]

Tim - Any ideas? Looks like this is a windows-specific bug.

Comment 20

[Tim Abraldes [:TimAbraldes] [:tabraldes]]

TBH I'm not sure where to start debugging this.  It seems like the app is deciding to keep the piano grayed out and unfortunately the runtime doesn't have good content-debugging tools in it yet (even if it did, I'm not well-versed in content debugging).

It sounds like we think either Flash 10+ is not being detected or audio is failing to load; do we have more information than that (e.g. what call specifically is returning a value that is not what we expect?)

Comment 21

[Jason Smith [:jsmith]]

FYI - See the duplicate bug to this one if it helps. It's another bug that has the same underlying core problem as this bug, but with vimeo videos using flash. 

Johnny - Is there anyone you could suggest that could help debug this problem (e.g. flash not being detected within the runtime)? Looking to find someone who can help that has experience with debugging flash plugins.

Comment 22

[Johnny Stenback (:jst)]

Hmm, good question. At first glance the webapprt process does find flash, as in flash appears in the pluginreg.dat file after launching the app, but it does not appear to work.

John, do you have a VM where you can you give this a look at this? Maybe enable plugin logging and see what it shows, if anything? Bug 754505 seems easier to reproduce here, the manifest url is listed in the bug, so you can simply call navigator.mozApps.install(<manifest url>) and then launch the app and follow the steps in that bug to reproduce.

Comment 23

[gbrandon]

SoundManager2 fails to init, here's its output (in reverse chronological order):

-- SoundManager 2 failed to load (security/load error) --
soundManager.disable(): Shutting down
soundManager: Failed to initialise.
soundManager: No Flash response within expected time. Likely causes: Flash blocked or JS-Flash security error.
soundManager: Getting impatient, still waiting for Flash (SWF loaded)...
soundManager::initMovie(): Waiting for ExternalInterface call from Flash..
soundManager::createMovie(): Trying to load /soundmanager2/soundmanager2_flash9_debug.swf
-- SoundManager 2 V2.97a.20110918 (AS3/Flash 9), high performance mode, normal polling, wmode: transparent --

For testing-- SoundManager 2 failed to load (security/load error) --
soundManager.disable(): Shutting down
soundManager: Failed to initialise.
soundManager: No Flash response within expected time. Likely causes: Flash blocked or JS-Flash security error.
soundManager: Getting impatient, still waiting for Flash (SWF loaded)...
soundManager::initMovie(): Waiting for ExternalInterface call from Flash..
soundManager::createMovie(): Trying to load /soundmanager2/soundmanager2_flash9_debug.swf
-- SoundManager 2 V2.97a.20110918 (AS3/Flash 9), high performance mode, normal polling, wmode: transparent --

Testing urls with SoundManager2 "debug mode" enabled, and only one piano key (less wait when refreshing):
http://www.multiplayerpiano.com/#test
http://www.multiplayerpiano.com/test.webapp

Comment 24

[John Schoenick [:johns]]

I'm still looking at this. Flash appears to load, but the flash <-> JS communication fails. A simpler testcase with just SoundManager2 *does* work, so it's likely some kind of painful race condition.

Comment 25

[Andrew Overholt [:overholt]]

I spoke with John about this.  It's very cryptic and needs much more debugging.  Unknown when a fix will be possible.  John plans to take another look in 1-2 weeks.

Comment 26

[Jason Smith [:jsmith]]

Note to anyone who overheard this in a triage - this actually only specific to the desktop web runtime. Does not reproduce on fennec native (i just did not wait long enough for this to load).

Comment 27

[Jason Smith [:jsmith]]

Although we've decided to not block on this, this still is a highly desired want to be fixed, as the bug that happens here is bad when it does happen. Flagging for tracking-firefox16.

Comment 28

[Alex Keybl [:akeybl]]

If this doesn't block the feature, no need to track. We'd definitely consider taking an uplift if/when ready, however.

Comment 29

[John Schoenick [:johns]]

This may be a variant of bug 771202, it looks like the same thing is happening here - nsObjectLoadingContent sets up its protochain properly, but is then destroyed by some subsequent wrapper operation

Comment 30

[Jason Smith [:jsmith]]

(In reply to John Schoenick [:johns] from comment #29)
> This may be a variant of bug 771202, it looks like the same thing is
> happening here - nsObjectLoadingContent sets up its protochain properly, but
> is then destroyed by some subsequent wrapper operation

Could fixing bug 771202 end up fixing this bug as well?

Comment 31

[John Schoenick [:johns]]

(In reply to Jason Smith [:jsmith] from comment #30)
> Could fixing bug 771202 end up fixing this bug as well?

It seems very likely - the condition is identical, our protochain hook is being destroyed, and I've confirmed that it's not the chain setup in nsObjectLoadingContent at fault

Comment 32

[Benjamin Smedberg]

I don't see how bug 771202 could be related to this bug; bugs of that sort should show up equally in-browser and in webapprt. I think that something like bug 771090 is much more likely.

Comment 33

[John Schoenick [:johns]]

(In reply to Benjamin Smedberg  [:bsmedberg] from comment #32)
> I don't see how bug 771202 could be related to this bug; bugs of that sort
> should show up equally in-browser and in webapprt. I think that something
> like bug 771090 is much more likely.

This doesn't manifest as a crash, the plugin starts, but cannot be accessed by JS. I haven't re-tested on a build with bug 769721 - moving flash OOP may have made resolved this, depending on if we care about in-process plugins on webrt

Comment 34

[John Schoenick [:johns]]

(In reply to John Schoenick [:johns] from comment #33)
> I haven't re-tested on a build with bug 769721

(with 769721 we do immediately hit bug 771090)

Comment 35

[Jason Smith [:jsmith]]

(In reply to John Schoenick [:johns] from comment #34)
> (In reply to John Schoenick [:johns] from comment #33)
> > I haven't re-tested on a build with bug 769721
> 
> (with 769721 we do immediately hit bug 771090)

Right. I had to set the following in pref.js to get the behavior you've described:

user_pref("dom.ipc.plugins.enabled", true);

Without that pref set though, I got the behavior described in this bug. Could that imply that we need to enable dom.ipc.plugins.enabled to true by default in the runtime?

Comment 36

[Jason Smith [:jsmith]]

(In reply to Jason Smith [:jsmith] from comment #35)
> (In reply to John Schoenick [:johns] from comment #34)
> > (In reply to John Schoenick [:johns] from comment #33)
> > > I haven't re-tested on a build with bug 769721
> > 
> > (with 769721 we do immediately hit bug 771090)
> 
> Right. I had to set the following in pref.js to get the behavior you've
> described:
> 
> user_pref("dom.ipc.plugins.enabled", true);
> 
> Without that pref set though, I got the behavior described in this bug.
> Could that imply that we need to enable dom.ipc.plugins.enabled to true by
> default in the runtime?

Or specifically - Enable Out of process plugins in the runtime for flash only, since flash is the only plugin supported in the web runtime?

Comment 37

[Benjamin Smedberg]

I don't know. But this seems like the kind of bug that we've seen when Flash is present but not working correctly. There are all kinds of reasons why this might be, but bug 771202 seems like a very unlikely candidate.

Comment 38

[John Schoenick [:johns]]

So after the landing of https://hg.mozilla.org/mozilla-central/rev/b39f4007be5a the webrt uses OOP plugins on vista+, which do not suffer from this bug. Do we still use in-process plugins for webrt on XP? If not, this is still a bug with in-process plugins, but is much less significant.

Comment 39

[Jason Smith [:jsmith]]

(In reply to John Schoenick [:johns] from comment #38)
> So after the landing of
> https://hg.mozilla.org/mozilla-central/rev/b39f4007be5a the webrt uses OOP
> plugins on vista+, which do not suffer from this bug. Do we still use
> in-process plugins for webrt on XP? If not, this is still a bug with
> in-process plugins, but is much less significant.

Let me retest. Marking qawanted to re-test with nightly to see what happens on XP, Vista, and 7.

Comment 40

[Jason Smith [:jsmith]]

Windows 7: Works
Windows Vista: Works
Windows XP: Fails

My hunch is we are probably still using in-process plugins for webrt on XP. Is making plugins out of process for XP an option?

Comment 41

[(not currently active) Ted Mielczarek]

This might just be a matter of flipping a pref. We run plugins OOP in the browser on XP, that patch simply forces it on for Vista for stability reasons.

Comment 42

[Jason Smith [:jsmith]]

(In reply to Ted Mielczarek [:ted] from comment #41)
> This might just be a matter of flipping a pref. We run plugins OOP in the
> browser on XP, that patch simply forces it on for Vista for stability
> reasons.

Right. In my earlier testing, when "dom.ipc.plugins.enabled" was set to true, this particular app in this bug worked fine. 

In a different bug in connection to this (can't remember which one), I recall Ben suggesting we might want to include those styles of prefs here (except not the java and sliverlight ones, as we don't support those plugins in the webrt):

http://mxr.mozilla.org/mozilla-central/source/browser/app/profile/firefox.js?mark=898-911#898

Comment 43

[Jason Smith [:jsmith]]

Attachment: Potential Patch for IPC Prefs, Not Tested

Is this basically what we need to do to enable the IPC prefs for Win XP?

Note: I haven't tested this yet.

Comment 44

[Jason Smith [:jsmith]]

What I meant to say in comment 43:

Enable the prefs to fix the problem on Windows XP

Comment 45

[Jason Smith [:jsmith]]

Marking for re-triage - Windows Vista and 7 support have a fix in place by a different bug to force flash out of process, so this issue no longer reproduces there (putting flash out of process does not cause the original problem to occur, as the problem only occurs when flash is in-process). Windows XP out of process work-around is being worked in bug 778858.

Comment 46

[Jason Smith [:jsmith]]

Comment on attachment 646565 [details] [diff] [review]
Potential Patch for IPC Prefs, Not Tested

Patch is moved to bug 778858.

Comment 47

[John Schoenick [:johns]]

Bug 778858 has landed, and this issue should no longer occur in normal setups. The combination of webapprt + in-process-plugins is still broken, but that requires manual tweaking of prefs.js files in webapps to encounter.

Comment 48

[Myk Melez [:myk] [@mykmelez]]

The remaining issue isn't worth fixing, given that we run Flash out-of-process by default, and that's the best way to run it.