1118615 - Flash hangs in HiDPI mode on OS X running peopleroulette app (ADBE 3921114)

Status: RESOLVED FIXED

(Core Graveyard :: Plug-ins, defect)

Description

[Chris Peterson [:cpeterson]]

STR:
1. Load https://apps.facebook.com/peopleroulette/chat.php
2. You will need to log into Facebook.
3. You may need to click through a couple People Roulette error pages saying "Hit refresh in your browser to try again, or click here."
4. When Facebook asks whether People Roulette can post to Facebook for you, just click "Not Now".

RESULT:
People Roulette displays a Flash loading progress bar, but Firefox beach balls and hangs. You might force quit Firefox.

I am using OS X 10.10 and Flash 16.0.0.235. This Firefox hang is 100% reproducible for me on Firefox 31 (ESR), Firefox 36, and Nightly 37.

Firefox dumps some debug logs to stdout:

2015-01-06 21:16:44: stackwalker.cc:125: INFO: Couldn't load symbols for: /usr/lib/system/libsystem_kernel.dylib|93E0E0A975B63904BB4E4BC7C05F4B6B0
2015-01-06 21:16:44: stackwalker.cc:125: INFO: Couldn't load symbols for: /usr/lib/system/libsystem_pthread.dylib|26B1897F0CD330F3B55A37CB45062D730
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x16
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x11dcd5d30
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x7fff5fbfd69c
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x1dcd5d30
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x16
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x145af0d40
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x134d333c8
2015-01-06 21:16:44: stackwalker.cc:125: INFO: Couldn't load symbols for: /Applications/Nightly.app/Contents/MacOS/XUL|EAA88496B72A367DAAFA6773CF6CF5AD0
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x134d333e8
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x134d333c8
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x0
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x54acc133
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0xd0b49
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x0
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x608b00f11d
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x5e9000005e900
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x1dcd6500
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x16
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x145af0d40
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x134d333c0
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x1004411a0
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x7fff5fbfd720
2015-01-06 21:16:44: stackwalker.cc:125: INFO: Couldn't load symbols for: /Applications/Nightly.app/Contents/MacOS/libnss3.dylib|163B85C2785F3E5B8DB3D748A299C5540
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x134d333c8
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x54acc14a
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x1526a858
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x54acc133
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x1000d0b47
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x134139460
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x1130099a0
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x134139460
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x57e4
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x38db927
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x7fff5fbfd750
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x10048c000
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x1263ab280
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x1263ab280
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x7fff5fbfd7b0
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x7fff5fbfd810
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x10006e000
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x7fff5fbfd838
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x7fff5fbfd830
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x1130099a0
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x1341395b8
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x1341395c0
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x7fff5fbfd7e0
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x800000008
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x7fff5fbfd838
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x1263ab280
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x1368d0a10
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x7fff5fbfd8d6
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x1368d0a10
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x7fff5fbfd7f0
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x134139460
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x10048c000
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x1263ab280
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x1368d0a10
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x7fff5fbfd8d6
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x1368d0a10
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x7fff5fbfd880
2015-01-06 21:16:44: basic_code_modules.cc:88: INFO: No module at 0x7fff5fbfd880

Comment 1

[Benjamin Smedberg]

Marcia, can you try reproducing on Windows to see whether this is mac-specific or a generic problem?

cpeterson, if you are using a release build of Firefox, does it recover after 45 seconds (or whatever dom.ipc.plugins.timeoutSecs is)?

Comment 2

[Chris Peterson [:cpeterson]]

I have two MacBook Pros, both running OS X 10.10:

* On MBP #1 (with Flash 16.0.0.235), this hang is 100% reproducible. After the plugin timeout, Firefox kills the plugin (displaying the "sad plugin" message) and is responsive again. Crash ID: bp-57d2259e-c05a-4b93-8bf8-e332f2150107

* On MBP #2 (with Flash 16.0.0.240), People Roulette's loading progress bar stops at 100%, but the app won't load until I disable Tracking Protection (privacy.trackingprotection.enabled). Then People Roulette loads without hanging.

Comment 3

[Chris Peterson [:cpeterson]]

Flash 16.0.0.235 is the current Flash release version. Flash 16.0.0.240 is the current Flash beta version from http://labs.adobe.com/downloads/flashplayer.html. If I update my hanging MBP #1 to the Flash beta version (like my MBP #2), Firefox still hangs. So this problem seems unrelated to Flash plugin version.

Comment 4

[Marcia Knous [:marcia]]

Will look into Windows shortly. On my Mac I can also repro the issue on nightly and eventually the tab crashes with this report: https://crash-stats.mozilla.com/report/index/51287066-f680-486d-a2ba-d223c2150108

Comment 5

[Marcia Knous [:marcia]]

Testing on Windows 7 Professional, SP1 with Flash Version Version: 16.0.0.240

(1) Mozilla/5.0 (Windows NT 6.1; WOW64; rv:34.0) Gecko/20100101 Firefox/34.0. No hang and no crash when site loaded

(2)Latest Firefox Nightly - no hang and no crash when site loaded

I will try some of the other Windows machines my lab and see if I get the same results.

Comment 6

[Marcia Knous [:marcia]]

Regarding Mac, I also tried on version 10.8.5 FF Nightly with the latest Flash and wasn't able to reproduce the issue.

Comment 7

[Chris Peterson [:cpeterson]]

bsmedberg: people-roulette.com is the ~#3 crashing Flash website, but its Alexa US rank is #1,110,765. Something must be seriously wrong if such a long-tail website is crashing as much as Facebook and YouTube!

Should someone investigate this bug? Marcia and I can reliably repro on two different Macs running OS X 10.10 (but not all OS X 10.10 Macs nor Windows).

Comment 8

[Steven Michaud [:smichaud] (Retired)]

I know it's dangerous to admit it, but I can reproduce this bug, too.

As best I can tell, the key factor is HiDPI support being on.  I only see this on my Retina MacBook Pro, and then only if gfx.hidpi.enabled is set to its default of '2'.  (To turn off HiDPI support, set gfx.hidpi.enabled to '-1' and restart Firefox.)  I see it in all the versions of OS X I have installed on that machine:  10.7.5, 10.9.5 and 10.10.1.

I don't see this bug in Opera (which, unlike Chrome, uses the NPAPI Flash plugin from Adobe).  But, interestingly, I *do* see something like it in Chrome -- it (like Firefox) "hangs" (for several 10s of seconds) waiting for permission to access your camera and microphone, and sometimes displays an "unresponsive" dialog before the Flash-specific dialog gets displayed.  Safari also waits for a very long time before displaying the dialog.

I suspect we're killing the Flash plugin because the wait for the camera and microphone access dialog is so long.  I don't know why (at least in Firefox) the wait is so much less long in non-HiDPI mode.  And I don't (yet) know how to turn off HiDPI support in Chrome and Safari (though I'm going to look for a way).

If I'm right, this bug is really a Flash bug -- when HiDPI support is on it waits far too long to display the camera and microphone access dialog.

Comment 9

[Steven Michaud [:smichaud] (Retired)]

Jeromie, is this a known Flash bug?  I see it even in the very latest version -- 16.0.0.257.

Comment 10

[Steven Michaud [:smichaud] (Retired)]

> I don't (yet) know how to turn off HiDPI support in Chrome and Safari

It's actually very simple:  Right click on the distro, choose Get Info and check "Open in Low Resolution".  With this setting, the camera and microphone access dialog is displayed *much* more quickly.

Case closed, I think :-)  This is clearly a Flash bug.

Comment 11

[Steven Michaud [:smichaud] (Retired)]

I'd bet the reason this bug doesn't effect Opera is that Opera doesn't inform Flash that it supports HiDPI mode.  I don't actually (yet) know this for sure ... but a little reverse engineering should do the trick, if someone wants to spend the time.

Comment 12

[Jeromie Clark]

This is ADBE 3921114.

Comment 13

[Chris Peterson [:cpeterson]]

* Jeromie: do you have an ETA on when ADBE 3921114 might be fixed?

* Steven: can we lie to people-roulette.com and tell it we're not using HiDPI? I wonder if Flash on Windows has the same bug. I don't know if that would work if the Flash plugin process was already spun up by another website before the user visits people-roulette.com.

Comment 14

[Jeromie Clark]

Nope.  It will be assigned and addressed in the context of our capacity and engineering priorities.  My guess is that the earliest it would land would be our Octavia release, which is in March.

Comment 15

[Steven Michaud [:smichaud] (Retired)]

> can we lie to people-roulette.com and tell it we're not using HiDPI?

I don't think it's practical to do this just for this one site.  But we *can* lie to all instances of the Flash plugin.  I just tried a proof-of-concept patch and it does work around this bug.

This will presumably mean that Flash "movies" won't use the full resolution of HiDPI mode, even in fullscreen mode.  But that's already the case, since we haven't yet figured out how to fix bug 846566 without causing bug 988156.

I'll work on a finished patch.

Comment 16

[Steven Michaud [:smichaud] (Retired)]

By the way, I assume this bug is Mac-specific.  But I don't have any way to test on other platforms, so others will need to.  Those tests will, of course, need to be done on hardware that supports HiDPI.

Comment 17

[Steven Michaud [:smichaud] (Retired)]

Attachment: Workaround

Here's my patch.

And here's a tryserver run I just started:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=44cfbd09d211

Comment 18

[Chris Peterson [:cpeterson]]

(In reply to Marcia Knous [:marcia - use needinfo] from comment #5)
> I will try some of the other Windows machines my lab and see if I get the
> same results.

Marcia: do you have any Windows machines with HiDPI screens? We know this is a HiDPI bug on OS X, but we don't know if Windows has the same root cause.

Comment 19

[Chris Peterson [:cpeterson]]

Comment on attachment 8549894 [details] [diff] [review]
Workaround

Review of attachment 8549894 [details] [diff] [review]:
-----------------------------------------------------------------

::: dom/plugins/base/nsNPAPIPlugin.cpp
@@ +2176,5 @@
> +    // support is available. This is Adobe bug ADBE 3921114, which should get
> +    // fixed in a future release. When this happens we'll no longer need this
> +    // workaround. See QUIRK_FLASH_HIDE_HIDPI_SUPPORT in PluginModuleChild.h,
> +    // and also bug 1118615.
> +    if (inst) {

Do you need a QUIRK_FLASH_HIDE_HIDPI_SUPPORT check here?

@@ +2181,5 @@
> +      const char *mimeType;
> +      inst->GetMIMEType(&mimeType);
> +      NS_NAMED_LITERAL_CSTRING(flash, "application/x-shockwave-flash");
> +      if (!PL_strncasecmp(mimeType, flash.get(), flash.Length())) {
> +        scaleFactor = 1.0;

Should we #ifdef XP_MACOSX these scaleFactor workarounds until we know Windows Flash has the same HiDPI bug?

Comment 20

[Marcia Knous [:marcia]]

(In reply to Chris Peterson [:cpeterson] from comment #18)
> (In reply to Marcia Knous [:marcia - use needinfo] from comment #5)
> > I will try some of the other Windows machines my lab and see if I get the
> > same results.
> 
> Marcia: do you have any Windows machines with HiDPI screens? We know this is
> a HiDPI bug on OS X, but we don't know if Windows has the same root cause.

Will try to see if we have one here in the office. I know right now I don't have one in my small QA array of machines that I am using for MSE.

Comment 21

[Steven Michaud [:smichaud] (Retired)]

(In reply to comment #19)

> Do you need a QUIRK_FLASH_HIDE_HIDPI_SUPPORT check here?

It's not available here.

For another quirk-like usage outside of PluginModuleChild.cpp, see this:

http://hg.mozilla.org/mozilla-central/annotate/89a190592267/dom/plugins/base/nsNPAPIPluginInstance.cpp#l745

> Should we #ifdef XP_MACOSX these scaleFactor workarounds

No need to.  That code is already in an #ifdef XP_MACOSX block.

Comment 22

[Benjamin Smedberg]

Comment on attachment 8549894 [details] [diff] [review]
Workaround

I'm going to delegate this to somebody who hopefully knows about mac hidpi; I know nothing useful about this at all.

Comment 23

[Markus Stange [:mstange]]

Comment on attachment 8549894 [details] [diff] [review]
Workaround

Seems reasonable enough, as far as workarounds go.

Comment 24

[Steven Michaud [:smichaud] (Retired)]

Comment on attachment 8549894 [details] [diff] [review]
Workaround

Landed on mozilla-inbound:
https://hg.mozilla.org/integration/mozilla-inbound/rev/dc36cbaa1cc3

Comment 25

[Steven Michaud [:smichaud] (Retired)]

[Tracking Requested - why for this release]:

Let's let this bake on the trunk for a few days.  Then (if no problems are discovered) let's uplift this to aurora and beta.  I think we want to get this into the next FF release (i.e. FF 36).

Comment 26

[Steven Michaud [:smichaud] (Retired)]

I expect there are *lots* of sites that trigger a Flash camera/microphone access dialog.  And we probably don't hear about most cases of this bug, since it's very hard to track by signature.

Comment 27

[Steven Michaud [:smichaud] (Retired)]

If it turns out that this bug also happens on Windows and/or Linux, please open new bugs for those cases.

Comment 28

[Steven Michaud [:smichaud] (Retired)]

> people-roulette.com is the ~#3 crashing Flash website

Chris, could you post the top 10 crashing Flash websites?  Is the list of crashing Flash websites publicly available, and if so where?  Can you give me full URLs (instead of just websites)?  I'm curious how many of these are this bug.

And is this list for all platforms?  If it is, I expect this bug also happens on Windows.

Comment 29

[Jeromie Clark]

Is there anything special that might help me reproduce this?  Maybe I'm missing a step?  Non-default Firefox settings, particular hardware models, etc?  It sounds like a 5K iMac isn't required.  I'll upgrade a couple MBPs and see if I can repro there... 

PASS: Firefox 35.0, 36 beta on MacOS 10.10.1 with Flash Player 16.0.0.257 using Mac Mini with 4k display
PASS: Firefox 35.0 on MacOS 10.9.5 with Flash Player 16.0.0.257 on 20-in iMac, mid-2007

Comment 30

[Stephen A Pohl [:spohl]]

Might be limited to MacBooks with Retina displays (see comment 8)?

Comment 31

[Steven Michaud [:smichaud] (Retired)]

Jeromie, you need a machine that supports HiDPI.  Just get a Retina MacBook Pro.

The 5K iMac also supports HiDPI.  But I'm not sure any of us at Mozilla have ever tested on one, and I've seen one report of very weird problems.  So if you test on on a 5K iMac the waters may get muddied by other problems.

Comment 32

[Chris Peterson [:cpeterson]]

(In reply to Steven Michaud from comment #28)
> > people-roulette.com is the ~#3 crashing Flash website
> 
> Chris, could you post the top 10 crashing Flash websites?  Is the list of
> crashing Flash websites publicly available, and if so where?  Can you give
> me full URLs (instead of just websites)?  I'm curious how many of these are
> this bug.
> 
> And is this list for all platforms?  If it is, I expect this bug also
> happens on Windows.

Windows is definitely affected, though I don't have a Windows machine.

The Flash crash sites can be queried in Socorro, though you need an extra permission bit to see the URLs. I compiled a list of the top 10 sites (merging similar URLs) and the Socorro queries I used on this etherpad:

https://etherpad.mozilla.org/candy-crash

Comment 33

[Steven Michaud [:smichaud] (Retired)]

Jim Mathies tested the STR from comment #0 on a SurfacePro with HiDPI.  He doesn't see the bug -- there's no hang and he sees the Flash camera/microphone access dialog.

So it looks like this bug *doesn't* happen on Windows.  But it'd still be nice to get extra confirmation.

Jeromie, could you create a test page containing a Flash object that does nothing more than try to access the camera/microphone (so that Flash tries to show the camera/microphone access dialog)?  That'd make everyone's testing easier, including yours.

Comment 34

[Jeromie Clark]

I'm able to reproduce this using a retina MBP, even when I explicitly allow webcam access through the native control panel for this domain.  I never get to the allow camera/mic dialog.  We'll take a look.

Comment 35

[Jim Mathies [:jimm]]

(In reply to Steven Michaud from comment #33)
> Jim Mathies tested the STR from comment #0 on a SurfacePro with HiDPI.  He
> doesn't see the bug -- there's no hang and he sees the Flash
> camera/microphone access dialog.
> 
> So it looks like this bug *doesn't* happen on Windows.  But it'd still be
> nice to get extra confirmation.

Note, due to bug 1092121, you should test in a non-e10s window on Windows.

Comment 36

[Steven Michaud [:smichaud] (Retired)]

> you need an extra permission bit to see the URLs

I have this, and have started searching on them.  I notice that there are *lots* of crashes on Windows at URLs that end with peopleroulette/chat.php.  So either those are different bug(s), or we don't yet have the full story on Windows for this bug.

Comment 37

[Steven Michaud [:smichaud] (Retired)]

> Jeromie, could you create a test page containing a Flash object that
> does nothing more than try to access the camera/microphone (so that
> Flash tries to show the camera/microphone access dialog)?  That'd
> make everyone's testing easier, including yours.

Jeromie, please do this.

I don't think we have anyone in Mozilla with the requisite knowledge.
I certainly don't know enough to do it myself.

Comment 38

[Jeromie Clark]

That was the first thing I looked at.  I just googled for a simple flash camera example... 

http://blog.gskinner.com/archives/2005/08/flash_8_webcam.html

Comment 39

[Steven Michaud [:smichaud] (Retired)]

> http://blog.gskinner.com/archives/2005/08/flash_8_webcam.html

I don't hang with this.  I tested in FF 34.0.5 on both OS X 10.7.5 and 10.10.1.

So more is required to trigger this bug than just making Flash display a camera/microphone access dialog while HiDPI support is on.

Comment 40

[Steven Michaud [:smichaud] (Retired)]

I looked through the URLs for the top several Flash crashes on OS X.  Of course any URL that wants to access your camera or microphone is likely to require a login.  But you can tell something from just looking at the content of the URL.  Most of them look like links to videos that you can watch, which wouldn't require access.  Only a few look like links to apps for two-way communication, which probably would require access.

So I'd guess this bug *isn't* responsible for a large proportion of Flash crashes, at least on OS X.

But of course we'll only know for sure when the patch lands on beta or gets into a release.

Comment 41

[Jeromie Clark]

We've done some debugging and are satisfied that this is not a widespread issue that affects all camera/mic dialogs.  My thesis is that we're seeing greater-than-expected reports from this particular site because of the prominence of this plug-in crash dialog.  

We have a couple theories based on what we're seeing about why this content is running so slowly on retina displays and will continue to investigate.

Comment 42

[Steven Michaud [:smichaud] (Retired)]

> http://blog.gskinner.com/archives/2005/08/flash_8_webcam.html

I did a search on 'flash webcam' (no quotes), and have been trying the Flash webcam plugins it turned up.  So far none of them hang (in FF 34.0.5) with HiDPI support on.  I'll keep looking, and will post any I find that do hang.  Here are a couple that don't:

http://www.testwebcam.com/
http://www.hellowebcam.com/

Comment 43

[Steven Michaud [:smichaud] (Retired)]

> http://www.hellowebcam.com/

I also tried all the examples linked from this page.  None of them hang.

Comment 44

[Jeromie Clark]

Yeah, this hang doesn't actually have anything to do with camera/mic.  They're in a tight loop with a long-running call that never returns.  It's a bug in the ActionScript, but we'll need to dive deeper to figure out what the actual root-cause is.  Since it's not a huge issue, we're going back to treating it like a normal bug.

Comment 45

[Steven Michaud [:smichaud] (Retired)]

I noticed something else, but it's probably just a coincidence:

All the dialogs that don't hang are "simple" ones.  The one that does is more "complex".  I'll post screenshots.

Comment 46

[Steven Michaud [:smichaud] (Retired)]

Attachment: "simple" camera/microphone access dialog

Comment 47

[Steven Michaud [:smichaud] (Retired)]

Attachment: "complex" camera/microphone access dialog

Comment 48

[Chris Peterson [:cpeterson]]

(In reply to Jeromie Clark from comment #44)
> Yeah, this hang doesn't actually have anything to do with camera/mic. 
> They're in a tight loop with a long-running call that never returns.  It's a
> bug in the ActionScript, but we'll need to dive deeper to figure out what
> the actual root-cause is.  Since it's not a huge issue, we're going back to
> treating it like a normal bug.

Is the HiDPI workaround a red herring then? Or is this a race condition and disabling HiDPI happens to reduce the time Flash spends pushing pixels to avoid the ActionScript problem?

Comment 49

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/dc36cbaa1cc3

Comment 50

[Jeromie Clark]

It's not clear yet.  There's definitely a delay in execution in this specific SWF on Retina iMacs when high DPI mode is enabled, which ultimately results in Firefox killing the process because it exceeds the hang detection's timeout.  In looking at the JITed code, it's clear that there's a tight loop, but we can't tell from looking at the debug output what ActionScript is involved at first glance.

Comment 51

[Steven Michaud [:smichaud] (Retired)]

I've found out that Flash *isn't* hanging trying to display the camera/microphone access dialog (in HiDPI mode), even on the peopleroulette site:

1) On a Retina MacBook Pro, disable HiDPI support by setting gfx.hidpi.enabled to '-1' in about:config, then restart Firefox 35.
2) Visit https://apps.facebook.com/peopleroulette/chat.php and perform the STR from comment #0.
3) When the camera/microphone access dialog appears, select "Allow" and "Remember".  The next time you visit this site, the dialog will no longer appear and it will automatically be granted access to your camera and microphone.
4) Reset gfx.hidpi.enabled in about:config, then restart Firefox again.
5) Perform the STR from comment #0 again.  Notice that you still hang.

So (from this evidence and the timing) it seems that the hang happens *before* Flash attempts to display the camera/microphone access dialog.  The progress bar (a Flash object) is still displayed.  Could it be the progress bar itself that's causing the hang?

Jeromie, is that progress bar a standard Flash object?  Can you point us to other examples to test with?

Comment 52

[Jeromie Clark]

That's the Flex progress bar.  We're not aware of a widespread issue affecting all Flex applications in Firefox.  This is a case of a developer shooting themselves in the foot.

We donated Flex to the Apache foundation a couple years back.  

Here's the current Flex showcase:
http://flex.apache.org/community-showcase.html

Comment 53

[Steven Michaud [:smichaud] (Retired)]

Thanks for the info, Jeromie.

> http://flex.apache.org/community-showcase.html

I tried all the demos on this page, most of which used at least one progress bar.  None of them hung.

Comment 54

[Steven Michaud [:smichaud] (Retired)]

(Following up comment #44)

> It's a bug in the ActionScript, but we'll need to dive deeper to
> figure out what the actual root-cause is.

So you seem to be right about this, Jeromie.

It'd be nice to have an explanation of what the People Roulette site
did wrong, so that we (Mozilla) or you (Adobe) can contact them about
it.

Comment 55

[Steven Michaud [:smichaud] (Retired)]

Comment on attachment 8549894 [details] [diff] [review]
Workaround

In the meantime, I don't think we'll do any harm by uplifting this patch to aurora and beta.

Approval Request Comment
[Feature/regressing bug #]: Hang in HiDPI mode in popular Facebook app, likely caused by bad Flash ActionScript on the site.
[User impact if declined]: A significant number of Mac users will hang using this Facebook app.
[Describe test coverage new/current, TreeHerder]: Baked for a few days on m-c with no reported problems.
[Risks and why]: Low risk.  Bug 846566 already prevents HiDPI support from working correctly in Flash.
[String/UUID change made/needed]: none

Comment 56

[Jeromie Clark]

Thanks for the validation.  I've had a little experience with this stuff.  :D

We're not going to reach out to individual developers to talk to them about their bugs unless they're massively important (Alexa Top-50, etc).  We've conducted similar developer outreach programs in the past, and they were phenomenally expensive and ineffective.  If you haven't done it before, just the task of trying to find the right developer at a huge corporation is frequently a long and arduous task.

You're absolutely more than welcome to reach out to these guys yourselves.  

The best advice here is to just to use Adobe Scout to profile their application.  It's a really good profiler, but a lot of people (especially outside the game space) don't know that they can profile Flash.  The long-running loop should be immediately obvious to their developer.  

Because they built this with Flex, some or all of the ActionScript is generated (Flex developers use a high-level markup called MXML).  We may not able to directly get at what they wrote, and we're not going to debug their application for them for free (we do offer enterprise support contracts).

If they're unable to resolve the issue themselves, we provide both documentation and user-to-user forums where they can seek help.  I'm pretty active and easy to reach in that venue as well.

Comment 57

[Steven Michaud [:smichaud] (Retired)]

Comment on attachment 8549894 [details] [diff] [review]
Workaround

Actually these comments are wrong -- the hangs don't happen accessing the camera and microphone access dialog, and they're probably not a Flash bug.

I'll fix them on upload, and at the same time on the trunk.

Comment 58

[Steven Michaud [:smichaud] (Retired)]

Attachment: Workaround for aurora and beta

What I landed on aurora and beta, with fixed comments.

Comment 59

[Steven Michaud [:smichaud] (Retired)]

Comment on attachment 8552551 [details] [diff] [review]
Workaround for aurora and beta

Landed on aurora and beta:
https://hg.mozilla.org/releases/mozilla-aurora/rev/e40719240cce
https://hg.mozilla.org/releases/mozilla-beta/rev/430bff48811d

Comment 60

[Steven Michaud [:smichaud] (Retired)]

Attachment: Patch to fix comments on trunk

What I landed on fx-team.

Comment 61

[Steven Michaud [:smichaud] (Retired)]

Comment on attachment 8552636 [details] [diff] [review]
Patch to fix comments on trunk

Landed on fx-team:
https://hg.mozilla.org/integration/fx-team/rev/1385f99516d5

Comment 62

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/1385f99516d5

Comment 63

[Steven Michaud [:smichaud] (Retired)]

This bug seems to have been mitigated in the latest Flash releases.  The peopleroulette chat app still stops loading (in HiDPI mode) at the same point (with a completed progress bar displayed), but this no longer results in us killing the Flash plugin.

Because of this, and because my patch triggered bug 1129267, I'm going to back this patch out on all branches where it landed -- 38, 37 and 36.  I'll post patches shortly.

The current version of the Flash plugin is 16.0.0.305.  But I also don't see these hangs/crashes (in FF 35 or 36b1) with 16.0.0.296.

Comment 64

[Steven Michaud [:smichaud] (Retired)]

> I'll post patches shortly.

I'm doing this at bug 1129267.

Comment 65

[[SV Manager] Florin Mezei, QA (:FlorinMezei)]

Will no longer verify this since it was backed out.