Reduce the amount of dual AMD/intel blacklisting

RESOLVED FIXED in Firefox 44

Status

()

defect
RESOLVED FIXED
5 years ago
4 years ago

People

(Reporter: jrmuizel, Assigned: BenWa)

Tracking

(Blocks 2 bugs, {qawanted})

unspecified
mozilla44
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox35 wontfix, firefox36 affected, firefox37 affected, firefox38 affected, firefox44 fixed)

Details

Attachments

(3 attachments, 6 obsolete attachments)

Bug 1093863 caused us to blacklist more than we'd like to. We need to figure out some criteria to reduce this.
Blocks: 1093863
Blocks: 1096934
Duplicate of this bug: 1093863
Duplicate of this bug: 1097162
yes, please fix this. The fonts are looking horrible so that I get headache after a few minutes. I'm back now to 34b6 until this is fixed.

Here my output from about:support

Direct2D aktiviert	true
DirectWrite aktiviert	true (6.2.9200.16581)
Geräte-ID	0x0166
GPU #2 aktiv	false
GPU-beschleunigte Fenster	1/1 Direct3D 11 (OMTC)
Karten-Beschreibung	Intel(R) HD Graphics 4000
Karten-RAM	Unknown
Karten-Treiber	igdumdim64 igd10iumd64 igd10iumd64 igdumdim32 igd10iumd32 igd10iumd32
Subsys-ID	05721028
Treiber-Datum	9-30-2014
Treiber-Version	10.18.10.3958
Vendor-ID	0x8086
WebGL-Renderer	Google Inc. -- ANGLE (Intel(R) HD Graphics 4000 Direct3D9Ex vs_3_0 ps_3_0)
windowLayerManagerRemote	true
AzureCanvasBackend	direct2d
AzureContentBackend	direct2d
AzureFallbackCanvasBackend	cairo
AzureSkiaAccelerated	0

10.18.10.3958 is the latest driver for the Intel HD 4000 and I don't use the HD 77xxM in my Dell Inspiron 7520 (15R SE) at all for Firefox. The Intel HD is fast enough for me.
I always used Firefox without any problems until 32.0.3. After 33.0 update, I got a few UI freezes (but not crashes), that were easily recovered by closing and reopening FF. After 33.0.3, the browser became laggy and the results in any benchmark were ridiculously low. Checking about:support, I noticed all graphics acceleration options were turned off, despite being selected in settings. Searching Bugzilla, I discover my configuration got blacklisted ("Unexpected Intel/AMD dual-GPU setup" error).

I don't know why I got blacklisted. I have a Samsung Series 7 i7 Sandy Bridge notebook (3 years old) with dual Intel/AMD GPUs running Win8.1. FF is assigned only to the Intel HD3000 GPU. AMD 6490M is not active for Firefox in any circumstance so FF should simply ignore it. Drivers are installed correctly and updated (AMD to 14.4 because of the known instabilities of 14.9 version). It amazes me that FF finds my configuration "unexpected".

I reverted to 32.0.3 and it's all good again (except for not having the latest security patches). If you need any further information to help to fix this issue, let me know. Thanks for the hard work.

Data do driver	3-20-2014
Descrição do adaptador	Intel(R) HD Graphics 3000
Direct2D ativo	true
DirectWrite ativo	true (6.3.9600.17111)
Drivers do adaptador	igdumd64 igd10umd64 igd10umd64 igdumd32 igd10umd32 igd10umd32
GPU #2 ativa	false
ID do dispositivo	0x0116
ID do fornecedor	0x8086
Janelas aceleradas pela GPU	1/1 Direct3D 10
Parâmetros ClearType	D [ Gamma: 2400 Pixel Structure: R ClearType Level: 50 Enhanced Contrast: 300 ] D [ Gamma: 2200 Pixel Structure: R ClearType Level: 100 Enhanced Contrast: 50 ]
RAM do adaptador	Unknown
Renderização WebGL	Google Inc. -- ANGLE (Intel(R) HD Graphics 3000 Direct3D9Ex vs_3_0 ps_3_0)
Versão do driver	9.17.10.3517
windowLayerManagerRemote	false
AzureCanvasBackend	direct2d
AzureContentBackend	direct2d
AzureFallbackCanvasBackend	cairo
AzureSkiaAccelerated	0
(In reply to Marcelo from comment #4)
> ...
> I reverted to 32.0.3 and it's all good again (except for not having the
> latest security patches). If you need any further information to help to fix
> this issue, let me know. Thanks for the hard work.

Bug 1095575 should allow to override the blacklisting using a preference. The current patch which should land soon and hopefully makes it into Firefox 34 looks at either gfx.direct2d.force-enabled or layers.acceleration.force-enabled to bypass the blacklisting.

This bug (1097321) is about blacklisting less systems, and hopefully the actual issue which requires the blacklisting will also get handled.
Avi, thanks for the update! 

I hope that D3D10 GPU's, like the HD3000, are kept supported natively in FF. D3D9 fallback is not a good alternative because of D2D support.
Thanks for the update Avi! Hopefully Firefox 34 will bring back D3D and beautiful font rendering as Firefox 33.0.2 below.
Another update:
We're having a machine shipped from Romania that should reproduce the original problem. With that machine in hand we should be able to figure out what's special about it and reduce the blacklisting to that configuration.
thanks for the update Jeff. Is it possible to have https://bugzilla.mozilla.org/show_bug.cgi?id=1095575 included in Firefox 34 and not only in nightly?
The laptop is supposed to arrive in Toronto in 19 Nov 2014
I have the laptop and can reproduce the problem.

Avi, or anyone else, can you see if you can reproduce the black screen in 33.0.2 if you force firefox to use the 'High performance' graphics setting.
I only tried with nightly 2014-11-26 so far on i7-4500u (+HD4400) + AMD 8850m.

By default Firefox uses only the Intel iGPU, and nightly currently blacklists all HW acceleration.

In order to force high performance (use the AMD GPU), I needed to rename it because "Firefox.exe" is blacklisted on the Catalyst 14.4 driver to disallow high performance.

After renaming it and forcing high performance, it works well and still HW accel is blacklisted.

After setting layers.acceleration.force-enabled=true and forcing high performance - I see the black window bug.
And if I only set layers.acceleration.force-enabled=true but don't force high performance, then I don't get the black window bug.
Posted patch Detect the brokeness (obsolete) — Splinter Review
Posted patch Detect the brokeness (obsolete) — Splinter Review
Attachment #8529888 - Attachment is obsolete: true
Posted patch detect-broke.patch (obsolete) — Splinter Review
Attachment #8529895 - Attachment is obsolete: true
Attachment #8530385 - Flags: review?(bas)
Attachment #8530385 - Flags: review?(bas) → review+
After FF 34 my Intel/AMD combo is still blacklisted. But now if I enable gfx.direct2d.force-enabled I get full hw acceleration and no problem at all. Thanks!

Curiously, I also got OMTC enabled on Direct3D 11 (as per about:support) even if layers.acceleration.force-enabled is disabled and my HD3000 only supports D3D10...
https://hg.mozilla.org/mozilla-central/rev/4c22e58f4398
Assignee: nobody → jmuizelaar
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla37
Comment on attachment 8530385 [details] [diff] [review]
detect-broke.patch

Approval Request Comment
[Feature/regressing bug #]: Bug 1093863
[User impact if declined]: Unnecessary black listing of AMD hardware
[Describe test coverage new/current, TBPL]: Has been on mozilla-central for a couple of days. I also tested this locally against a machine that has the problem
[Risks and why]: This changes a rough heuristic for detecting black screens to code that actually detects the problem. This means that we'll only black list on machines that truly have the problem. I'd like to get this in beta to get the widest testing possible. If this ends up causing crashes it should be easy to localize to this code and back out the patch from beta.
Attachment #8530385 - Flags: approval-mozilla-beta?
Attachment #8530385 - Flags: approval-mozilla-aurora?
Attachment #8530385 - Flags: approval-mozilla-beta?
Attachment #8530385 - Flags: approval-mozilla-beta+
Attachment #8530385 - Flags: approval-mozilla-aurora?
Attachment #8530385 - Flags: approval-mozilla-aurora+
I hope we will be able to get some confirmation on this before we release :)
I'll test the next 35 Beta version which includes this patch and set gfx.direct2d.force-enabled to false and look if I'm still blocked or not (Dell Inspirion laptop with Intel HD4000 + AMD 7730M).
(In reply to André Ziegler from comment #21)
> I'll test the next 35 Beta version which includes this patch and set
> gfx.direct2d.force-enabled to false and look if I'm still blocked or not
> (Dell Inspirion laptop with Intel HD4000 + AMD 7730M).

I tested this with nightly on a Dell 3537 with HD4400 + AMD 8850m.

Before this patch and without forcing acceleration via prefs, this system was blacklisted.

With this patch and default prefs I now get Direct3D 11 (OMTC) and direct2d 1.1 for Azure{Canvas|Content}Backend.

So success on this system.
I tested the candidate of 35b3 and it works fine. With gfx.direct2d.force-enabled = false I still have HW accelerated graphic.
(In reply to André Ziegler from comment #24)
> I tested the candidate of 35b3 and it works fine. With
> gfx.direct2d.force-enabled = false I still have HW accelerated graphic.

Marking as Verified for Firefox 35.
Actually, I think we should back this out fast from both Aurora and Beta. The signatures of bug 1098597 and bug 1074378 (re)appeared on Aurora starting with the 12/12 build, the first that has this patch, and on 35 beta 3, which is the first beta containing it, the 3 signatures of those two bugs together are 75% of all crashes.
Jeff- can you prep a backout for Beta before this Thursday's go to build?
Flags: needinfo?(jmuizelaar)
Cancel that request - if Ryan can cleanly back this out, we can gtb with a 35.0b4 build #2 later today.
Flags: needinfo?(jmuizelaar) → needinfo?(ryanvm)
which issues do you see? For me everything is fine my an Intel Hd 4000/AMD 7730M
Backed out from beta while the crash spike is investigated.
https://hg.mozilla.org/releases/mozilla-beta/rev/c364b0e2691f
Flags: needinfo?(ryanvm)
(In reply to André Ziegler from comment #29)
> which issues do you see? For me everything is fine my an Intel Hd 4000/AMD
> 7730M

For you it might, but we have a lot of people crashing on startup on Aurora and Beta (seen in crash reports/stats). We need to figure those cases out before trying this unblocking again.
So none of the crashes that I looked at would have been blacklisted by the code path the patch on this bug removed. However, it may be that this broke some of the detection code that we had. I'll investigate further.
I looked into this further on an intel machine that should be blacklisted. It currently appears as though it is not, and I have no idea why.
(In reply to Robert Kaiser (:kairo@mozilla.com) from comment #31)
> For you it might, but we have a lot of people crashing on startup on Aurora
> and Beta (seen in crash reports/stats). We need to figure those cases out
> before trying this unblocking again.

ok. You should tell the users to update their drivers if you detect old ones.
(In reply to André Ziegler from comment #34)
> ok. You should tell the users to update their drivers if you detect old ones.

We found out that updating drivers is a so intricate thing to do in any cases that a non-technical user will never do it. If we tell those people to update their driver, they will rather switch to a different browser than actually do that too complicated and risky action.

All that said, Jeff and the graphics team will continue to work on unblocking those machines that do not actually have issues, but something went wrong with the patch this time.


Jeff, given that you could verify yourself that the patch doesn't work as intended and unblacklists machines it shouldn't, would it be good to back it out from Dev Edition and Nightly as well or will you get to a fix very soon?
Flags: needinfo?(jmuizelaar)
(In reply to Robert Kaiser (:kairo@mozilla.com) from comment #35)
> 
> Jeff, given that you could verify yourself that the patch doesn't work as
> intended and unblacklists machines it shouldn't, would it be good to back it
> out from Dev Edition and Nightly as well or will you get to a fix very soon?

I figured out what was going wrong and it appears to be unrelated to this bug. The machine I had was being blocked properly. Did backing this out from beta make the crashes go away?
Flags: needinfo?(jmuizelaar) → needinfo?(kairo)
(In reply to Jeff Muizelaar [:jrmuizel] from comment #36)
> I figured out what was going wrong and it appears to be unrelated to this
> bug. The machine I had was being blocked properly.

Oh, hrm, I hoped you would have a case you can investigate and find the problem with. :(

> Did backing this out from beta make the crashes go away?

Yes, the crash spike is gone from beta, so we can confirm that it was related to this patch. We still have those crashes go on at least on Dev Edition, though (I haven't explicitly checked Nightly but guess it's the same there) - even though fewer people use those affected machines there (and those still there probably will go away given they still have ongoing startup crashes).

FWIW, if I have read some stats I've seen correctly, we probably have effectively purged the Nightly and Aurora channels from people with broken drivers by all the previous issues we sent their way - we saw significant decline of users on those channels when OMTC hit them first.
Flags: needinfo?(kairo)
So to recap, this is currently backed out from Fx35/Fx36 but remains in Fx37. Jeff, do we have any further plans for this bug?
Flags: needinfo?(jmuizelaar)
Yes we should back this out of everywhere else. I'll try to write a new patch that fixes the problem.
Status: RESOLVED → REOPENED
Flags: needinfo?(jmuizelaar)
Resolution: FIXED → ---
Target Milestone: mozilla37 → ---
Attachment #8530385 - Attachment is obsolete: true
Attachment #8530385 - Flags: approval-mozilla-beta+
Attachment #8530385 - Flags: approval-mozilla-aurora+
I am affected by this bug on a dual-gpu system with Intel HD Graphics 4000 and AMD Radeon HD 8570, Windows 8.1 x64.

Drivers for both GPUs are updated to latest versions (Intel 10.18.10.4061 from 12/18/2014 and AMD 14.501.1003.0 from 11/20/2014). I had to force the Intel driver update as it is not "manufacturer-validated" (thanks Samsung). I am testing the hardware acceleration with the WebGL animation at acko.net and MSI Afterburner to see which GPU is doing the work. On the screenshots, GPU1 is Intel, GPU2 is AMD.

Case #1, Firefox 36.0.1:
about:support https://pastebin.mozilla.org/8825658
acko.net http://i.imgur.com/XenO7J7 
Direct2D is disabled, neither GPU seems to be doing much, performance is terrible at 13fps.
Forcing gfx.direct2d.force-enabled results in the black window bug.

Case #2, Nightly 39.0a1
about:support https://pastebin.mozilla.org/8825662
acko.net http://i.imgur.com/5Zz9VDu.jpg
Direct2D is still disabled, both GPUs seem to do something, with Intel GPU doing the bulk of the work. Performance is choppy at ~40fps with tearing. Forcing gfx.direct2d.force-enabled results in the black window bug.

Case #3, Nightly 39.0a1
I arrived at this by accident. If I manually force the update of the Intel driver (by pointing to the folder and choosing the driver from the list in device manager), but *do not restart the system*, it seems to work!
about:support https://pastebin.mozilla.org/8825663
acko.net http://i.imgur.com/HwTZpog.jpg
Direct2D is enabled, and the bulk of work is done by the AMD GPU as it's supposed to. Acko.net is buttery smooth at 60fps. 

Rebooting the laptop brings me back to Case #2.

Google Chrome behaves correctly (like in case #3) all the time (without driver-forcing not-rebooting shenanigans) so it doesn't seem to be a hardware limitation.

I'd be happy to provide additional information if required.
Do you still see the same behaviour (in case #3 in particular) with the latest nightly?
Flags: needinfo?(fijam7)
Yes, unchanged with Nightly 39.0a1 / 20150327030212
Flags: needinfo?(fijam7)
Strange that we don't see the second GPU in the about:support you posted.  I only see the first card, Intel, and never see the description of the AMD GPU, driver versions, etc.
Blocks: 1148088
Regarding 1148088, it appears that my AMD/Intel (AMD Radeon HD 6770M/Intel HD Graphics 3000) combo is blacklisted. If I enable gfx.direct2d.force-enabled I get a black screen when launching Firefox just like fijam7@gmail.com is reporting. Is any type of work being done to fix this or is this a permanent issue with Direct2D and AMD/Intel setups?
Any update on this?

My friend has Intel HD Graphics 4600 + AMD Radeon HD 89670M + Win 10 x64, his Firefox works flawlessly.

Mine (HD Graphics 3000 + AMD Radeon HD 7400M + Win 10 x64) got that

(#0) Error	Unexpected Intel/AMD dual-GPU setup
(#1) Error	Unexpected Intel/AMD dual-GPU setup
Posted patch A new attempt at this (obsolete) — Splinter Review
Please review carefully
Attachment #8671501 - Flags: review?(bgirard)
Blocks: 1099354
Posted patch patch (obsolete) — Splinter Review
jeff's patch. I just fixed it up.
Attachment #8671501 - Attachment is obsolete: true
Attachment #8671501 - Flags: review?(bgirard)
Attachment #8671639 - Flags: review?(bgirard)
Comment on attachment 8671639 [details] [diff] [review]
patch

Review of attachment 8671639 [details] [diff] [review]:
-----------------------------------------------------------------

::: gfx/thebes/gfxWindowsPlatform.cpp
@@ +1919,5 @@
> +  desc.CPUAccessFlags = D3D11_CPU_ACCESS_READ;
> +  desc.MiscFlags = 0;
> +  desc.BindFlags = 0;
> +  if (FAILED(device->CreateTexture2D(&desc, nullptr, byRef(cpuTexture)))) {
> +    return false;

We should have a graphics critical error for this maybe?

@@ +1927,5 @@
> +  nsRefPtr<ID3D11DeviceContext> deviceContext;
> +  sharedResource->QueryInterface(__uuidof(IDXGIKeyedMutex), (void**)getter_AddRefs(sharedMutex));
> +  device->GetImmediateContext(getter_AddRefs(deviceContext));
> +  if (FAILED(sharedMutex->AcquireSync(0, 30*1000))) {
> +    gfxCriticalError() << "DoesD3D11TextureSharingWorkAcquireSyncTimeout";

perhaps this would be better:
DoesD3D11TextureSharingWorkAcquire_SyncTimeout

@@ +1941,5 @@
> +  if (SUCCEEDED(deviceContext->Map(cpuTexture, 0, D3D11_MAP_READ, 0, &mapped))) {
> +    // read the texture
> +    resultColor = *(int*)mapped.pData;
> +  } else {
> +    gfxCriticalError() << "DoesD3D11TextureSharingWorkMapFailed";

DoesD3D11TextureSharingWork_MapFailed
Attachment #8671639 - Flags: review?(bgirard)
Attachment #8671639 - Flags: review?(bas)
Attachment #8671639 - Flags: review+
Comment on attachment 8671639 [details] [diff] [review]
patch

Review of attachment 8671639 [details] [diff] [review]:
-----------------------------------------------------------------

::: gfx/thebes/gfxWindowsPlatform.cpp
@@ +1852,5 @@
>        nsString vendorID, vendorID2;
>        gfxInfo->GetAdapterVendorID(vendorID);
>        gfxInfo->GetAdapterVendorID2(vendorID2);
>        if (vendorID.EqualsLiteral("0x8086") && vendorID2.IsEmpty()) {
> +        gfxCriticalError(CriticalLog::DefaultOptions(false)) << "PossiblyBrokenSurfaceSharingUnexpectedAMDGPU";

Can we just put spaces in here? This is a string literal...

@@ +1872,5 @@
>    desc.CPUAccessFlags = 0;
>    desc.MiscFlags = D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX;
>    desc.BindFlags = bindflags;
> +
> +  int color[texture_size * texture_size];

nit: uint32_t

@@ +1878,5 @@
> +    color[i] = 0xff00ffff;
> +  }
> +  // We're going to check that sharing actually works with this format
> +  D3D11_SUBRESOURCE_DATA data;
> +  data.pSysMem = color;

Does this compile? Shouldn't this require a reinterpret cast?

@@ +1927,5 @@
> +  nsRefPtr<ID3D11DeviceContext> deviceContext;
> +  sharedResource->QueryInterface(__uuidof(IDXGIKeyedMutex), (void**)getter_AddRefs(sharedMutex));
> +  device->GetImmediateContext(getter_AddRefs(deviceContext));
> +  if (FAILED(sharedMutex->AcquireSync(0, 30*1000))) {
> +    gfxCriticalError() << "DoesD3D11TextureSharingWorkAcquireSyncTimeout";

Again.. no spaces?

@@ +1941,5 @@
> +  if (SUCCEEDED(deviceContext->Map(cpuTexture, 0, D3D11_MAP_READ, 0, &mapped))) {
> +    // read the texture
> +    resultColor = *(int*)mapped.pData;
> +  } else {
> +    gfxCriticalError() << "DoesD3D11TextureSharingWorkMapFailed";

Wait.. why do we hate spaces?

@@ +1944,5 @@
> +  } else {
> +    gfxCriticalError() << "DoesD3D11TextureSharingWorkMapFailed";
> +    return false;
> +  }
> +  deviceContext->Unmap(cpuTexture, 0);

Move unmap into the if SUCCEEDED

@@ +1952,5 @@
> +  // check that the color we put in is the color we get out
> +  if (resultColor != color[0]) {
> +    // Shared surfaces seem to be broken on dual AMD & Intel HW when using the
> +    // AMD GPU
> +    gfxCriticalNote << "SharedSurfaceArentShared";

And apostrophes too?
Attachment #8671639 - Flags: review?(bas) → review-
It compiles and the hate for spaces is because it's hard to search in crashstats I think.
(In reply to Benoit Girard (:BenWa) from comment #53)
> It compiles and the hate for spaces is because it's hard to search in
> crashstats I think.

Yeah, it's LPVOID I guess.

Why would a unique string with spaces be harder than a unique string without spaces?
(In reply to Bas Schouten (:bas.schouten) from comment #54)
> (In reply to Benoit Girard (:BenWa) from comment #53)
> > It compiles and the hate for spaces is because it's hard to search in
> > crashstats I think.
> 
> Yeah, it's LPVOID I guess.
> 
> Why would a unique string with spaces be harder than a unique string without
> spaces?

SuperSearch on CrashStats tokenizes by spaces so omitting spaces makes faceting by "Graphics Critical Error" much nicer.
By the way - gfxCriticalError(CriticalLog::DefaultOptions(false)) can be replaced with gfxCriticalNote for brevity.
(In reply to Jeff Muizelaar [:jrmuizel] from comment #55)
> 
> SuperSearch on CrashStats tokenizes by spaces so omitting spaces makes
> faceting by "Graphics Critical Error" much nicer.

True, but it's enough to have one unique string in there, and make the rest read better.

On a side note, if this is part of startup testing, David should comment on it.  There may be additional flags/prefs/telemetry/whatever to make it consistent.
Flags: needinfo?(dvander)
(In reply to Jeff Muizelaar [:jrmuizel] from comment #55)
> SuperSearch on CrashStats tokenizes by spaces so omitting spaces makes
> faceting by "Graphics Critical Error" much nicer.

Not sure if that's true for every field type. For App Notes, it is, but for other field, the ES indexing might be possible to be set differently.
Seems reasonable, we already send the result of DoesD3D11TextureSharingWork to Telemetry so we should be able to see if that or D3D11 uptake changes.
Flags: needinfo?(dvander)
I switched to using underscore. I'd rather not run our chances at having tokenizer problems and make sure we have an easier time searching.

Not sure what you mean by apostrophe. They are are hex 0x22 which is double quotes.
Assignee: jmuizelaar → bgirard
Attachment #8671639 - Attachment is obsolete: true
Status: REOPENED → ASSIGNED
Attachment #8672136 - Flags: review?(bas)
Attachment #8672136 - Flags: review?(bas) → review+
Bug 1097321 - Fix DoesD3D11TextureSharingWorkInternal to handle Intel/AMD configurations. r=Bas
Bug 1097321 - Add layers.amd-switchable-gfx.enabled pref. r=jrmuizel
Attachment #8676457 - Flags: review?(jmuizelaar)
Comment on attachment 8676457 [details]
MozReview Request: Bug 1097321 - Add layers.amd-switchable-gfx.enabled pref. r=jrmuizel

https://reviewboard.mozilla.org/r/22719/#review20185
Attachment #8676457 - Flags: review?(jmuizelaar) → review+
https://hg.mozilla.org/integration/mozilla-inbound/rev/5155bb5301f4138e28ab1df547e755f9afe01edf
Bug 1097321 - Fix DoesD3D11TextureSharingWorkInternal to handle Intel/AMD configurations. r=Bas

https://hg.mozilla.org/integration/mozilla-inbound/rev/ae31deae1e250a46be941c91d9112a45cd781246
Bug 1097321 - Add layers.amd-switchable-gfx.enabled pref. r=jrmuizel
https://hg.mozilla.org/mozilla-central/rev/5155bb5301f4
https://hg.mozilla.org/mozilla-central/rev/ae31deae1e25
Status: ASSIGNED → RESOLVED
Closed: 5 years ago4 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla44
Depends on: 1221348
You need to log in before you can comment on or make changes to this bug.