1086685 - WebGL success rates down since Firefox 33 compared to Firefox 32

Status: RESOLVED WORKSFORME

(Core :: Graphics: CanvasWebGL, defect)

Description

[Benoit Jacob [:bjacob] (mostly away)]

Looking at percentage, among Windows crash reports, of 'WebGL+' (successfully initialized WebGL) among 'WebGL?' (attempted to initialize WebGL).

Sample command line (for Firefox 32):

for d in `seq -w 14 20`; do echo -n "October $d: " && echo "1000*`zcat 201410$d-pub-crashdata.csv.gz | grep -P 'Firefox\t32\.0' | grep -P '\trelease\t' | grep -F 'WebGL+' | grep 'Windows.NT' | wc -l`/`zcat 201410$d-pub-crashdata.csv.gz | grep -P 'Firefox\t32\.0' | grep -P '\trelease\t' | grep -F 'WebGL?' | grep 'Windows.NT' | wc -l`" | bc; done

Result:

Date    | Firefox 32 WebGL success rate | Firefox 33 WebGL success rate |
--------+-------------------------------+-------------------------------+
Oct. 14 | 88.3%                         | 67.7%                         |
Oct. 15 | 88.2%                         | 74.9%                         |
Oct. 16 | 87.6%                         | 75.2%                         |
Oct. 17 | 87.8%                         | 75.5%                         |
Oct. 18 | 86.9%                         | 74.9%                         |
Oct. 19 | 86.3%                         | 75.0%                         |
Oct. 20 | 87.2%                         | 76.4%                         |

Explanation:

Firefox 33 ships with new blacklist entries that block all features on many more Intel drivers than previously, see these 4 changesets:

changeset:   211343:0970007a8468
user:        Nicolas Silva <[e-mail]>
date:        Fri Oct 17 07:14:00 2014 -0400
summary:     Bug 1083071 - Add some old intel drivers to the blocklist. r=Bas

changeset:   208492:13ba124946bd
user:        Nicolas Silva <[e-mail]>
date:        Thu Oct 02 19:31:27 2014 +0200
summary:     Bug 1074378 - Blocklist driver Intel GMAX4500HD v 8,15,10,1749. r=Bas

changeset:   207954:a96a47727207
user:        Bas Schouten <[e-mail]>
date:        Mon Sep 29 17:57:54 2014 +0200
summary:     Bug 1060736: Disable all acceleration on HD 3000 for driver versions 2321 or lower. r=nical

Worse things to come:

Firefox 33.0.1 has this additional changeset that landed yesterday, that will block many more users still:

changeset:   211352:fba30348f0c6
user:        Nicolas Silva <[e-mail]>
date:        Mon Oct 20 08:37:00 2014 -0400
summary:     Bug 1083071 - Blacklist device family IntelGMAX4500HD drivers older than 7-19-2011 because of OMTC issues on Windows. r=Bas

With that, I expect that WebGL success rates will dip *** below 70% ***.

Comment 1

[Benoit Jacob [:bjacob] (mostly away)]

Note: the reason why the fall is less deep on these graphs,
http://people.mozilla.org/~bjacob/gfx_features_stats/
is that these measure uniformly all users on all Firefox versions, and Firefox 33 hasn't been unthrottled yet, so only a minority of users are already using it. If Firefox 33 gets fully released and unthrottled then expect these graphs to drop as described above.

Comment 2

[Benoit Jacob [:bjacob] (mostly away)]

Hrm, I still don't see an improvement in Firefox 33.0.2, but here's hoping that that's just because of blacklisting decision caching.

Recording my more grown-up command line for future usage:

bjacob@bjacob-dell:~/hack/gfx-stats$ version='33.0.2'; for d in `seq -w 22 28`; do echo -n "October $d: " && echo "`zcat 201410$d-pub-crashdata.csv.gz | grep -P "Firefox\t$version" | grep -P '\trelease\t' | grep -F 'WebGL+' | grep 'Windows.NT' | wc -l` `zcat 201410$d-pub-crashdata.csv.gz | grep -P "Firefox\t$version" | grep -P '\trelease\t' | grep -F 'WebGL?' | grep 'Windows.NT' | wc -l`" | awk '{ printf("%.1f %%\n", 100*$1/$2); }' ; done
October 22: 75.3 %
October 23: 76.6 %
October 24: 75.2 %
October 25: 74.4 %
October 26: 76.0 %
October 27: 75.5 %
October 28: 75.7 %

Comment 3

[Benoit Jacob [:bjacob] (mostly away)]

Getting worse and at this point I have no idea what is causing it, since the above-mentioned blacklist entries have in principle been backed out.

Comment 4

[Lawrence Mandel [:lmandel] (use needinfo)]

I reviewed this with Jeff. As bjacob said in comment 3, we don't know what's causing this and won't have a fix for 34. I have tracked for 35 and 36 so that we remember to follow up.

Comment 5

[Milan Sreckovic [:milan] (needinfo for best results)]

Jeff, Dan, let's find out what's going on here.

Comment 6

[Lukas Blakk [:lsblakk] use ?needinfo]

Since there's nothing new to go on in this bug and given both the time of year and that we're too late in FF35 for speculative fixes, wontfixing for this release  but since there's an assignee, leaving tracking for 36/37 to try and get some investigation and possible fix here.

Comment 7

[Sylvestre Ledru [:Sylvestre]]

Looks like it is going to be a wontfix for 36 too.

Comment 8

[Milan Sreckovic [:milan] (needinfo for best results)]

Right, this is a meta bug, it isn't clear how to make it actionable.

Comment 9

[Lawrence Mandel [:lmandel] (use needinfo)]

As this bug is in-actionable in it's current state and we don't have any leads, at this point it looks like a possible wontfix. At any rate, I'm dropping tracking because I don't think this is a high priority item for the 37 release.

Comment 10

[Benoit Jacob [:bjacob] (mostly away)]

This bug is actionable by investigating:
 1) whether the downward trend continues, has stabilized, or has reversed.
 2) if the WebGL success rate is not yet back to normal, given the scale of the phenomenon observed above, it should be possible to find a machine reproducing this issue and use it to understand its causes.

Not claiming that this bug should be prioritized - but it is definitely actionable if prioritized.

Comment 11

[Benoit Jacob [:bjacob] (mostly away)]

Nothing meta around here.

Comment 12

[Benoit Jacob [:bjacob] (mostly away)]

The problem was somehow resolved as of Firefox 36.

$ version='36'; for d in `seq -w 28 31`; do echo -n "March $d: " && echo "`zcat 201503$d-pub-crashdata.csv.gz | grep -P "Firefox\t$version" | grep -P '\trelease\t' | grep -F 'WebGL+' | grep 'Windows.NT' | wc -l` `zcat 201503$d-pub-crashdata.csv.gz | grep -P "Firefox\t$version" | grep -P '\trelease\t' | grep -F 'WebGL?' | grep 'Windows.NT' | wc -l`" | awk '{ printf("%.1f %%\n", 100*$1/$2); }' ; done
March 28: 82.3 %
March 29: 83.4 %
March 30: 83.0 %
March 31: 84.4 %

Comment 13

[Benoit Jacob [:bjacob] (mostly away)]

Ah no, in Firefox 32 we used to get 88%, see comment 0. The 83% we're now getting in Firefox 36 is better than the 75% that Firefox 33 got, but it's still a solid 5 percentage points below where we used to be.

Comment 14

[Milan Sreckovic [:milan] (needinfo for best results)]

It is also possible that we now crash more on configurations that also don't support WebGL?

Comment 15

[Benoit Jacob [:bjacob] (mostly away)]

The simplest explanation for the increase back from 75% to 83%, is that this is roughly the natural pace of increase we've continuously observed for WebGL success rates; in other words, we never recovered from the loss of WebGL success rate from Firefox 33, only time is making up for it. Without that loss, we'd be well above 90% today.

Comment 16

[u279076]

According to http://people.mozilla.org/~bgirard/gfx_features_stats/ we're around 87% success as of July 2015. Benoit, could you update your chart on people.m.o to see if this is still an issue?

Comment 17

[Benoit Girard (:BenWa)]

Attachment: Latest webgl data (Data is suspicious)

Here's the latest data that I have. It's still local only because starting in June 2015 the XP population seems to drop by 66%. So it's causing a lot of numbers to be funny including the WebGL success rate which was likely below average on XP.

I would take this data with a grain of salt.

Comment 18

[u279076]

Thanks Benoit. There doesn't seem to be anything actionable here. Is there something I can help with to make this actionable? If not, should we just close this bug as incomplete?

Comment 19

[Benoit Girard (:BenWa)]

We did have a dip in Oct 2014 but it correct itself before our data got suspicious. I don't think there's any value to had in investigating this bug given the trend corrected itself.