WebGL performance varies tremendously due to GPU blacklisting and users don’t know why

RESOLVED INACTIVE

Status

()

Core
Canvas: WebGL
RESOLVED INACTIVE
6 years ago
2 days ago

People

(Reporter: michaelbraithwaite, Unassigned)

Tracking

16 Branch
x86_64
Windows 7
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

6 years ago
User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4

Steps to reproduce:

* Run a WebGL sample maximized on a blacklisted driver, e.g. ATI 12.8 on Windows 7 


Actual results:

* Very low frame rate, eg ~8FPS v 60FPs without black list.


Expected results:

Broadly something to let the user know why its so slow.

If a user attempts to play a game while they are on drivers black listed for GPU composition they can experience very poor performance which they don’t know the cause of. We are concerned users will assume it is the title that is slow and have a negative view of it.

Developers or users need a way to understand why they are getting poor performance to be able to solve it. 

As an example when the Windows 7 ATI 12.8 drivers were blacklisted for GPU composition the performance of WebGL titles dropped tremendously. For a full screen (1920x1080) the performance on my GPU (AMD Radeon HD 6800) :-
* Before blacklist 60FPS
* After blacklist with default Angle  ~8FPS
* After blacklist with forced native GL ~14FPS

Yes 12.10 are now out and ATI do eventually prompt user to update but that is really not the point here.

A few possible actions:-
* Expose to JavaSctipt some of the GPU state information, maybe a subset of what is in about:support, so titles can warn users.
* The browser does something to tell the user they try to use WebGL on blacklisted drivers they may experience sever slowdown.
* Additionally can’t the slow paths be sped up to make it markedly faster? For Angle https://bugzilla.mozilla.org/show_bug.cgi?id=751113 would part fix it but that would only have been ~14FPS the above case.

Do you know what % of users hit these slow paths? The ATI 12.8 were the current Windows 7 version at the time.
We do have data on what percent of users should be hitting these paths. Improving slow paths can help, there's only so much we can do along those lines.

A warning to the user and/or app that the user will have degraded performance seem like it'd be a good thing to have.
Status: UNCONFIRMED → NEW
Ever confirmed: true
(Reporter)

Comment 3

6 years ago
A bit worryingly low % support, at least on Windows? I supposes there a good overlap of people who get both or neither of WebGL and Layers.

Are there thoughts/plans to try to increase the rate? Maybe more fine grained blacklist on GPU as well as driver initially? I know its only one anecdotal case and I doubt our code executes all code paths but 12.8 driver seemed stable for me on Firefox and Chrome ran fine on it too.

Comment 4

6 years ago
A performance warning from the browser and a recommendation to update the video card driver will be a very good thing indeed. As web developers it is hard for us to differentiate between low FPS because the browser decided to use a slow path vs. low FPS because the hardware is not fast enough. We do not have access to hardware information, the browser has.

At the moment we are seriously considering warning all Firefox users about possible low performance, encouraging then to either update their video card drivers or to switch to Google Chrome, but nobody benefits from something like that, the percentage of Firefox users on our site has dropped to 20% already. Both Chrome and Firefox blacklist drivers but so far Firefox is more often the one with a higher rate of failure for us on Windows.

It seems to us that browser developers are a bit trigger happy with blacklisting drivers, it is not that easy for users to update their drivers (on some laptops it is just not possible), so I wonder how much effort are you guys expending on making your code more stable against bad drivers? Before we were WebGL developers we were experienced D3D and OpenGL developers and we never had so many problems as we have now with WebGL, and it is quite frustrating for us because there is nothing we can do about it from our side.
(In reply to michaelbraithwaite from comment #3)
> A bit worryingly low % support, at least on Windows? I supposes there a good
> overlap of people who get both or neither of WebGL and Layers.

Yes, we too are worried about this, and indeed, most people get either the full thing (WebGL and Direct3D 10 Layers, needed for fast WebGL compositing) or nothing at all. That's because the blacklists and hw requirements are similar.

> 
> Are there thoughts/plans to try to increase the rate? Maybe more fine
> grained blacklist on GPU as well as driver initially?

Yes we are considering that. We started by relaxing the blacklist for D3D9 layers as a test in bug 792190. If that is conclusive, we'll try to generalize this driver version requirement relaxation to other features (D3D10 layers, WebGL).

Another approach is to provide a software fallback to WebGL, which Chrome does and we currently don't (bug 731983) but that comes with a whole new set of performance pitfalls.
(In reply to davidgaleano from comment #4)
> A performance warning from the browser and a recommendation to update the
> video card driver will be a very good thing indeed.

I don't know. Most of our users don't know what a driver is. Even if shown such a message, all they would do is look for the little 'x' button at the top-right and carry along.

Of course, there is a minority of our users that would properly take action upon such a message. And these users may well be the majority on a specific website like yours.

So yes, there certainly must exist a solution that will be helpful for your website without showing all users messages they don't care about, all the time, but that is nontrivial to design (note: many non-WebGL pages still create WebGL contexts, e.g. pages that use certain versions of modernizr.js. That includes mainstream newspapers such as economist.com, figaro.fr etc).

> As web developers it is
> hard for us to differentiate between low FPS because the browser decided to
> use a slow path vs. low FPS because the hardware is not fast enough. We do
> not have access to hardware information, the browser has.
> 
> At the moment we are seriously considering warning all Firefox users about
> possible low performance, encouraging then to either update their video card
> drivers or to switch to Google Chrome, but nobody benefits from something
> like that,

Good thing you acknowledge it --- I'll remind you of that if you do it.

> the percentage of Firefox users on our site has dropped to 20%
> already. Both Chrome and Firefox blacklist drivers but so far Firefox is
> more often the one with a higher rate of failure for us on Windows.

Yes, that is a combination of not shipping a WebGL software fallback, and having a more conservative blacklist (requiring a mid-2010 driver instead of a early-2009 driver). See previous comment, we are considering relaxing this a bit, but this takes time as we have to let such change go through alpha+beta channels to ensure we don't cause an increase in crashiness.

> 
> It seems to us that browser developers are a bit trigger happy with
> blacklisting drivers,

I would dispute this "trigger happy" qualification. If that were true, we's simply blacklist any crashy driver without investigation. Instead, we do investigate any such bug and spend hours looking for work-arounds before we consider blacklisting. We also live with unfixed driver crashers when they are low-volume enough and not security sensitive.

> it is not that easy for users to update their drivers
> (on some laptops it is just not possible), so I wonder how much effort are
> you guys expending on making your code more stable against bad drivers?

The answer is "tons of effort" even with the current level of blacklisting. Yet even so we are considering lowering the blacklists, but this is highly nontrivial.

> Before we were WebGL developers we were experienced D3D and OpenGL
> developers and we never had so many problems as we have now with WebGL,

Did your application have a crash reporter with enough details to allow investigating driver-related crashes, something like this,
https://crash-stats.mozilla.com/report/index/a828ee08-4a72-4c5b-bb38-e6ccb2121028

If you did have such a thing, there remains the possibility that your users were far more technical than Firefox users therefore had better drivers. Again we do want to cater to such users here but, as explained at the beginning of this comment, it is nontrivial to do so in a way that isn't detrimental to the majority of less technical users and non-WebGL pages.

Ideas that would take this into account, would be very welcome.

> and
> it is quite frustrating for us because there is nothing we can do about it
> from our side.

Understood.
Proposal: add a Web API allowing pages to opt *in* to make the browser show the user a message when their WebGL context creation fails.
(Reporter)

Comment 8

6 years ago
(In reply to Benoit Jacob [:bjacob] from comment #7)
> Proposal: add a Web API allowing pages to opt *in* to make the browser show
> the user a message when their WebGL context creation fails.

We can and do detect the context failing to create so will this give more information?

There would be the issue of it running but slowly with gpu layers disabled and, if implemented, a software rasterizer maybe well be too slow for anything complex or/and a large canvas. 

Perhaps this message could be shown if one of the gpu accelerated features is disabled? e.g. developer pass in some flags to warn about if disabled, or feature string to check? eg mozCheckGPUAcceleration("WebGL");
(In reply to michaelbraithwaite from comment #8)
> (In reply to Benoit Jacob [:bjacob] from comment #7)
> > Proposal: add a Web API allowing pages to opt *in* to make the browser show
> > the user a message when their WebGL context creation fails.
> 
> We can and do detect the context failing to create so will this give more
> information?

It could give a little bit more information i.e. the difference between just telling the user "go update your driver" and telling them "Here is the GPU that you have. Here is the website of your GPU vendor. Go there."

But yes, I was losing focus of the thing you actually asked about here.


> There would be the issue of it running but slowly with gpu layers disabled
> and, if implemented, a software rasterizer maybe well be too slow for
> anything complex or/and a large canvas.
> 
> Perhaps this message could be shown if one of the gpu accelerated features
> is disabled? e.g. developer pass in some flags to warn about if disabled, or
> feature string to check? eg mozCheckGPUAcceleration("WebGL");

Yes, good idea. We could have a few variants of that message and suggest people to update their driver even if WebGL context creation succeeds.

That could take the form of WebGL context creation attributes, warnOnCreationFailure and warnOnLowPerformance.

canvas.getContext("experimental-webgl", {warnOnCreationFailure: true, warnOnLowPerformance: true})

Comment 10

6 years ago
It is all about user experience, if your blacklist makes the game run at unusable FPS then someone should warn the user about it and we game developers do not always know the reasons. Sometimes it is better to fail than to emulate and provide a poor user experience, I do not think providing a software rasterizer is a good idea for games. Your warning flags for getContext are an acceptable solution. We already discussed a similar subject on the WebGL mailing list several months ago and nothing was done about it.

At the end of the day site creators want to provide the best experience possible and if that means we need to recommend using a different browser then we will do it. We already recommended to not use old browser versions and to use either Chrome or Fifefox for a plugin free experience. If the games run slow on Firefox then we have no option than to recommend a switch. At a recent conference several developers asked me if they should focus on only supporting Chrome for their games, my answer was that they should code defensibly and support as many browsers as possible, that Firefox was getting faster with every release; but I do understand the cost of testing and supporting so many browsers on so many different platforms and how tempting is to sacrifice a low percentage of your visitors for an easier life.

Desktop games for ages had to face the same driver issues the browsers are now facing, some of them provided tools and links to help experienced user, the less experienced ones could always call their grand-sons for help. I still consider "trigger happy" blacklisting drivers in bulk. An old driver may actually be more stable than a recent one, some drivers only broke specific APIs that were easy to workaround by not using that API or by using it in a slightly different way, etc. You could do blacklisting at a finer level than what are you doing now.

I really appreciate that you are trying to help but progress feels very slow. Is it the problem that Mozilla is a big organization with too many groups pulling in different directions?

Btw, we are not having a go at Firefox, Chrome has its shares of issues too, we are just trying to provide the best browser gaming experience and we do need Firefox to be good.
(In reply to davidgaleano from comment #10)
> I really appreciate that you are trying to help but progress feels very
> slow. Is it the problem that Mozilla is a big organization with too many
> groups pulling in different directions?

No, really not. It's more that this hasn't been prioritized (i.e. worked on) at all until now, but hearing specific requests from developers definitely helps with prioritizing. And also, the discussions we had in the past were about fixing this by adding a content-facing API, which requires much more coordination -- needs to be discussed at the level of the WG, etc. The new approach, bug 806359, should be much more quickly actionable. I probably won't have time for it this week, but it should only require one day of work.
Summary: WebGL performance varies tremendously due to GPU blacklisting and user don’t know why → WebGL performance varies tremendously due to GPU blacklisting and users don’t know why

Comment 12

2 days ago
Per policy at https://wiki.mozilla.org/Bug_Triage/Projects/Bug_Handling/Bug_Husbandry#Inactive_Bugs. If this bug is not an enhancement request or a bug not present in a supported release of Firefox, then it may be reopened.
Status: NEW → RESOLVED
Last Resolved: 2 days ago
Resolution: --- → INACTIVE
You need to log in before you can comment on or make changes to this bug.