884226 - Consider enabling authors to force usage of a software canvas to enable fast read backs

Status: RESOLVED FIXED

(Core :: Graphics: Canvas2D, defect)

Description

[Till Schneidereit [:till]]

With accelerated Canvas backends, using getImageData() or toImageData() becomes very slow, which is a problem for some scenarios where Canvas is used.

Apparently, there have been discussions about how to solve this, but no bug has been filed about it.

From IRC:
<jrmuizel> till: the only real solution we've been throwing around is giving authors a way to specify that they want a software canvas

Comment 1

[Boris Zbarsky [:bzbarsky]]

There has also been talk of detecting when lots of readbacks are happening and automatically falling back to software...

Comment 2

[Robert O'Callahan (:roc) (email my personal email if necessary)]

(In reply to Boris Zbarsky (:bz) from comment #1)
> There has also been talk of detecting when lots of readbacks are happening
> and automatically falling back to software...

We should do that. Probably just on the first readback.

Comment 3

[Jeff Muizelaar [:jrmuizel]]

(In reply to Robert O'Callahan (:roc) (Mozilla Corporation) from comment #2)
> (In reply to Boris Zbarsky (:bz) from comment #1)
> > There has also been talk of detecting when lots of readbacks are happening
> > and automatically falling back to software...
> 
> We should do that. Probably just on the first readback.

I would prefer an approach that gives authors more predictable performance instead of just trying to guess. I'm also not a huge fan of the complexity of handling backend switches.

Comment 4

[Robert O'Callahan (:roc) (email my personal email if necessary)]

The problem is that some content (or just existing content) is not going to use the new API. Then our performance will suffer compared to a competitor that does use a heuristic.

Comment 5

[matteo sisti sette]

Google Chrome doesn't exhibit this issue on Ubuntu 12.10 (same box where FF does). Does this imply that Chrome doesn't have an "accelerated canvas backend"? Because that sounds unlikely to me. Is there a way I can quickly test this?

Comment 6

[Boris Zbarsky [:bzbarsky]]

> Does this imply that Chrome doesn't have an "accelerated canvas backend"? 

On Linux, that's correct, in many cases.  They do software rendering via Skia instead.

Comment 7

[Jeff Muizelaar [:jrmuizel]]

(In reply to Robert O'Callahan (:roc) (Mozilla Corporation) from comment #4)
> The problem is that some content (or just existing content) is not going to
> use the new API. Then our performance will suffer compared to a competitor
> that does use a heuristic.

Sure, but if we avoid using a heuristic at least we don't start that arms race. If we add a flag for this we give authors more control and create the potential for content that works better in FF than other browsers. This can help create an arms race towards a better solution.

Comment 8

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

(In reply to Jeff Muizelaar [:jrmuizel] from comment #7)
> (In reply to Robert O'Callahan (:roc) (Mozilla Corporation) from comment #4)
> > The problem is that some content (or just existing content) is not going to
> > use the new API. Then our performance will suffer compared to a competitor
> > that does use a heuristic.
> 
> Sure, but if we avoid using a heuristic at least we don't start that arms
> race. If we add a flag for this we give authors more control and create the
> potential for content that works better in FF than other browsers. This can
> help create an arms race towards a better solution.

Yep, I agree.  Adding a flag (context creation flags, we already have a mechanism!) does not preclude a heuristic when the flag isn't present, but we should give developers the tools they need to be explicit if we can.

Comment 9

[James Willcox (:snorp) ([e-mail]) (he/him)]

I like having both a heuristic and a flag. I'll try to get a patch up soon.

Comment 11

[Boris Zbarsky [:bzbarsky]]

Note the opposite proposal in https://www.w3.org/Bugs/Public/show_bug.cgi?id=23773 by the way.  We should probably respond to that somehow...

Comment 12

[James Willcox (:snorp) ([e-mail]) (he/him)]

Attachment: Demote hardware accelerated canvases when frequent readbacks are predicte

Comment 13

[George Wright (:gw280) (needinfo me!)]

Falling back on a single put/getImageData is not the solution imho. I don't think this is landable, because most things (as far as I can tell) use put/getImageData at least once.

Comment 14

[Jeff Muizelaar [:jrmuizel]]

(In reply to George Wright (:gw280) from comment #13)
> Falling back on a single put/getImageData is not the solution imho. I don't
> think this is landable, because most things (as far as I can tell) use
> put/getImageData at least once.

I'm surprised by this claim. I would expect that most things either do no put/get or lots.

Comment 15

[George Wright (:gw280) (needinfo me!)]

(In reply to Jeff Muizelaar [:jrmuizel] from comment #14)
> (In reply to George Wright (:gw280) from comment #13)
> > Falling back on a single put/getImageData is not the solution imho. I don't
> > think this is landable, because most things (as far as I can tell) use
> > put/getImageData at least once.
> 
> I'm surprised by this claim. I would expect that most things either do no
> put/get or lots.

We should find out what the impact is then before we land this.

Comment 16

[matteo sisti sette]

For god's sake don't forget the flag, or whatever mechanism to allow the author to force a choice.
Having an automatic selection by default based on an heuristic is great, but it can not, in any case, be a complete solution, no matter how good the heuristic. Giving the developer (meaning the js developer, i.e. the author) the ability to choose is an absolute MUST.

Comment 17

[James Willcox (:snorp) ([e-mail]) (he/him)]

(In reply to matteo sisti sette from comment #16)
> For god's sake don't forget the flag, or whatever mechanism to allow the
> author to force a choice.
> Having an automatic selection by default based on an heuristic is great, but
> it can not, in any case, be a complete solution, no matter how good the
> heuristic. Giving the developer (meaning the js developer, i.e. the author)
> the ability to choose is an absolute MUST.

I think we want this too, but that will require some more coordination with spec people. We can get the heuristic in immediately.

Comment 18

[James Willcox (:snorp) ([e-mail]) (he/him)]

(In reply to George Wright (:gw280) from comment #15)
> (In reply to Jeff Muizelaar [:jrmuizel] from comment #14)
> > (In reply to George Wright (:gw280) from comment #13)
> > > Falling back on a single put/getImageData is not the solution imho. I don't
> > > think this is landable, because most things (as far as I can tell) use
> > > put/getImageData at least once.
> > 
> > I'm surprised by this claim. I would expect that most things either do no
> > put/get or lots.
> 
> We should find out what the impact is then before we land this.

We should, but that's going to be hard. We already understand where this is going to be faster/slower than the current behavior. What we don't know is what percentage of canvas users in the wild will hit the slow path (e.g. drawImage() magically becomes slow because you did a toBlob())

I think we do know that the current readback behavior is slow in common cases. I guess it's fairly standard to create a sprite sheet with canvas and use that all over the place with drawImage(). It's worth noting that this specific case should be MUCH better once we get all SkiaGL canvases in one GLContext.

Comment 19

[George Wright (:gw280) (needinfo me!)]

Comment on attachment 8334882 [details] [diff] [review]
Demote hardware accelerated canvases when frequent readbacks are predicte

Review of attachment 8334882 [details] [diff] [review]:
-----------------------------------------------------------------

I really don't think this is the way we want to go. Speaking to jrmuizel, we don't see any urgency here, so we'd rather spec out an explicit method for a content provider to signal the use of a software backed canvas instead.

Comment 20

[Robert O'Callahan (:roc) (email my personal email if necessary)]

So what's the plan here? Specify and implement this explicit signal, and evangelize every developer who has ever used getImageData/putImageData to update their site/app?

Comment 21

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Sorry, that was unnecessarily snarky. But I am genuinely concerned about how a new explicit signal is going to help us deal with the mass of content already out there.

Comment 22

[James Willcox (:snorp) ([e-mail]) (he/him)]

(In reply to Robert O'Callahan (:roc) (Mozilla Corporation) from comment #21)
> Sorry, that was unnecessarily snarky. But I am genuinely concerned about how
> a new explicit signal is going to help us deal with the mass of content
> already out there.

Yeah, we need the heuristic no matter what. Jeff made a good point about the flag to me yesterday -- that it should just set the initial state. So if content says "I'm going to do a lot of readback", you can choose the initial backend appropriately. If it turns out they lied and aren't actually doing that much, you can switch later. The inverse, of course, also applies.

Comment 23

[matteo sisti sette]

OMFG.
If that's going to happen, then we'll need two flags. One for the initial state, and one for whether the initial state can be changed by Google Chrome on its own initiative or not.

There always MUST be a way for the author to force a given type of backend.

Comment 24

[Till Schneidereit [:till]]

(In reply to matteo sisti sette from comment #23)
> There always MUST be a way for the author to force a given type of backend.

Can you back this claim up with some argument? Ideally one that involves describing a scenario where the heuristic with an author-selected initial value would do very bad things, and a comparison of the likelyhood of that compared to the author making wrong decisions based on assumptions about how the various browsers' graphics backends work.

This last point is important, because it's easy to make wrong assumptions about this. Or, to make assumptions that hold for now, maybe even in all browsers, but won't always.

Comment 25

[James Willcox (:snorp) ([e-mail]) (he/him)]

Attachment: Add 'mozReadbackHint' getContext() option for 2D canva

This adds a flag indicating that content is going to be doing frequent readback. Right now this will also force a software Skia context instead of SkiaGL. Other graphics backends are unaffected.

Example usage:

canvas.getContext('2d', { mozReadbackHint: true });

Comment 26

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

"mozWillUseGetImageDataHint"

Comment 27

[James Willcox (:snorp) ([e-mail]) (he/him)]

(In reply to Vladimir Vukicevic [:vlad] [:vladv] from comment #26)
> "mozWillUseGetImageDataHint"

there are other readback ops besides that one...

Comment 28

[James Willcox (:snorp) ([e-mail]) (he/him)]

maybe mozWillReadHint?

Comment 29

[(no longer active)]

Can somebody explain to me why we think this is a good idea?  I tend to agree with roc, I see very little point in exposing a flag here to solve a problem that already affects web content which will not change.

Comment 30

[James Willcox (:snorp) ([e-mail]) (he/him)]

(In reply to :Ehsan Akhgari (needinfo? me!) from comment #29)
> Can somebody explain to me why we think this is a good idea?  I tend to
> agree with roc, I see very little point in exposing a flag here to solve a
> problem that already affects web content which will not change.

I mostly agree as well. We are going to do heuristics too, but need this flag to do something in the mean time. Once we have the heuristics, this flag will just be a suggestion or help set the initial state.

Comment 31

[(no longer active)]

Two more questions:

1. What about comment 11?
2. Have we talked to any other browser vendor about this?  Is anybody else interested in implementing such a hint?  If yes, please provide a link to the discussion.  Thanks!

Comment 32

[(no longer active)]

(In reply to comment #30)
> (In reply to :Ehsan Akhgari (needinfo? me!) from comment #29)
> > Can somebody explain to me why we think this is a good idea?  I tend to
> > agree with roc, I see very little point in exposing a flag here to solve a
> > problem that already affects web content which will not change.
> 
> I mostly agree as well. We are going to do heuristics too, but need this flag
> to do something in the mean time. Once we have the heuristics, this flag will
> just be a suggestion or help set the initial state.

I don't understand why we can't implement the heuristic right now!

Comment 33

[(no longer active)]

(In reply to comment #30)
> (In reply to :Ehsan Akhgari (needinfo? me!) from comment #29)
> > Can somebody explain to me why we think this is a good idea?  I tend to
> > agree with roc, I see very little point in exposing a flag here to solve a
> > problem that already affects web content which will not change.
> 
> I mostly agree as well. We are going to do heuristics too, but need this flag
> to do something in the mean time. Once we have the heuristics, this flag will
> just be a suggestion or help set the initial state.

Also, exposing an API for something which we know is only going to be a temporary measure is a bad idea!

Comment 34

[James Willcox (:snorp) ([e-mail]) (he/him)]

(In reply to :Ehsan Akhgari (needinfo? me!) from comment #32)
> (In reply to comment #30)
> > (In reply to :Ehsan Akhgari (needinfo? me!) from comment #29)
> > > Can somebody explain to me why we think this is a good idea?  I tend to
> > > agree with roc, I see very little point in exposing a flag here to solve a
> > > problem that already affects web content which will not change.
> > 
> > I mostly agree as well. We are going to do heuristics too, but need this flag
> > to do something in the mean time. Once we have the heuristics, this flag will
> > just be a suggestion or help set the initial state.
> 
> I don't understand why we can't implement the heuristic right now!

If you can describe such a thing precisely, we'll do it :)

Comment 35

[James Willcox (:snorp) ([e-mail]) (he/him)]

(In reply to :Ehsan Akhgari (needinfo? me!) from comment #31)
> Two more questions:
> 
> 1. What about comment 11?

I disagree with the proposed attribute. We should definitely respond.

> 2. Have we talked to any other browser vendor about this?  Is anybody else
> interested in implementing such a hint?  If yes, please provide a link to
> the discussion.  Thanks!

Not formally, but I think the Chrome folks may have something in mind. I'll ask.

Comment 36

[James Willcox (:snorp) ([e-mail]) (he/him)]

(In reply to :Ehsan Akhgari (needinfo? me!) from comment #33)
> (In reply to comment #30)
> > (In reply to :Ehsan Akhgari (needinfo? me!) from comment #29)
> > > Can somebody explain to me why we think this is a good idea?  I tend to
> > > agree with roc, I see very little point in exposing a flag here to solve a
> > > problem that already affects web content which will not change.
> > 
> > I mostly agree as well. We are going to do heuristics too, but need this flag
> > to do something in the mean time. Once we have the heuristics, this flag will
> > just be a suggestion or help set the initial state.
> 
> Also, exposing an API for something which we know is only going to be a
> temporary measure is a bad idea!

It's not temporary, the affects will just (possibly) be different down the road. The hint is just that, a hint. If it turns out you aren't *actually* doing a bunch of readback then we could do whatever thing is fastest.

Comment 37

[Jeff Muizelaar [:jrmuizel]]

FWIW, we did talk with Chrome guys about this. Justin Novosad proposed waiting to see how the heuristic but most people at Mozilla agree that we should try the flag first.

Comment 38

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

(In reply to Jeff Muizelaar [:jrmuizel] from comment #14)
> (In reply to George Wright (:gw280) from comment #13)
> > Falling back on a single put/getImageData is not the solution imho. I don't
> > think this is landable, because most things (as far as I can tell) use
> > put/getImageData at least once.
> 
> I'm surprised by this claim. I would expect that most things either do no
> put/get or lots.

Canvas game (2D or webgl), never does readback -- but wants to add a "screenshot" function for the user.

Comment 39

[(no longer active)]

Comment on attachment 8348947 [details] [diff] [review]
Add 'mozReadbackHint' getContext() option for 2D canva

Review of attachment 8348947 [details] [diff] [review]:
-----------------------------------------------------------------

I'm having a hard time understanding everything that has happened here after comment 24 so let's assume that we're convinced that we should add this hint here (and it's a good idea and not a potential footgun etc.)  Here's my take on this assuming that:

1. We should implement it behind a pref and let authors test it and gather feedback on whether they find it useful.
2. We should work on specifying it and get other vendors interested in implementing it.  Gathering feedback from a hidden behind a pref implementation may be a good way of convincing other vendors.
3. Let's not use a moz prefix since that won't buy us anything, and we're not shipping moz prefixes in new APIs any more.

r- for the moz prefix and the fact that this is not hidden behind a pref.

Comment 40

[(no longer active)]

(In reply to James Willcox (:snorp) ([e-mail]) from comment #35)
> (In reply to :Ehsan Akhgari (needinfo? me!) from comment #31)
> > Two more questions:
> > 
> > 1. What about comment 11?
> 
> I disagree with the proposed attribute. We should definitely respond.

Thanks, please do that!

Comment 41

[(no longer active)]

(In reply to Vladimir Vukicevic [:vlad] [:vladv] from comment #38)
> (In reply to Jeff Muizelaar [:jrmuizel] from comment #14)
> > (In reply to George Wright (:gw280) from comment #13)
> > > Falling back on a single put/getImageData is not the solution imho. I don't
> > > think this is landable, because most things (as far as I can tell) use
> > > put/getImageData at least once.
> > 
> > I'm surprised by this claim. I would expect that most things either do no
> > put/get or lots.
> 
> Canvas game (2D or webgl), never does readback -- but wants to add a
> "screenshot" function for the user.

So in that case, should the author specify this hint or not?  I'm a bit worried that it would be very hard for authors to make a good decision here.

Comment 42

[James Willcox (:snorp) ([e-mail]) (he/him)]

(In reply to :Ehsan Akhgari (needinfo? me!) from comment #41)
> (In reply to Vladimir Vukicevic [:vlad] [:vladv] from comment #38)
> > (In reply to Jeff Muizelaar [:jrmuizel] from comment #14)
> > > (In reply to George Wright (:gw280) from comment #13)
> > > > Falling back on a single put/getImageData is not the solution imho. I don't
> > > > think this is landable, because most things (as far as I can tell) use
> > > > put/getImageData at least once.
> > > 
> > > I'm surprised by this claim. I would expect that most things either do no
> > > put/get or lots.
> > 
> > Canvas game (2D or webgl), never does readback -- but wants to add a
> > "screenshot" function for the user.
> 
> So in that case, should the author specify this hint or not?  I'm a bit
> worried that it would be very hard for authors to make a good decision here.

They would definitely not want this flag. I'll change it to 'frequentReadHint' maybe?

Comment 43

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

frequentGetImageData.  Be very explicit about what operation it's optimizing for.

Comment 44

[James Willcox (:snorp) ([e-mail]) (he/him)]

(In reply to Vladimir Vukicevic [:vlad] [:vladv] from comment #43)
> frequentGetImageData.  Be very explicit about what operation it's optimizing
> for.

But there are a ton of other operations that are similarly affected. Why not call it frequentToBlob?

Comment 45

[James Willcox (:snorp) ([e-mail]) (he/him)]

Attachment: Add 'willReadFrequently' getContext() option for 2D canvas

Comment 46

[James Willcox (:snorp) ([e-mail]) (he/him)]

Comment on attachment 8349036 [details] [diff] [review]
Add 'willReadFrequently' getContext() option for 2D canvas

This changes the option to 'willReadFrequently', and guards the enforcement with a bool pref 'gfx.canvas.willReadFrequently.enable'.

Comment 47

[Matt Woodrow (:mattwoodrow)]

Comment on attachment 8349036 [details] [diff] [review]
Add 'willReadFrequently' getContext() option for 2D canvas

Review of attachment 8349036 [details] [diff] [review]:
-----------------------------------------------------------------

I don't know anywhere enough about webidl to be a useful reviewer for this patch sorry.

Comment 48

[(no longer active)]

(Please don't consider my r+ as code-review!)

Comment 49

[George Wright (:gw280) (needinfo me!)]

Comment on attachment 8349036 [details] [diff] [review]
Add 'willReadFrequently' getContext() option for 2D canvas

Review of attachment 8349036 [details] [diff] [review]:
-----------------------------------------------------------------

::: content/canvas/src/CanvasRenderingContext2D.cpp
@@ +1064,5 @@
> +
> +  ContextAttributes2D attributes;
> +  NS_ENSURE_TRUE(attributes.Init(aCx, aOptions), NS_ERROR_UNEXPECTED);
> +
> +#if USE_SKIA_GPU

ifdef

Comment 50

[James Willcox (:snorp) ([e-mail]) (he/him)]

Fixed the s/if/ifdef/ and pushed.

https://hg.mozilla.org/integration/mozilla-inbound/rev/48f72b096106

Comment 51

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/48f72b096106

Comment 52

[matteo sisti sette]

So does this finally just give the author a way to specify HINTS for the INITIAL setting, which may at the end overridden by the heuristic?
Or does it also provide a way to force a given backend with 100% guarantee that it will be honored (if supported)?

Comment 53

[Till Schneidereit [:till]]

This still only gives a hint. As I said in comment 24, you really must make a convincing argument for a non-heuristic parameter. Just saying that it must exist won't do.

Comment 54

[matteo sisti sette]

Why don't you try to make a convincing argument for non having it? What if saying that you don't need it won't do?

Comment 55

[matteo sisti sette]

Do you really believe the heuristic will give the best choice in 100% of cases? Write down the heuristic and anybody will be able to write a counterexample showing a program that runs slower with the backend chosen by the heuristic. Do we have to waste our time doing that?

Comment 56

[matteo sisti sette]

And here's a simple use case: somebody wants to create a web page that demonstrates how hardware acceleration makes rendering faster, so he creates a page with a software canvas and a hardware-accelerated one running the same code so the visitor can appreciate the difference.


By the way, why not allow even to programmatically switch? (given that runtime switching is already possible, the engine already does that based on the heuristic, right?)

Comment 57

[matteo sisti sette]

"it's easy to make wrong assumptions about this. Or, to make assumptions that hold for now, maybe even in all browsers, but won't always."
It's even easier to make wrong assumptions about javascript code using the canvas, or assumptions that hold for all the javascript code that has ever been written, but won't always for all the code to be written in the future.
What about the unfortunate author that incurs in the very unlikely case? Will you tell him to just accept that his code has to run slower because he's doing something too unusual? 
If you're providing a way to specify a hint, which sets an initial value that may be overridden by the heuristic, why don't you want to allow to set a flag to force that initial value to be kept? If the heuristic is already there and the mechanism to provide a flag is already there (you're using that for the hint), providing the possibility to make the hint "imperative" should be zero effort: why do you want the machine to be arrogant at all cost?

Comment 58

[Till Schneidereit [:till]]

(In reply to matteo sisti sette from comment #54)
> Why don't you try to make a convincing argument for non having it? What if
> saying that you don't need it won't do?

For one, I did make an argument in comment 24, whereas you didn't. Also, it's just how arguing works: you're trying to change a decision, so you should argue for the change, instead of just demanding it.

(In reply to matteo sisti sette from comment #55)
> Do you really believe the heuristic will give the best choice in 100% of
> cases? Write down the heuristic and anybody will be able to write a
> counterexample showing a program that runs slower with the backend chosen by
> the heuristic. Do we have to waste our time doing that?

No, I don't think the heuristic will turn out right in 100% of cases. Neither will authors choosing. Your reply is probably going to be that that's the developer's problem, not ours. It is ours, though, if performance sucks in our browser, but not in others, because their canvas implementation is sufficiently different from ours to make decisions like this not valid across browsers.

It's even likely that in some cases the right decision will change over time, in the same browser. How do you propose we deal with that?

Additionally, there's plenty of evidence that flags like this are very frequently misused. Here's a concrete example: Flash has a flag on display objects named "cacheAsBitmap", that forces the player to, well, cache the rendering of the object in a bitmap. People are using this flag very frequently. Sadly, in the many cases, they're actually slowing things down, because they're forcing the invalidation of this bitmap much more than they think they do. Additionally, and very relevantly for this discussion, the Flash rendering system has improved since the flag was introduced, and many cases that used to be faster with cacheAsBitmap are now actually slower or just as fast as without, but take a lot more memory. Various Adobe people have since admitted that introducing this flag was a mistake. (An explanation of the first issue is described in [1]. Sadly, I can't find the source for the second part right now; I apologize for that and ask that you take my word for it. If you don't, consider just the first part.)


(In reply to matteo sisti sette from comment #56)
> And here's a simple use case: somebody wants to create a web page that
> demonstrates how hardware acceleration makes rendering faster, so he creates
> a page with a software canvas and a hardware-accelerated one running the
> same code so the visitor can appreciate the difference.

Well, such a test case won't be possible - precisely because it doesn't make sense. However, what could be demonstrated on such a page is that setting the flag can change startup performance, until the heuristics have enough data to properly apply. If you're doing something that really needs the buffer to be in main memory, then setting the flag to false will worsen performance at the beginning. Same is true if you're doing something that really doesn't, but set the flag to true.


> It's even easier to make wrong assumptions about javascript code using the
> canvas, or assumptions that hold for all the javascript code that has ever
> been written, but won't always for all the code to be written in the future.

I don't understand this paragraph: why would we need to make assumptions about js code? We make decisions based on actual patterns of usage, while said usage is happening.

> What about the unfortunate author that incurs in the very unlikely case?
> Will you tell him to just accept that his code has to run slower because
> he's doing something too unusual?

Essentially: yes. Stackoverflow, etc will also help them change their code to be more performant. The alternative would be to tell the unfortunate users of a page that has a flag set in the wrong way but forces our hands that they'll have to suffer from bad performance in our browser but not in some other browser. Stackoverflow could only tell them to switch their browser.


> If you're providing a way to specify a hint, which sets an initial value
> that may be overridden by the heuristic, why don't you want to allow to set
> a flag to force that initial value to be kept? If the heuristic is already
> there and the mechanism to provide a flag is already there (you're using
> that for the hint), providing the possibility to make the hint "imperative"
> should be zero effort: why do you want the machine to be arrogant at all
> cost?

You're right: it wouldn't be difficult to implement this. Same as it wouldn't be difficult to implement functionality that lets the author force certain JS functions to be JIT compiled immediately, say. The reason this isn't done is, as explained above, that in the extremely diverse landscape of browsers and devices, it's too hard to figure out which way to go on these things, so authors get it wrong too frequently.

Comment 59

[matteo sisti sette]

> No, I don't think the heuristic will turn out right in 100% of cases. 

Then the heuristic alone cannot do, full stop.

> Neither will authors choosing. 
> Your reply is probably going to be that that's the developer's problem, not ours.

No, my reply is not that, if you read carefully my last comment.
I'm haven't said that the hint should always be forced. I have said that the author should be allowed to choose between giving just a hint or giving an imperative choice.

Comment 60

[matteo sisti sette]

> The reason this isn't done is, as explained above, that in the extremely diverse landscape of browsers and devices, it's too hard to figure out which way to go on these things, so authors get it wrong too frequently.

Then **** those rare authors that may get it right.

Comment 61

[matteo sisti sette]

> It's even likely that in some cases the right decision will change over time, in the same browser. How do you propose we deal with that?

You can't.
The question is: how does an _author_ deal with that when he writes code and he detects that it performs ****-slow because the browser is taking a wrong decision? (or a decision that is right in most cases but in this case is causing the code to run slowly)?
Sit back and wait for the browser to evolve? If there's an urgence to publish the content and have it work _now_, he must be able to choose to force the backend to what he knows (and he can test and verify) to run faster.

Note that being able to force the choice will also allow the author to test what actually works better, so he won't have to base the decision on his wrong assumptions at all. And he'll be able to check whether the automatically selected backend is the fastest or not. And if not, report it so that you can improve the heuristic.

Comment 62

[matteo sisti sette]

>> And here's a simple use case: somebody wants to create a web page that
>> demonstrates how hardware acceleration makes rendering faster, so he creates
>> a page with a software canvas and a hardware-accelerated one running the
>> same code so the visitor can appreciate the difference.

> Well, such a test case won't be possible - precisely because it doesn't make sense.

I hope what doesn't make sense to you is the statement that you apparently took literally that "hardware acceleration makes rendering faster". I OBVIOUSLY oversimplified. I meant: make a webpage that allows to compare the performance of some given code with and without hardware acceleration, by having one instance running in one mode and the other in the other mode.

Comment 63

[Boris Zbarsky [:bzbarsky]]

> Note that being able to force the choice will also allow the author to test what
> actually works better

You're assuming that authors will test in all browsers.  And keep retesting with every new version of every browser.

Both assumptions unfortunately do not hold: authors simply don't behave that way.

Till's point is that browsers are different enough from each other, and changing enough, that except in the simplest cases forcing one behavior or the other will end up being wrong in some of the versions of some of the browsers.  So it's not that authors who "get it right" will be rare.  Except in the simplest cases they will simply not exist, because there will be no right answer, short of lots of browser type and version detection.  And in the simplest cases, a reasonable heuristic will in fact handle things fine.

Given that, providing a way to override the heuristic should be a last resort of we in fact get reliable reports of there being cases that the heuristic misdetects.  Because once we provide such a way, we are absolutely guaranteed that pages will be produced that in fact behave incorrectly in some browsers, forcing those browsers to treat the override as just a hint to remain competitive.  And then you're back to square one, of course.

Comment 64

[Jason Smith [:jsmith]]

Blocks a blocker - moving to koi+

Comment 65

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/cf7dbb8a1a1d
https://hg.mozilla.org/releases/mozilla-b2g26_v1_2/rev/79961e120d99

Comment 67

[Florian Scholz (Open Web Docs)]

Added "willReadFrequently" to the reference:
https://developer.mozilla.org/en-US/docs/Web/API/HTMLCanvasElement.getContext
And mentioned it in the release notes:
https://developer.mozilla.org/en-US/Firefox/Releases/28#Interfaces.2FAPIs.2FDOM

Comment 68

[Till Schneidereit [:till]]

Florian, this flag is only available if the gfx.canvas.willReadFrequently.enable setting is set to true. I guess that means that we shouldn't have it in the documentation or, at the very least point out this restriction?

Comment 69

[Florian Scholz (Open Web Docs)]

Till, great catch, thanks for pointing that out!

Looks like it is enabled by default for B2G
http://dxr.mozilla.org/mozilla-central/source/b2g/app/b2g.js#1024

I added the restriction note there.