1748783 - (Canvas 2d) getImageData() regressions from remote Canvas

Status: RESOLVED FIXED

(Core :: Graphics, defect, P2)

Description

[Philipp Rosenboom]

Attachment: image (2).png

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36

Steps to reproduce:

1. Disable the Hardware acceleration
2. Play a Browser Game (like https://empire.goodgamestudios.com/)
3. Messure the FPS
4. Enable the Hardware acceleration
5. Play the same Browser Game again
6. Messure the FPS

Actual results:

The FPS is even lower then with an activated Performance: Hardware acceleration. We captures performance mostly under 30 FPS. See the captured trace with the Firefox Profiler here: https://share.firefox.dev/3qguYWM

Expected results:

The Performance should be better than without this feature. So the FPS should be above 50 FPS, or like on other browsers by 60 FPS.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::Graphics' component, and is moving the bug to that component. Please revert this change in case you think the bot is wrong.

Comment 2

[Philipp Rosenboom]

Please find another Firefox Profile trace (from another game) here as well: https://share.firefox.dev/3mo4jWM
It have the exact same problem.

Comment 3

[Darkspirit]

Thanks for the report! Please re-enable "Use recommended performance settings", restart Firefox, open about:support, click on "Copy text to clipboard" and paste it here.

Comment 4

[Jim Mathies [:jimm]]

(In reply to Philipp Rosenboom from comment #2)

Please find another Firefox Profile trace (from another game) here as well: https://share.firefox.dev/3mo4jWM
It have the exact same problem.

Could you try regenerating this, but when you do, select the Graphics profile option in the profiler drop down? Thanks!

Comment 5

[Philipp Rosenboom]

We can provide this on next monday...

Comment 6

[Philipp Rosenboom]

The performance data:
QA Client: https://share.firefox.dev/3nx1uDz
Live Client: https://share.firefox.dev/3rsTrZo

Comment 7

[Dzmitry Malyshau [:kvark]]

Thank you for the profiles!
The canvas worker looks most busy. In particular, just locking and unlocking textures takes huge amount of time. Both contain heavy calls to "long NDXGI::CDevice::WaitForSynchronizationObjectFromCpuCB". Looks like CPU is blocked on GPU doing things?
Lee may know canvas stuff to see further.

Comment 8

[Bob Owen (:bobowen)]

I see quite an improvement on my machine with acceleration enabled.

Would you mind testing with gfx.canvas.remote = false, to see if that makes a difference.

Comment 9

[Philipp Rosenboom]

Hi,
when we should test it with "gfx.canvas.remote = false" I would need some more details how I can do this (what need to be changed before I run it again)?

Comment 10

[Bob Owen (:bobowen)]

(In reply to Philipp Rosenboom from comment #9)

Hi,
when we should test it with "gfx.canvas.remote = false" I would need some more details how I can do this (what need to be changed before I run it again)?

Sorry, I should have given more detail:

• in a new tab navigate to about:config and accept the warning
• type canvas.r in the search box gfx.canvas.remote should be the only pref I think
• toggle it to false and restart the browser

Thanks

Comment 11

[Gian-Carlo Pascutto [:gcp]]

Philipp, can you also go to about:support and copy-paste the result into a .txt file and attach it to this bug?

Comment 12

[Philipp Rosenboom]

We will check both and share it end of this week.

Comment 13

[Philipp Rosenboom]

Hi all,

we did the retest with the gfx.canvas.remote = false, please find the Profiler Link here: https://share.firefox.dev/3GpkI4M
Please also find the about info attached.

Comment 14

[Philipp Rosenboom]

Attachment: About info details.txt

Comment 15

[Gian-Carlo Pascutto [:gcp]]

From the about:support

Target Frame Rate: 32
DirectWrite: true (10.0.19041.1415)
GPU #1
Active: Yes
Description: Intel(R) HD Graphics 4600
...
RAM: 0
GPU #2
Active: No
Description: Microsoft Remote Display Adapter

Display0: 2560x1440@32Hz : Microsoft Remote Display Adapter

Are these tests being run through Windows Remote Desktop? Notice the target frame rate!

Comment 16

[Gian-Carlo Pascutto [:gcp]]

Philipp, can you comment on the performance, i.e. when flipping that setting, did it affect FPS? See Bob's question:

Would you mind testing with gfx.canvas.remote = false, to see if that makes a difference.

I notice from the debug info you posted that these tests are being run through Remote Desktop. It seems that in that setup Firefox is told (by Windows) that the display maxes out at 32 FPS. This may have thrown off all your performance tests and explain why Firefox consistently gets ~30FPS no matter what settings - maybe it's better to check on a local machine.

Comment 17

[Philipp Rosenboom]

We have retest the same again without Remote Desktop, please find the profile link here: https://share.firefox.dev/3Bcwnmb
Also we attached the new About info (aboutinfodetails2).

Comment 18

[Philipp Rosenboom]

Attachment: aboutinfodetails2.txt

Comment 19

[Jim Mathies [:jimm]]

(In reply to Philipp Rosenboom from comment #17)

We have retest the same again without Remote Desktop, please find the profile link here: https://share.firefox.dev/3Bcwnmb
Also we attached the new About info (aboutinfodetails2).

Would you mind generating a new Graphics profile with hardware acceleration turned off so we can compare the two? Thanks!

Also adding this to triage since we have a good profile now. I see lots of canvas and js during the jank periods.

Comment 20

[Philipp Rosenboom]

Sure, we can test it again - but only start of next week as we are currently having a release to provide this week.

Comment 21

[Philipp Rosenboom]

Please find the profile with turned off acceleration: https://share.firefox.dev/3BLysWH
Also I added the Info again (number 3 now).

Comment 22

[Philipp Rosenboom]

Attachment: aboutinfodetails3.txt

Comment 23

[Jeff Muizelaar [:jrmuizel]]

Phillip, to confirm is performance acceptable with gfx.canvas.remote = false and remote desktop off?

Comment 24

[Philipp Rosenboom]

We do a final testrun again with two different devices and set-ups, mesure the FPS and link the profiler below:

Device #1
Processor Intel(R) Core(TM) i5-10400H CPU @ 2.60GHz 2.59 GHz
Installed RAM 16,0 GB (15,6 GB usable)
System type 64-bit operating system, x64-based processor

Case 1: gfx.canvas.remote = false --> 42FPS (ok) https://share.firefox.dev/3he5022
Case 2: gfx.canvas.remote = true --> 22FPS (poor) https://share.firefox.dev/3JOFWuK

Device #2
Processor Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz 2.71 GHz
Installed RAM 8,00 GB (7,88 GB usable)
System type 64-bit operating system, x64-based processor

Case 3: gfx.canvas.remote = false --> 42FPS (ok) https://share.firefox.dev/35sNNiB
Case 4: gfx.canvas.remote = true --> 16FPS (poor) https://share.firefox.dev/3Ih21lk

So in summary, yes with gfx.canvas.remote = false it is ok.

Comment 25

[Bob Owen (:bobowen)]

(In reply to Philipp Rosenboom from comment #24)

We do a final testrun again with two different devices and set-ups, mesure the FPS and link the profiler below:

Thanks for these, unfortunately the profiles were not done with Graphics settings, so we can't see the canvas activity in the GPU process to try and work out what the content process is waiting on.

Could you please take new profiles with the Graphics settings.
This will mean that there should be CanvasWorkers threads in the GPU Process for the profile.

Comment 26

[Philipp Rosenboom]

Just to get you right, you would like to have the Graphics in - these files (zipped) for all cases combined are 122MB (too much to upload here). If this

Comment 27

[Bob Owen (:bobowen)]

(In reply to Philipp Rosenboom from comment #26)

Just to get you right, you would like to have the Graphics in - these files (zipped) for all cases combined are 122MB (too much to upload here). If this

No, just the profiles recreating with the Graphics settings selected.
You can select this via the Profiler addon icon or by going to about:profiling.
These Graphics settings mean that the profiles will include threads that will hopefully shed more light on the issue.

Comment 28

[Philipp Rosenboom]

Ah ok, sure here it is:
Case 1: https://share.firefox.dev/3pEi0m9
Case 2: https://share.firefox.dev/3vFLzYA
Case 3: https://share.firefox.dev/3sGwJPm
Case 4: https://share.firefox.dev/35RMPwQ

Comment 29

[Bob Owen (:bobowen)]

(In reply to Philipp Rosenboom from comment #28)

Ah ok, sure here it is:
Case 1: https://share.firefox.dev/3pEi0m9
Case 2: https://share.firefox.dev/3vFLzYA
Case 3: https://share.firefox.dev/3sGwJPm
Case 4: https://share.firefox.dev/35RMPwQ

Thanks for these.
It's not totally clear, but it looks like much of the time is being spent in getImageData.
If this is used a lot, particularly in the middle of a frame then this might explain the performance issue.

I've dug out an old laptop to try and test locally.
When I try the URL that seems to be being used for the profiles it just gets to 25% loading and then stops:
https://empire-html5-test.goodgamestudios.com/qa/rc_m2/index.html?forceManualLogin=false&inGameShop=1

Is there a way I can run this test locally?

Comment 30

[Philipp Rosenboom]

Hi,
this is not so simply possible. The test-servers are only accessible over VPN. Is there another way to test it?
If not I need to talk to IT, not sure if this at all an option…

Comment 31

[Bob Owen (:bobowen)]

(In reply to Philipp Rosenboom from comment #30)

Hi,
this is not so simply possible. The test-servers are only accessible over VPN. Is there another way to test it?
If not I need to talk to IT, not sure if this at all an option…

I guess if you have something that can be accessed publicly that displays the same issue, that would be best.
I did try the URL in the description before, but couldn't see any obvious issue.

I've just retried with the URL in the description with an old laptop and initially didn't see much difference between remote canvas enabled/disabled.
However when I moved the cursor around the screen there was a big frame rate drop.
When I tried back on my main machine I could see a similar, but smaller drop.

It seems that the performance issue is from getImageData being called as part of t._testHit (from my profiling).

Comment 32

[Philipp Rosenboom]

Why you can test it live directly (as there we are facing the problem): https://empire.goodgamestudios.com
This would be best per my understanding.

Comment 33

[Philipp Rosenboom]

Our devs enabled you now also a test enviorment now, which you can access under: https://html5-game-integration-extqa.ggs-ep.com/empire?env=qa&branch=main
Please make sure, that you use "EXT-QA: 1" as server to play the game, other servers might work sometimes but can be shutdown or incompatible.
You can select the server at the bottom before you login, right side of the langauge selection.

Comment 34

[Bob Owen (:bobowen)]

(In reply to Philipp Rosenboom from comment #33)

Our devs enabled you now also a test enviorment now, which you can access under: https://html5-game-integration-extqa.ggs-ep.com/empire?env=qa&branch=main
Please make sure, that you use "EXT-QA: 1" as server to play the game, other servers might work sometimes but can be shutdown or incompatible.
You can select the server at the bottom before you login, right side of the langauge selection.

Thanks I will try that, although as I said in the rest of my comment, it looks like it is the way you are doing hit testing with getImageData that is causes the performance issue on Firefox.
I'm not too sure how easy it will be to speed things up when information is being read out of the canvas many times.

Comment 35

[Bob Owen (:bobowen)]

(In reply to Philipp Rosenboom from comment #33)

Our devs enabled you now also a test enviorment now, which you can access under: https://html5-game-integration-extqa.ggs-ep.com/empire?env=qa&branch=main
Please make sure, that you use "EXT-QA: 1" as server to play the game, other servers might work sometimes but can be shutdown or incompatible.
You can select the server at the bottom before you login, right side of the langauge selection.

According to our devtools the FPS seems better for the test environment, although the FPS in the game itself doesn't seem to tally.
I still see lots of getImageData in the profile anyway.

lsalzman - I was wondering how close we are for the new accelerated canvas and whether it is likely to perform better for this case?
Trying to determine whether it is worth spending time trying to improve the current remote canvas.

Comment 36

[Lee Salzman [:lsalzman]]

It's going to be quite a while before WebGL-accelerated canvas2d is ready to deploy.

Comment 37

[Bob Owen (:bobowen)]

Sorry about the delay here.
I landed a quick fix in bug 1766402, which makes quite a difference in the amount of time spent in getImageData for the profiles on this machine.

The fix is in the latest Nightly and it would be interesting to know whether this improves things for your test machines.

I'm still looking at other possible improvements, but they are a bit more complicated.

Comment 38

[Philipp Rosenboom]

Sorry as well, we needed some time to check this. It seams that we have +50% improvement with the newest Beta-Version of Firefox. With this we are on a level where we are ok.
Thanks a lot for your support here!

Comment 39

[Philipp Rosenboom]

Hi again,
from our side we can close this ticket. The problem is solved in the new version and with your hints we have even improved our side. Now the performance is fine and not a problem anymore.

Thank you!

Comment 40

[Bob Hood [:bhood]]

Marking resolved, per Philipp's instructions.