Closed Bug 1748783 Opened 6 months ago Closed 1 month ago

(Canvas 2d) getImageData() regressions from remote Canvas

Categories

(Core :: Graphics, defect, P2)

Firefox 97
defect

Tracking

()

RESOLVED FIXED

People

(Reporter: prosenboom, Assigned: bobowen)

References

(Blocks 1 open bug)

Details

Attachments

(4 files)

Attached image image (2).png

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36

Steps to reproduce:

  1. Disable the Hardware acceleration
  2. Play a Browser Game (like https://empire.goodgamestudios.com/)
  3. Messure the FPS
  4. Enable the Hardware acceleration
  5. Play the same Browser Game again
  6. Messure the FPS

Actual results:

The FPS is even lower then with an activated Performance: Hardware acceleration. We captures performance mostly under 30 FPS. See the captured trace with the Firefox Profiler here: https://share.firefox.dev/3qguYWM

Expected results:

The Performance should be better than without this feature. So the FPS should be above 50 FPS, or like on other browsers by 60 FPS.

The Bugbug bot thinks this bug should belong to the 'Core::Graphics' component, and is moving the bug to that component. Please revert this change in case you think the bot is wrong.

Component: Untriaged → Graphics
Product: Firefox → Core

Please find another Firefox Profile trace (from another game) here as well: https://share.firefox.dev/3mo4jWM
It have the exact same problem.

Thanks for the report! Please re-enable "Use recommended performance settings", restart Firefox, open about:support, click on "Copy text to clipboard" and paste it here.

Flags: needinfo?(prosenboom)

(In reply to Philipp Rosenboom from comment #2)

Please find another Firefox Profile trace (from another game) here as well: https://share.firefox.dev/3mo4jWM
It have the exact same problem.

Could you try regenerating this, but when you do, select the Graphics profile option in the profiler drop down? Thanks!

Blocks: wr-perf
Severity: -- → S3

We can provide this on next monday...

The performance data:
QA Client: https://share.firefox.dev/3nx1uDz
Live Client: https://share.firefox.dev/3rsTrZo

Flags: needinfo?(prosenboom)

Thank you for the profiles!
The canvas worker looks most busy. In particular, just locking and unlocking textures takes huge amount of time. Both contain heavy calls to "long NDXGI::CDevice::WaitForSynchronizationObjectFromCpuCB". Looks like CPU is blocked on GPU doing things?
Lee may know canvas stuff to see further.

Flags: needinfo?(lsalzman)
Flags: needinfo?(lsalzman) → needinfo?(bobowencode)

I see quite an improvement on my machine with acceleration enabled.

Would you mind testing with gfx.canvas.remote = false, to see if that makes a difference.

Flags: needinfo?(bobowencode) → needinfo?(prosenboom)

Hi,
when we should test it with "gfx.canvas.remote = false" I would need some more details how I can do this (what need to be changed before I run it again)?

Flags: needinfo?(prosenboom) → needinfo?(bobowencode)

(In reply to Philipp Rosenboom from comment #9)

Hi,
when we should test it with "gfx.canvas.remote = false" I would need some more details how I can do this (what need to be changed before I run it again)?

Sorry, I should have given more detail:

  • in a new tab navigate to about:config and accept the warning
  • type canvas.r in the search box gfx.canvas.remote should be the only pref I think
  • toggle it to false and restart the browser

Thanks

Flags: needinfo?(bobowencode) → needinfo?(prosenboom)

Philipp, can you also go to about:support and copy-paste the result into a .txt file and attach it to this bug?

We will check both and share it end of this week.

Hi all,

we did the retest with the gfx.canvas.remote = false, please find the Profiler Link here: https://share.firefox.dev/3GpkI4M
Please also find the about info attached.

Flags: needinfo?(prosenboom)
Attached file About info details.txt

From the about:support

Target Frame Rate: 32
DirectWrite: true (10.0.19041.1415)
GPU #1
Active: Yes
Description: Intel(R) HD Graphics 4600
...
RAM: 0
GPU #2
Active: No
Description: Microsoft Remote Display Adapter

Display0: 2560x1440@32Hz : Microsoft Remote Display Adapter

Are these tests being run through Windows Remote Desktop? Notice the target frame rate!

Philipp, can you comment on the performance, i.e. when flipping that setting, did it affect FPS? See Bob's question:

Would you mind testing with gfx.canvas.remote = false, to see if that makes a difference.

I notice from the debug info you posted that these tests are being run through Remote Desktop. It seems that in that setup Firefox is told (by Windows) that the display maxes out at 32 FPS. This may have thrown off all your performance tests and explain why Firefox consistently gets ~30FPS no matter what settings - maybe it's better to check on a local machine.

Flags: needinfo?(prosenboom)

We have retest the same again without Remote Desktop, please find the profile link here: https://share.firefox.dev/3Bcwnmb
Also we attached the new About info (aboutinfodetails2).

Flags: needinfo?(prosenboom)
Attached file aboutinfodetails2.txt

(In reply to Philipp Rosenboom from comment #17)

We have retest the same again without Remote Desktop, please find the profile link here: https://share.firefox.dev/3Bcwnmb
Also we attached the new About info (aboutinfodetails2).

Would you mind generating a new Graphics profile with hardware acceleration turned off so we can compare the two? Thanks!

Also adding this to triage since we have a good profile now. I see lots of canvas and js during the jank periods.

Blocks: gfx-triage
Flags: needinfo?(prosenboom)

Sure, we can test it again - but only start of next week as we are currently having a release to provide this week.

Please find the profile with turned off acceleration: https://share.firefox.dev/3BLysWH
Also I added the Info again (number 3 now).

Flags: needinfo?(prosenboom)
Attached file aboutinfodetails3.txt
Summary: Performance: Hardware acceleration is not working → Performance: Hardware acceleration is not working (Canvas 2d)
Flags: needinfo?(jmuizelaar)

Phillip, to confirm is performance acceptable with gfx.canvas.remote = false and remote desktop off?

Flags: needinfo?(prosenboom)

We do a final testrun again with two different devices and set-ups, mesure the FPS and link the profiler below:

Device #1
Processor Intel(R) Core(TM) i5-10400H CPU @ 2.60GHz 2.59 GHz
Installed RAM 16,0 GB (15,6 GB usable)
System type 64-bit operating system, x64-based processor

Case 1: gfx.canvas.remote = false --> 42FPS (ok) https://share.firefox.dev/3he5022
Case 2: gfx.canvas.remote = true --> 22FPS (poor) https://share.firefox.dev/3JOFWuK

Device #2
Processor Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz 2.71 GHz
Installed RAM 8,00 GB (7,88 GB usable)
System type 64-bit operating system, x64-based processor

Case 3: gfx.canvas.remote = false --> 42FPS (ok) https://share.firefox.dev/35sNNiB
Case 4: gfx.canvas.remote = true --> 16FPS (poor) https://share.firefox.dev/3Ih21lk

So in summary, yes with gfx.canvas.remote = false it is ok.

Flags: needinfo?(prosenboom)

(In reply to Philipp Rosenboom from comment #24)

We do a final testrun again with two different devices and set-ups, mesure the FPS and link the profiler below:

Thanks for these, unfortunately the profiles were not done with Graphics settings, so we can't see the canvas activity in the GPU process to try and work out what the content process is waiting on.

Could you please take new profiles with the Graphics settings.
This will mean that there should be CanvasWorkers threads in the GPU Process for the profile.

Flags: needinfo?(prosenboom)
Assignee: nobody → bobowencode
Flags: needinfo?(jmuizelaar)
Severity: S3 → S2
Priority: -- → P2

Just to get you right, you would like to have the Graphics in - these files (zipped) for all cases combined are 122MB (too much to upload here). If this

Flags: needinfo?(prosenboom)
Flags: needinfo?(bobowencode)

(In reply to Philipp Rosenboom from comment #26)

Just to get you right, you would like to have the Graphics in - these files (zipped) for all cases combined are 122MB (too much to upload here). If this

No, just the profiles recreating with the Graphics settings selected.
You can select this via the Profiler addon icon or by going to about:profiling.
These Graphics settings mean that the profiles will include threads that will hopefully shed more light on the issue.

Flags: needinfo?(bobowencode) → needinfo?(prosenboom)
Flags: needinfo?(prosenboom) → needinfo?(bobowencode)

(In reply to Philipp Rosenboom from comment #28)

Ah ok, sure here it is:
Case 1: https://share.firefox.dev/3pEi0m9
Case 2: https://share.firefox.dev/3vFLzYA
Case 3: https://share.firefox.dev/3sGwJPm
Case 4: https://share.firefox.dev/35RMPwQ

Thanks for these.
It's not totally clear, but it looks like much of the time is being spent in getImageData.
If this is used a lot, particularly in the middle of a frame then this might explain the performance issue.

I've dug out an old laptop to try and test locally.
When I try the URL that seems to be being used for the profiles it just gets to 25% loading and then stops:
https://empire-html5-test.goodgamestudios.com/qa/rc_m2/index.html?forceManualLogin=false&inGameShop=1

Is there a way I can run this test locally?

Flags: needinfo?(bobowencode) → needinfo?(prosenboom)
No longer blocks: gfx-triage

Hi,
this is not so simply possible. The test-servers are only accessible over VPN. Is there another way to test it?
If not I need to talk to IT, not sure if this at all an option…

Flags: needinfo?(prosenboom) → needinfo?(bobowencode)

(In reply to Philipp Rosenboom from comment #30)

Hi,
this is not so simply possible. The test-servers are only accessible over VPN. Is there another way to test it?
If not I need to talk to IT, not sure if this at all an option…

I guess if you have something that can be accessed publicly that displays the same issue, that would be best.
I did try the URL in the description before, but couldn't see any obvious issue.

I've just retried with the URL in the description with an old laptop and initially didn't see much difference between remote canvas enabled/disabled.
However when I moved the cursor around the screen there was a big frame rate drop.
When I tried back on my main machine I could see a similar, but smaller drop.

It seems that the performance issue is from getImageData being called as part of t._testHit (from my profiling).

Flags: needinfo?(bobowencode)

Why you can test it live directly (as there we are facing the problem): https://empire.goodgamestudios.com
This would be best per my understanding.

Flags: needinfo?(bobowencode)

Our devs enabled you now also a test enviorment now, which you can access under: https://html5-game-integration-extqa.ggs-ep.com/empire?env=qa&branch=main
Please make sure, that you use "EXT-QA: 1" as server to play the game, other servers might work sometimes but can be shutdown or incompatible.
You can select the server at the bottom before you login, right side of the langauge selection.

(In reply to Philipp Rosenboom from comment #33)

Our devs enabled you now also a test enviorment now, which you can access under: https://html5-game-integration-extqa.ggs-ep.com/empire?env=qa&branch=main
Please make sure, that you use "EXT-QA: 1" as server to play the game, other servers might work sometimes but can be shutdown or incompatible.
You can select the server at the bottom before you login, right side of the langauge selection.

Thanks I will try that, although as I said in the rest of my comment, it looks like it is the way you are doing hit testing with getImageData that is causes the performance issue on Firefox.
I'm not too sure how easy it will be to speed things up when information is being read out of the canvas many times.

Flags: needinfo?(bobowencode)

(In reply to Philipp Rosenboom from comment #33)

Our devs enabled you now also a test enviorment now, which you can access under: https://html5-game-integration-extqa.ggs-ep.com/empire?env=qa&branch=main
Please make sure, that you use "EXT-QA: 1" as server to play the game, other servers might work sometimes but can be shutdown or incompatible.
You can select the server at the bottom before you login, right side of the langauge selection.

According to our devtools the FPS seems better for the test environment, although the FPS in the game itself doesn't seem to tally.
I still see lots of getImageData in the profile anyway.

lsalzman - I was wondering how close we are for the new accelerated canvas and whether it is likely to perform better for this case?
Trying to determine whether it is worth spending time trying to improve the current remote canvas.

Flags: needinfo?(lsalzman)

It's going to be quite a while before WebGL-accelerated canvas2d is ready to deploy.

Flags: needinfo?(lsalzman)
Depends on: 1766402

Sorry about the delay here.
I landed a quick fix in bug 1766402, which makes quite a difference in the amount of time spent in getImageData for the profiles on this machine.

The fix is in the latest Nightly and it would be interesting to know whether this improves things for your test machines.

I'm still looking at other possible improvements, but they are a bit more complicated.

Flags: needinfo?(prosenboom)

Sorry as well, we needed some time to check this. It seams that we have +50% improvement with the newest Beta-Version of Firefox. With this we are on a level where we are ok.
Thanks a lot for your support here!

Flags: needinfo?(prosenboom)
Summary: Performance: Hardware acceleration is not working (Canvas 2d) → (Canvas 2d) getImageData() regressions from remote Canvas

Hi again,
from our side we can close this ticket. The problem is solved in the new version and with your hints we have even improved our side. Now the performance is fine and not a problem anymore.

Thank you!

Marking resolved, per Philipp's instructions.

Status: UNCONFIRMED → RESOLVED
Closed: 1 month ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.