1658443 - windows10 accessing extra files in browser_startup_mainthreadio.js and related tests

Status: NEW

(Core :: Performance Engineering, defect, P3)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

I am working to get our windows10 tests running on hardware (and fallback to the cloud when busy) in order to use our spare cpu time in the datacenter. most tests pass juts fine, but a few fail.

https://firefoxci.taskcluster-artifacts.net/GkRzzJOpTyaS0AqwF3GAZQ/0/public/logs/live_backing.log

TEST-UNEXPECTED-FAIL | browser/base/content/test/performance/browser_startup_content_mainthreadio.js | unexpected read on C:\ProgramData\Intel\ShaderCache\firefox_1 in Web Content process - 
TEST-UNEXPECTED-FAIL | browser/base/content/test/performance/browser_startup_content_mainthreadio.js | unexpected read on C:\ProgramData\Intel\ShaderCache\firefox_0 in Web Content process - 
TEST-UNEXPECTED-FAIL | browser/base/content/test/performance/browser_startup_content_mainthreadio.js | unexpected read on C:\ProgramData\Intel\ShaderCache\firefox_1 in WebExtensions process - 
TEST-UNEXPECTED-FAIL | browser/base/content/test/performance/browser_startup_content_mainthreadio.js | unexpected read on C:\ProgramData\Intel\ShaderCache\firefox_0 in WebExtensions process - 

TEST-UNEXPECTED-FAIL | browser/base/content/test/performance/browser_startup_mainthreadio.js | unexpected stat on C:\Users\task_1596845536\AppData\Local\Temp\tmpudqlbh.mozrunner\wmfvpxvideo.guard before handling user events - 

TEST-UNEXPECTED-FAIL | browser/base/content/test/performance/browser_startup_syncIPC.js | unexpected PContent::Reply_BeginDriverCrashGuard sync IPC before handling user events - 
TEST-UNEXPECTED-FAIL | browser/base/content/test/performance/browser_startup_syncIPC.js | unexpected PContent::Reply_EndDriverCrashGuard sync IPC before handling user events - 

I can add exceptions to the files, is there more work that needs to be done before doing that?

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

:gijs, I saw your name as a reviewer for some of these test files, was hoping you could help lead me in the right direction!

Comment 2

[:Gijs (he/him)]

(In reply to Joel Maher ( :jmaher ) (UTC-4) from comment #1)

:gijs, I saw your name as a reviewer for some of these test files, was hoping you could help lead me in the right direction!

The files at issue all seem to be graphics files.

Joel, can you give some more details about the environment you're running these tests in?

Bas, do you know what'd cause us (or stuff we load in our process(es)) to read these files, and if we've got bugs on file to fix some of these (I assume that, if the gfx process is in use, we shouldn't need shader cache data in the (webextension or web) content processes) ?

Similar-ish question for the IPC - is there a bug on getting rid of that, or is that not possible? (I assume we cache the decision; the tests will run into the "first time we run this profile on this hardware, time to test some stuff" case.)

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

The environment is the same hardware and OS we run perf tests on (although I have updated the OS to windos 10 1803 to support fonts better):
https://wiki.mozilla.org/TestEngineering/Performance/Platforms#HPE_Moonshot

I have another bug (bug 1657707) which is about canvas2d and some related tests, I didn't think this would be related. If I should file bugs for each error, or errors in each file I am happy to. Here is an example try push:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=0950eb2495a1d7ebf64281110ae2f6d368c3d9ad

Comment 4

[Release mgmt bot [:suhaib / :marco/ :calixte]]

Bugbug thinks this bug should belong to this component, but please revert this change in case of error.

Comment 5

[:Gijs (he/him)]

bad bot, no botsnack for you.

Comment 6

[Joel Maher ( :jmaher ) (UTC -8)]

:gijs, can I just create a patch with the modified file access needed to get these tests passing?

Comment 7

[:Gijs (he/him)]

(In reply to Joel Maher ( :jmaher ) (UTC-4) from comment #6)

:gijs, can I just create a patch with the modified file access needed to get these tests passing?

Yes, but please file 3 bugs, one for each of the set of failures, and add the bug numbers in a comment for each of the exceptions you're adding to the test.

Comment 8

[Simona Badau, Desktop QA]

Setting this to Firefox: General, if this is not the right component, please move it to the correct one. Thanks!

Comment 9

[Jeff Muizelaar [:jrmuizel]]

Is it possible to get stacks for the file accesses?

Comment 10

[:Gijs (he/him)]

(In reply to Jeff Muizelaar [:jrmuizel] from comment #9)

Is it possible to get stacks for the file accesses?

There are stacks produced by the profiler markers in the test, but they're stacks from Windows opt builds, so they're not very helpful. The log from comment #0 has e.g.:

[task 2020-08-08T00:32:01.840Z] 00:32:01     INFO - TEST-UNEXPECTED-FAIL | browser/base/content/test/performance/browser_startup_content_mainthreadio.js | unexpected read on C:\ProgramData\Intel\ShaderCache\firefox_1 in Web Content process - 
[task 2020-08-08T00:32:01.840Z] 00:32:01     INFO - Stack trace:
[task 2020-08-08T00:32:01.840Z] 00:32:01     INFO -   PContent::Msg_SetXPCOMProcessAttributes
[task 2020-08-08T00:32:01.840Z] 00:32:01     INFO -   XRE_InitChildProcess

For the crash guard:

[task 2020-08-08T00:32:02.028Z] 00:32:02     INFO - TEST-UNEXPECTED-FAIL | browser/base/content/test/performance/browser_startup_mainthreadio.js | unexpected stat on C:\Users\task_1596845536\AppData\Local\Temp\tmpudqlbh.mozrunner\wmfvpxvideo.guard before handling user events - 
[task 2020-08-08T00:32:02.028Z] 00:32:02     INFO - Stack trace:
[task 2020-08-08T00:32:02.029Z] 00:32:02     INFO -   nsLocalFile::ResolveAndStat
[task 2020-08-08T00:32:02.029Z] 00:32:02     INFO -   PContent::Msg_BeginDriverCrashGuard
[task 2020-08-08T00:32:02.029Z] 00:32:02     INFO -   XREMain::XRE_main

Does that help at all?

Comment 11

[:Gijs (he/him)]

Florian, is reducing these types of mainthread IO / sync IPC on startup still a focus? This bug hasn't moved forward for a while, and the exceptions added (to keep tests passing) in bug 1660582, bug 1660590 and bug 1660480 are all ignoreIfUnused so I guess we have no idea if we're still doing this or not...

Comment 12

[Jared Hirsch [:jhirsch] (he/him) (Needinfo please)]

Hey Florian, any thoughts on what to do with this bug?

Comment 13

[Mike Conley (:mconley) (:⚙️)]

This might be more of a Core :: Graphics issue, but let's see what our perf triagers think.