Use "preflight_run_cmd_suites" for Marionette for unique screen size and to disable various Windows notifications

RESOLVED FIXED in Firefox 67

Status

enhancement
P1
normal
RESOLVED FIXED
5 months ago
4 months ago

People

(Reporter: whimboo, Assigned: whimboo)

Tracking

(Blocks 1 bug)

unspecified
mozilla67
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox67 fixed)

Details

Attachments

(1 attachment)

No description provided.

Just noticed when comparing test results for the following try job:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=458888f88dae183bba2a7170fac0eb02e1ebeea7&searchStr=windows

Each job on a Win10 tester has a resolution of 1024x768 only.

It would be great to increase it to 1600x1200 like for other platforms and Windows 7.

Flags: needinfo?(jmaher)
Summary: Win10 jobs only run on a 1024x768 screen resolution: → Win10 jobs only run on a 1024x768 screen resolution instead of 1600x1200 as Windows 7

btw, I think most windows 10 test tasks change the resolution to 1280x1024, as seen here:

https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=222598601&repo=mozilla-central&lineNumber=934-941

00:32:04 INFO - Running command: ('c:\mozilla-build\python\python.exe', 'Z:\task_1547771067\mozharness\external_tools\mouse_and_screen_resolution.py', '--configuration-file', 'Z:\task_1547771067\mozharness\external_tools\machine-configuration.json') in Z:\task_1547771067\build
00:32:04 INFO - Copy/paste: c:\mozilla-build\python\python.exe Z:\task_1547771067\mozharness\external_tools\mouse_and_screen_resolution.py --configuration-file Z:\task_1547771067\mozharness\external_tools\machine-configuration.json
00:32:07 INFO - Screen resolution (current): (1024, 768)
00:32:07 INFO - Changing the screen resolution...
00:32:07 INFO - Screen resolution (new): (1280, 1024)
00:32:07 INFO - Mouse position (current): (640, 512)
00:32:07 INFO - Mouse position (new): (1010, 10)
00:32:07 INFO - Return code: 0

and windows 7 tests change from 1600x1200 to 1280x1024 (if logs are to accurate).

So that is more a problem of Marionette jobs only?

Windows tasks based on desktop_unittest.py use

https://searchfox.org/mozilla-central/rev/dac799c9f4e9f5f05c1071cba94f2522aa31f7eb/testing/mozharness/configs/unittests/win_taskcluster_unittest.py#230-243

to run the screen resolution change. wpt runs the same command. So, yes, maybe only Marionette is different.

thanks :gbrown, I have been slow to respond to this and you dug in and did better than me in answering this.

Flags: needinfo?(jmaher)

Oh, we don't use any of those preflight_run_cmd_suites commands. Some of them sound very important, especially those for notifications, which our tests might be affected from.

Geoff, is there anything from that list of commands which should not be run for Marionette?

Flags: needinfo?(gbrown)

They all sound useful to me: Probably all of them should be run for Marionette.

Flags: needinfo?(gbrown)

Taking with the hope that this will fix a couple of our intermittent failures which are focus related.

Assignee: nobody → hskupin
Status: NEW → ASSIGNED
Component: General → Marionette
Priority: -- → P1
Summary: Win10 jobs only run on a 1024x768 screen resolution instead of 1600x1200 as Windows 7 → Use "preflight_run_cmd_suites" for Marionette for unique screen size and to disable various Windows notifications

Please note that we also don't run all those preflight run commands for web-platform-tests. I wonder if that is on purpose, or if it should also be added. I think it was most likely missed.

Flags: needinfo?(james)
Flags: needinfo?(gbrown)
Blocks: 1521447

First try works fine on Linux and MacOS but has string formatting problems for Windows:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=27824123bf24171d667e11dd1741204472f9be45&selectedJob=223020713

I reverted those extra changes for Windows and pushed another try build:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=c672fab1d46691ab8835a6a8df87370fd1f01307

When running Marionette tests in Taskcluster a default window size
per platform has to be set. Further no screen saver, nor notifications
should be active which could interfere with the focus of the
application. Further there is no need to display the taskbar too.

(In reply to Henrik Skupin (:whimboo) [⌚️UTC+1] from comment #9)

Please note that we also don't run all those preflight run commands for web-platform-tests. I wonder if that is on purpose, or if it should also be added. I think it was most likely missed.

I also suspect it was an accidental omission, but I don't know; hopefully jgraham can say with more certainty.

Flags: needinfo?(gbrown)

Ok, the patch as attached will be for Marionette only. Once James replied I will file a separate bug for web-platform-tests.

Geoff, I wonder why each and every test job has to opt-in to use those scripts. Wouldn't it be better to enable those by default, and those which eg. don't want to set the screensize can opt-out? I think that this would help a lot in lowering the number of intermittent failures, given that most people aren't aware of those settings.

Flags: needinfo?(gbrown)

Yes, enabled by default sounds right to me.

Flags: needinfo?(gbrown)

Yes, I guess? I don't really know what this does but if it's a thing that all testsuites are supposed to do then wpt should probably do it too. But I agree with the premise that if it's a thing that everyone is supposed to do it shouldn't rely on everyone providing the same dozen lines of configuration.

Flags: needinfo?(james)

(In reply to Geoff Brown [:gbrown] from comment #14)

Yes, enabled by default sounds right to me.

Ok, I filed bug 1527840 to track enabling it by default in all suite configs.

(In reply to James Graham [:jgraham] from comment #15)

Yes, I guess? I don't really know what this does but if it's a thing that all testsuites are supposed to do then wpt should probably do it too.

I will file a new bug for it.

Blocks: 1527842

I updated the Marionette configs to just be a 1:1 copy from unittests:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=2fd1ab90b39c3f9934c049b0771b960a60cdaf97

Comment 18

4 months ago
Pushed by hskupin@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/2e6a0be2fb27
[marionette] Use "preflight_run_cmd_suites" in mozharness config. r=jmaher,gbrown

Comment 19

4 months ago
bugherder
Status: ASSIGNED → RESOLVED
Closed: 4 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla67
You need to log in before you can comment on or make changes to this bug.