1316858 - Perma failures in windows 7 cpp tests with INFRA-ERROR: The new screen resolution or mouse positions are not what we expected

Status: RESOLVED FIXED

(Taskcluster :: General, defect)

Description

[Carsten Book [:Tomcat]]

CPP Tests on windows 7 are currently perma failing with:

 INFRA-ERROR: The new screen resolution or mouse positions are not what we expected 

like https://treeherder.mozilla.org/logviewer.html#?job_id=39045605&repo=mozilla-inbound#L1848

its tier 2 but we should fix / backout the regression asap since this makes sheriff work harder and we could miss other issues

Comment 1

[Carsten Book [:Tomcat]]

as discussed on irc, perma failure on cpp tests on win7 currently

Comment 2

[Rob Thijssen [:grenade (EET/UTC+0300)]]

g-w reverted to 7.0.0 on testers: https://github.com/mozilla-releng/OpenCloudConfig/commit/4b9c0f728a7c46fcad5fc2413905c4c2ee3ce4d7

will take several hours to propagate through ami cycles...

Comment 3

[Rob Thijssen [:grenade (EET/UTC+0300)]]

Attachment: Bug 1316858 - set win tests to tier 3;

Review commit: https://reviewboard.mozilla.org/r/92318/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/92318/

Comment 4

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8809802 [details]
Bug 1316858 - set win tests to tier 3;

https://reviewboard.mozilla.org/r/92318/#review92344

::: taskcluster/taskgraph/transforms/tests/all_kinds.py:52
(Diff revision 1)
>              if test['test-platform'] in ['linux64/debug',
>                                           'linux64-asan/opt',
>                                           'android-4.3-arm7-api-15/debug',
>                                           'android-x86/opt']:
>                  test['tier'] = 1
>              else:

The check would be better down here -- `elif test['test-platform'].startswith('win'): ..`  Then it can be overridden for specific win tests.

Comment 5

[Rob Thijssen [:grenade (EET/UTC+0300)]]

that was actually the problem. the external-media-tests suite has an override for level 2 (to prevent it being a level 1 on linux) i can't override twice so i opted for the carpet bomb approach...

Comment 6

[Rob Thijssen [:grenade (EET/UTC+0300)]]

Comment on attachment 8809802 [details]
Bug 1316858 - set win tests to tier 3;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/92318/diff/1-2/

Comment 7

[Rob Thijssen [:grenade (EET/UTC+0300)]]

Comment on attachment 8809802 [details]
Bug 1316858 - set win tests to tier 3;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/92318/diff/2-3/

Comment 8

[Intermittent Failures Robot]

9 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-inbound: 8
* autoland: 1

Platform breakdown:
* windows7-32-vm: 9

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1316858&startday=2016-11-07&endday=2016-11-13&tree=all

Comment 9

[Pete Moore [:pmoore][:pete]]

Note - the problem here is that the test assumes the process is running in the active desktop, and thereby assumes that changes the display resolution will alter the desktop size.

Previously this was the case, as the active desktop was the desktop used by taskcluster tasks / buildbot jobs.

However, now we have sandboxed taskcluster tasks so that they do not share a desktop, but rather have their own desktop which cannot be exploited to gain access to an administrative account (since the previous desktop was running under an administrator account).

This means, the assumption that was previously made, is no longer viable. Instead the test should either get a handle to the process's desktop, and adjust that (if that is possible using windows API), or if not, make sure to open the process's desktop in the display, before changing the screen resolution (e.g. using https://msdn.microsoft.com/en-us/library/windows/desktop/ms684303(v=vs.85).aspx).

Note, this is backwardly compatible, so shouldn't introduce any problems anywhere - it simply removes an assumption which is no longer valid.

Comment 10

[Dustin J. Mitchell [:dustin] (he/him)]

That sounds non-trivial.  Who will be able to track that down?  Also, I wonder how many tests are affected -- this could be a *lot* of work to fix everywhere.

Comment 11

[Greg Arndt [:garndt]]

Pete, I'll assign this to you as I think you were investigating if there are changes we can make here and the tradeoffs for different approaches.

Comment 12

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8809802 [details]
Bug 1316858 - set win tests to tier 3;

I think this landed, in modified form, right?

Comment 13

[Pete Moore [:pmoore][:pete]]

So this bug has been resolved, regardless of the move to tier 3, I believe.

Joel, can you confirm this is no longer occurring? In which case, we should be able to close...

Comment 14

[Joel Maher ( :jmaher ) (UTC -8)]

I have disabled this in my try pushes so we can just focus on getting firefox running.  When we have a final solution for running firefox, it would make sense to turn this back on and update it if needed.

Comment 15

[Pete Moore [:pmoore][:pete]]

I've retriggered original job, which should run against new generic worker version - let's see if it resolves the issue or not.

https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=fb5ae665310e4f60464ccd43b2626fdfc2087a67&selectedJob=39458738

Comment 16

[Pete Moore [:pmoore][:pete]]

And she's green!