1180351 - test_pointerlock-api.html fails on linux with --run-by-dir enabled

Status: RESOLVED FIXED

(Core :: DOM: Events, defect)

Description

[Kaustabh Datta Choudhury]

User Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0
Build ID: 20150525141253

Comment 1

[Kaustabh Datta Choudhury]

We want to make run-by-dir run reliably i.e. to minimize oranges. But as seen on try : https://treeherder.mozilla.org/#/jobs?repo=try&revision=cdbdecb24efa we are getting oranges due to test_pointerlock-api.html. This problem is specific to linux & does not occur in windows or mac.
 
From the try :
 264 INFO SimpleTest START
22:56:19     INFO -  265 INFO must wait for focus
22:56:19     INFO -  266 INFO TEST-PASS | dom/tests/mochitest/pointerlock/test_pointerlock-api.html | file_approval.html: Should only receive mozpointerlockchange when we've been approved for fullscreen.
22:56:19     INFO -  267 INFO TEST-PASS | dom/tests/mochitest/pointerlock/test_pointerlock-api.html | file_approval.html: Should only receive mozpointerlockchange when we've been approved for fullscreen.
22:56:19     INFO -  268 INFO must wait for focus
22:56:19     INFO -  269 INFO must wait for focus
22:56:19     INFO -  270 INFO TEST-PASS | dom/tests/mochitest/pointerlock/test_pointerlock-api.html | file_screenClientXYConst.html: Pointer should be unlocked
22:56:19     INFO -  271 INFO TEST-PASS | dom/tests/mochitest/pointerlock/test_pointerlock-api.html | file_screenClientXYConst.html: Pointer should be locked
22:56:19     INFO -  272 INFO TEST-UNEXPECTED-FAIL | dom/tests/mochitest/pointerlock/test_pointerlock-api.html | file_screenClientXYConst.html: clientX should be equal to where the mouse was originaly locked - got 250, expected 800
22:56:19     INFO -  273 INFO TEST-UNEXPECTED-FAIL | dom/tests/mochitest/pointerlock/test_pointerlock-api.html | file_screenClientXYConst.html: clientY should be equal to where the mouse was originaly locked - got 265, expected 600
22:56:19     INFO -  274 INFO TEST-UNEXPECTED-FAIL | dom/tests/mochitest/pointerlock/test_pointerlock-api.html | file_screenClientXYConst.html: screenX should be equal to where the mouse was originaly locked - got 250, expected 866
22:56:19     INFO -  275 INFO TEST-UNEXPECTED-FAIL | dom/tests/mochitest/pointerlock/test_pointerlock-api.html | file_screenClientXYConst.html: screenY should be equal to where the mouse was originaly locked - got 265, expected 652
22:56:19     INFO -  [POINTERLOCK] Finishing file_screenClientXYConst.html
22:56:19     INFO -  276 INFO must wait for focus

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

David, you are the original author of this, any tips for what might be causing this failure?  I believe this is a solo test when it is run-by-dir.

Comment 3

[Kaustabh Datta Choudhury]

A recent view of try : https://treeherder.mozilla.org/#/jobs?repo=try&revision=8e64962080d6

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

as a note, this is disabled on everything bug osx:
https://dxr.mozilla.org/mozilla-central/source/dom/tests/mochitest/pointerlock/mochitest.ini#24

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

clearing the needinfo here

Comment 6

[Xidorn Quan [:xidorn] UTC+11]

Can anyone explain why the test is also disabled on Windows?

Comment 7

[Joel Maher ( :jmaher ) (UTC -8)]

we should have updated the title of this, we did have frequent failures with this test on windows (and perma failures on linux) when run-by-dir is used (which is the case for our automation).

Comment 8

[Xidorn Quan [:xidorn] UTC+11]

It looks pretty good for Linux now: https://treeherder.mozilla.org/#/jobs?repo=try&revision=c78a159bd996

Most of the oranges are just "Test attempted to use a flaky timeout value 250" which is fairly simple to suppress. There is one true intermittent, though, which is "file_infiniteMovement.html: Should have moved more than one screen's worth in width.TotalX: 625 Screensize X: 1584". But it happens only once for 200 runs there, so I think it is acceptable.

I'll see whether it is also good for Windows now.

Comment 9

[Xidorn Quan [:xidorn] UTC+11]

It looks like there is a high frequent intermittent failure on Win8 x64 opt build. That is the same intermittent as bug 1174323 which is observed on OS X. We probably can disable this specific subtest on Windows build as well. Or alternatively we can make the subtest more reliable and enable it everywhere.

Comment 10

[Xidorn Quan [:xidorn] UTC+11]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=66ad8399cea2

Comment 11

[Xidorn Quan [:xidorn] UTC+11]

Attachment: MozReview Request: Bug 1180351 - Enable pointerlock tests on Windows and Linux.

Review commit: https://reviewboard.mozilla.org/r/29185/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/29185/

Comment 12

[Olli Pettay [:smaug][[e-mail]]]

Comment on attachment 8702798 [details]
MozReview Request: Bug 1180351 - Enable pointerlock tests on Windows and Linux.

https://reviewboard.mozilla.org/r/29185/#review26005

::: dom/tests/mochitest/pointerlock/file_screenClientXYConst.html:27
(Diff revision 1)
> +      SimpleTest.requestFlakyTimeout("We may need to wait for window's moving");

I don't understand requestFlakyTimeout usage when no setTimeout calls are added

::: dom/tests/mochitest/pointerlock/test_pointerlock-api.html:30
(Diff revision 1)
> +        SimpleTest.requestCompleteLog();

We don't want requestCompleteLog(), right? Is that some debugging only leftover?

::: dom/tests/mochitest/pointerlock/test_pointerlock-api.html:30
(Diff revision 1)
> +        SimpleTest.requestCompleteLog();

We don't want requestCompleteLog(), right? Is that some debugging only leftover?

Comment 13

[Xidorn Quan [:xidorn] UTC+11]

(In reply to Olli Pettay [:smaug] from comment #12)
> Comment on attachment 8702798 [details]
> MozReview Request: Bug 1180351 - Enable pointerlock tests on Windows and
> Linux.
> 
> https://reviewboard.mozilla.org/r/29185/#review26005
> 
> ::: dom/tests/mochitest/pointerlock/file_screenClientXYConst.html:27
> (Diff revision 1)
> > +      SimpleTest.requestFlakyTimeout("We may need to wait for window's moving");
> 
> I don't understand requestFlakyTimeout usage when no setTimeout calls are
> added
> 

There is an existing setTimeout call in the test which is called intermittently. This requestFlakyTimeout is added for that. You can view the complete file in MozReview.

> ::: dom/tests/mochitest/pointerlock/test_pointerlock-api.html:30
> (Diff revision 1)
> > +        SimpleTest.requestCompleteLog();
> 
> We don't want requestCompleteLog(), right? Is that some debugging only
> leftover?

I've removed debugging messages... If you don't think we should keep this, I can remove it.

Comment 14

[Olli Pettay [:smaug][[e-mail]]]

Isn't requestCompleteLog going to create larger logs? I think we should use whatever default we have.

Comment 15

[Olli Pettay [:smaug][[e-mail]]]

Comment on attachment 8702798 [details]
MozReview Request: Bug 1180351 - Enable pointerlock tests on Windows and Linux.

https://reviewboard.mozilla.org/r/29185/#review26009

::: dom/tests/mochitest/pointerlock/test_pointerlock-api.html:30
(Diff revision 1)
> +        SimpleTest.requestCompleteLog();

So I don't see any need for this.

Comment 16

[Xidorn Quan [:xidorn] UTC+11]

https://hg.mozilla.org/integration/mozilla-inbound/rev/b38f36136308100bdeec15e5a29ec99b62991d43
Bug 1180351 - Enable pointerlock tests on Windows and Linux. r=smaug

Comment 17

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/95b69afcce38

Comment 18

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Followup: Disable pointerlock tests on Windows for frequently failing tests on Windows XP opt: https://treeherder.mozilla.org/logviewer.html#?job_id=19158884&repo=mozilla-inbound
https://hg.mozilla.org/integration/mozilla-inbound/rev/95b69afcce38

This also happens on Linux pgo, but rarer: https://treeherder.mozilla.org/logviewer.html#?job_id=19157802&repo=mozilla-inbound

Comment 19

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/b38f36136308
https://hg.mozilla.org/mozilla-central/rev/95b69afcce38