1384724 - Stylo: Windows reftest runs frequently crash at suite start

Status: RESOLVED FIXED

(Core :: CSS Parsing and Computation, defect, P1)

Description

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

Many (but not all) Windows reftest runs are crashing during suite start:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=6813000f2a777de4ed8d410522ef9dc4bf8f6de5&filter-searchStr=stylo%20windows%20tc-R

It seems more prevalent for stylo vs. gecko runs.

An example log:

https://treeherder.mozilla.org/logviewer.html#?job_id=117750949&repo=try&lineNumber=1742

Nothing in the log seemed like an obvious cause to me.

Comment 1

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

I'll try to reproduce on a Windows loaner.

Comment 2

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

(In reply to J. Ryan Stinnett [:jryans] (use ni?) from comment #1)
> I'll try to reproduce on a Windows loaner.

Currently this is failing because the loaner dies in ~5 minutes each time.  Working on it with TC team in bug 1384752.

Comment 3

[Chris Peterson [:cpeterson]]

Assigning to jryans because it looks like he is working on it. :)

Comment 4

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=64aa9060b3dec950c54bffb1a2084fc7c5bf96f8

Comment 5

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

Attachment: Bug 1384724 - Use matching virtualization for Stylo tests.

We need to use the same virtualization setting as the regular tests that these
Stylo tests were copied from to ensure we get the right machine type for
Windows runs.

Review commit: https://reviewboard.mozilla.org/r/163620/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/163620/

Comment 6

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

After :bholley's hint in today's meeting, I discovered that the special Stylo tests don't use the same virtualization setting as the main tests there were copied from.  Setting that appears to fix this issue.

Comment 7

[Chris AtLee [:catlee]]

Comment on attachment 8892633 [details]
Bug 1384724 - Use matching virtualization for Stylo tests.

https://reviewboard.mozilla.org/r/163620/#review169228

::: taskcluster/ci/test/tests.yml:669
(Diff revision 1)
>  mochitest-chrome-style:
>      description: "Mochitest chrome run for style system"
>      suite: mochitest/chrome-style
>      treeherder-symbol: tc-M(cs)
>      loopback-video: true
> +    virtualization: hardware

I don't see mochitest-chrome in your try push. Did you see this working elsewhere?

You may need to specify the worker type to buildbot-bridge as in mochitest-chrome.

Comment 8

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

Comment on attachment 8892633 [details]
Bug 1384724 - Use matching virtualization for Stylo tests.

https://reviewboard.mozilla.org/r/163620/#review169228

> I don't see mochitest-chrome in your try push. Did you see this working elsewhere?
> 
> You may need to specify the worker type to buildbot-bridge as in mochitest-chrome.

Ah, good call.  A try run[1] that includes them has those jobs stuck pending, so perhaps this is to blame.

I'll try to roll back to regular virtualization or virtual with GPU for this job first.

[1]: https://treeherder.mozilla.org/#/jobs?repo=try&revision=a31ea51c6ac70267fb3abfe427ad5085501ec9a0&filter-searchStr=M(cs)

Comment 9

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

mochitest-chrome-style using virtual:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=03fe4e55d8e202c5d7cdee9e2c9ad426bca43fb5

mochitest-chrome-style using virtual-with-gpu:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=7a8e876e213504d6e03bb488653166e21890eba2

Comment 10

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

(In reply to J. Ryan Stinnett [:jryans] (use ni?) from comment #9)
> mochitest-chrome-style using virtual:
> 
> https://treeherder.mozilla.org/#/
> jobs?repo=try&revision=03fe4e55d8e202c5d7cdee9e2c9ad426bca43fb5
> 
> mochitest-chrome-style using virtual-with-gpu:
> 
> https://treeherder.mozilla.org/#/
> jobs?repo=try&revision=7a8e876e213504d6e03bb488653166e21890eba2

Looks like regular virtualization is enough for these jobs for now (I had only added "hardware" to make things more consistent).

Something to watch out for if we grow the mochitest-chrome-style task to run more directories, but for now I'll revert it to regular virtualization.

Comment 11

[J. Ryan Stinnett [:jryans] (Use needinfo, replies may be slow)]

Comment on attachment 8892633 [details]
Bug 1384724 - Use matching virtualization for Stylo tests.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/163620/diff/1-2/

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2b56de144b83
Use matching virtualization for Stylo tests. r=catlee

Comment 13

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/2b56de144b83

Comment 14

[Chris Peterson [:cpeterson]]

We aren't running Stylo tests on Windows (bug 1385027) in Beta 56, so we don't need to uplift this test fix to Beta 56.

Comment 15

[Aki Sasaki (not active)]

https://hg.mozilla.org/projects/date/rev/2b56de144b83884068364c4569e249c375fcd67b
Bug 1384724 - Use matching virtualization for Stylo tests. r=catlee