1764564 - Perma ccov tier2 AssertionError: WINDOWSSDKDIR should be in MOZ_CONFIGURE_OPTIONS

Status: RESOLVED FIXED

(Testing :: Code Coverage, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: nerli [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=374329000&repo=mozilla-central
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Wv7J9gyDTqKSfF5Hg-gPAQ/runs/0/artifacts/public/logs/live_backing.log

---

[task 2022-04-13T12:40:08.171Z] 12:40:08     INFO -  AssertionError: WINDOWSSDKDIR should be in MOZ_CONFIGURE_OPTIONS
[task 2022-04-13T12:40:08.231Z] 12:40:08    ERROR - Return code: 1
[task 2022-04-13T12:40:08.231Z] 12:40:08    ERROR - 1 not in success codes: [0]
[task 2022-04-13T12:40:08.231Z] 12:40:08  WARNING - setting return code to 2
[task 2022-04-13T12:40:08.231Z] 12:40:08    FATAL - Halting on failure while running ['z:\\task_164984584216202\\build\\src\\obj-x86_64-pc-mingw32\\_virtualenvs\\common\\Scripts\\python.exe', 'mach', 'python', 'testing\\parse_build_tests_ccov.py']
[task 2022-04-13T12:40:08.240Z] 12:40:08    FATAL - Running post_fatal callback...
[task 2022-04-13T12:40:08.240Z] 12:40:08    FATAL - Exiting 2
[task 2022-04-13T12:40:08.253Z] Creating default state directory: C:\Users\task_164984584216202/.mozbuild
[task 2022-04-13T12:40:08.253Z] Running "pip check" to verify compatibility between the system Python and the "mach" site.
[task 2022-04-13T12:40:08.253Z] Running "pip check" to verify compatibility between the system Python and the "common" site.
[taskcluster 2022-04-13T12:40:08.335Z]    Exit Code: 2
[taskcluster 2022-04-13T12:40:08.335Z]    User Time: 0s
[taskcluster 2022-04-13T12:40:08.335Z]  Kernel Time: 31.25ms
[taskcluster 2022-04-13T12:40:08.335Z]    Wall Time: 1h2m11.913127s
[taskcluster 2022-04-13T12:40:08.335Z]       Result: FAILED
[taskcluster 2022-04-13T12:40:08.335Z] === Task Finished ===
[taskcluster 2022-04-13T12:40:08.335Z] Task Duration: 1h2m11.9160897s
[taskcluster:error] Uploading error artifact public/code-coverage-grcov.zip from file workspace\obj-build\code-coverage-grcov.zip with message "Could not read file 'Z:\\task_164984584216202\\workspace\\obj-build\\code-coverage-grcov.zip'", reason "file-missing-on-worker" and expiry 2023-04-13T09:46:21.111Z
[taskcluster:error] TASK FAILURE during artifact upload: file-missing-on-worker: Could not read file 'Z:\task_164984584216202\workspace\obj-build\code-coverage-grcov.zip'

Comment 1

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Can you look into this Windows code coverage bustage?

Comment 2

[Mike Hommey [:glandium]]

Attachment: Bug 1764564 - Use shellutil.split to split MOZ_CONFIGURE_OPTIONS.

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/80bc28aff0f4
Use shellutil.split to split MOZ_CONFIGURE_OPTIONS. r=marco

Comment 4

[Sandor Molnar[:smolnar]]

@Mike, the assertion failure has disappeared but the build bustage is still ongoing

Failure log

Comment 5

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/80bc28aff0f4

Comment 6

[Mike Hommey [:glandium]]

Attachment: Bug 1764564 - Remove the code to ignore the Windows SDK directory from coverage.

It turns out the only reason it was excluded before is that the SDK used
to be downloaded into the source directory. Since it has moved out of
there, as per the code comment, we no longer need to ignore the SDK and
the code can be removed.

Comment 7

[Mike Hommey [:glandium]]

Attachment: Bug 1764564 - Remove the code to ignore the Windows SDK directory from coverage.

It turns out the only reason it was excluded before is that the SDK used
to be downloaded into the source directory. Since it has moved out of
there, as per the code comment, we no longer need to ignore the SDK and
the code can be removed.

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/709b507ed708
Remove the code to ignore the Windows SDK directory from coverage. r=marco

Comment 9

[Marian-Vasile Laza]

https://hg.mozilla.org/mozilla-central/rev/709b507ed708

Comment 10

[Intermittent Failures Robot]

29 failures in 2948 pushes (0.01 failures/push) were associated with this bug in the last 7 days.

This is the #30 most frequent failure this week.

Repository breakdown:

• mozilla-central: 20
• autoland: 9

Platform and build breakdown:

• windows2012-64: 29
  • ccov: 29

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1764564&startday=2022-04-11&endday=2022-04-17&tree=all