1900155 - Perma [tier 2] Windows 11 x64 22H2 WebRender Ref HW opt M-mda-gpu Halting on failure while running ('powershell', '-command', 'if (-Not ((Get-CimInstance win32_VideoController).InstalledDisplayDrivers | Out-String).contains("nvgrid")) { echo "Missing nvgr

Status: RESOLVED FIXED

(Testing :: General, defect)

Description

[Norisz Fay [:noriszfay]]

Full log: https://firefox-ci-tc.services.mozilla.com/tasks/A0rvshnoQtKGLMap29aL9Q/runs/5/logs/public/logs/live.log#L1549

[task 2024-05-31T23:44:22.749Z] 23:44:22     INFO - Running pre test command ensure proper graphics driver with 'powershell -command if (-Not ((Get-CimInstance win32_VideoController).InstalledDisplayDrivers | Out-String).contains("nvgrid")) { echo "Missing nvgrid driver: " + (Get-CimInstance win32_VideoController).InstalledDisplayDrivers; exit 4; }'
[task 2024-05-31T23:44:22.749Z] 23:44:22     INFO - Running command: ('powershell', '-command', 'if (-Not ((Get-CimInstance win32_VideoController).InstalledDisplayDrivers | Out-String).contains("nvgrid")) { echo "Missing nvgrid driver: " + (Get-CimInstance win32_VideoController).InstalledDisplayDrivers; exit 4; }') in C:\task_171719878214741\build
[task 2024-05-31T23:44:22.749Z] 23:44:22     INFO - Copy/paste: powershell -command "if (-Not ((Get-CimInstance win32_VideoController).InstalledDisplayDrivers | Out-String).contains(\"nvgrid\")) { echo \"Missing nvgrid driver: \" + (Get-CimInstance win32_VideoController).InstalledDisplayDrivers; exit 4; }"
[task 2024-05-31T23:44:23.250Z] 23:44:23     INFO -  Missing nvgrid driver:
[task 2024-05-31T23:44:23.250Z] 23:44:23     INFO -  +
[task 2024-05-31T23:44:23.250Z] 23:44:23     INFO -  <>,C:\Windows\System32\DriverStore\FileRepository\iigd_dch.inf_amd64_50491fb294f8338f\igd10iumd64.dll,C:\Windows\System32\DriverStore\FileRepository\iigd_dch.inf_amd64_50491fb294f8338f\igd10iumd64.dll,C:\Windows\System32\DriverStore\FileRepository\iigd_dch.inf_amd64_50491fb294f8338f\igd12umd64.dll
[task 2024-05-31T23:44:23.261Z] 23:44:23     INFO - Return code: 4
[task 2024-05-31T23:44:23.261Z] 23:44:23    ERROR - 4 not in success codes: [0]
[task 2024-05-31T23:44:23.261Z] 23:44:23  WARNING - setting return code to 4
[task 2024-05-31T23:44:23.261Z] 23:44:23    FATAL - Halting on failure while running ('powershell', '-command', 'if (-Not ((Get-CimInstance win32_VideoController).InstalledDisplayDrivers | Out-String).contains("nvgrid")) { echo "Missing nvgrid driver: " + (Get-CimInstance win32_VideoController).InstalledDisplayDrivers; exit 4; }')
[task 2024-05-31T23:44:23.261Z] 23:44:23    FATAL - Running post_fatal callback...
[task 2024-05-31T23:44:23.261Z] 23:44:23    FATAL - Exiting 4
[task 2024-05-31T23:44:23.261Z] 23:44:23     INFO - Running post-action listener: _package_coverage_data
[task 2024-05-31T23:44:23.261Z] 23:44:23     INFO - Running post-action listener: _resource_record_post_action
[task 2024-05-31T23:44:23.261Z] 23:44:23     INFO - Running post-action listener: process_java_coverage_data
[task 2024-05-31T23:44:23.261Z] 23:44:23     INFO - [mozharness: 2024-05-31 23:44:23.261264Z] Finished run-tests step (failed)
[task 2024-05-31T23:44:23.261Z] 23:44:23     INFO - Running post-run listener: _resource_record_post_run
[task 2024-05-31T23:44:23.344Z] 23:44:23     INFO - instance_metadata.json not found; unable to determine instance type
[task 2024-05-31T23:44:23.344Z] 23:44:23     INFO - Validating Perfherder data against C:\task_171719878214741\mozharness\external_tools\performance-artifact-schema.json
[task 2024-05-31T23:44:23.348Z] 23:44:23     INFO - PERFHERDER_DATA: {"framework": {"name": "job_resource_usage"}, "suites": [{"name": "mochitest.mochitest-media.overall", "extraOptions": ["e10s", "buildbot-unknown"], "subtests": [{"name": "cpu_percent", "value": 4.201944444444444}, {"name": "io_write_bytes", "value": 155680768}, {"name": "io.read_bytes", "value": 14596096}, {"name": "io_write_time", "value": 0}, {"name": "io_read_time", "value": 0}]}, {"name": "mochitest.mochitest-media.start-pulseaudio", "subtests": [{"name": "time", "value": 0.0}, {"name": "cpu_percent", "value": 0}]}, {"name": "mochitest.mochitest-media.install", "subtests": [{"name": "time", "value": 1.9209999999999923}, {"name": "cpu_percent", "value": 5.026041666666666}]}, {"name": "mochitest.mochitest-media.stage-files", "subtests": [{"name": "time", "value": 0.0}, {"name": "cpu_percent", "value": 0}]}, {"name": "mochitest.mochitest-media.run-tests", "subtests": [{"name": "time", "value": 1.4380000000000166}, {"name": "cpu_percent", "value": 3.2974358974358973}]}]}
[task 2024-05-31T23:44:23.348Z] 23:44:23     INFO - Total resource usage - Wall time: 3s; CPU: 3%; Read bytes: 14596096; Write bytes: 155680768; Read time: 0; Write time: 0
[task 2024-05-31T23:44:23.348Z] 23:44:23     INFO - TinderboxPrint: CPU usage<br/>3.3%
[task 2024-05-31T23:44:23.348Z] 23:44:23     INFO - TinderboxPrint: I/O read bytes / time<br/>14,596,096 / 0
[task 2024-05-31T23:44:23.348Z] 23:44:23     INFO - TinderboxPrint: I/O write bytes / time<br/>155,680,768 / 0
[task 2024-05-31T23:44:23.348Z] 23:44:23     INFO - TinderboxPrint: CPU idle<br/>34.7 (95.8%)
[task 2024-05-31T23:44:23.348Z] 23:44:23     INFO - TinderboxPrint: CPU system<br/>0.6 (1.6%)
[task 2024-05-31T23:44:23.348Z] 23:44:23     INFO - TinderboxPrint: CPU user<br/>0.9 (2.5%)
[task 2024-05-31T23:44:23.348Z] 23:44:23     INFO - start-pulseaudio - Wall time: 0s; CPU: Can't collect data; Read bytes: 0; Write bytes: 0; Read time: 0; Write time: 0
[task 2024-05-31T23:44:23.348Z] 23:44:23     INFO - install - Wall time: 2s; CPU: 5%; Read bytes: 11607552; Write bytes: 132280320; Read time: 0; Write time: 0
[task 2024-05-31T23:44:23.348Z] 23:44:23     INFO - stage-files - Wall time: 0s; CPU: Can't collect data; Read bytes: 0; Write bytes: 0; Read time: 0; Write time: 0
[task 2024-05-31T23:44:23.348Z] 23:44:23     INFO - run-tests - Wall time: 1s; CPU: 3%; Read bytes: 2988544; Write bytes: 15056896; Read time: 0; Write time: 0
[taskcluster 2024-05-31T23:44:23.396Z]    Exit Code: 4
[taskcluster 2024-05-31T23:44:23.396Z]    User Time: 0s
[taskcluster 2024-05-31T23:44:23.396Z]  Kernel Time: 0s
[taskcluster 2024-05-31T23:44:23.396Z]    Wall Time: 1m19.3256518s
[taskcluster 2024-05-31T23:44:23.396Z]       Result: FAILED
[taskcluster 2024-05-31T23:44:23.396Z] === Task Finished ===
[taskcluster 2024-05-31T23:44:23.396Z] Task Duration: 1m19.3273761s
[taskcluster 2024-05-31T23:44:23.592Z] Uploading artifact public/logs/localconfig.json from file C:\task_171719878214741\logs\localconfig.json with content encoding "gzip", mime type "application/json" and expiry 2025-05-31T23:18:45.147Z
[taskcluster 2024-05-31T23:44:23.876Z] Uploading artifact public/test_info/profile_resource-usage.json from file C:\task_171719878214741\build\blobber_upload_dir\profile_resource-usage.json with content encoding "gzip", mime type "application/json" and expiry 2025-05-31T23:18:45.147Z
[taskcluster 2024-05-31T23:44:24.143Z] Uploading artifact public/test_info/resource-usage.json from file C:\task_171719878214741\build\blobber_upload_dir\resource-usage.json with content encoding "gzip", mime type "application/json" and expiry 2025-05-31T23:18:45.147Z
[taskcluster 2024-05-31T23:44:24.613Z] Uploading link artifact public/logs/live.log to artifact public/logs/live_backing.log with expiry 2025-05-31T23:18:45.147Z
[taskcluster:error] task appears to have failed intermittently - exit code 4 found in task payload.onExitStatus list

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

ah, I should have known there is an edge case. I can find a way to work around this. These jobs were not on try by default (requires the --full)

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1899536

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: Bug 1900155 - do not add requires-gpu for hardware tasks. r=aryx

Comment 4

[Intermittent Failures Robot]

37 failures in 4202 pushes (0.009 failures/push) were associated with this bug in the last 7 days.

This is the #50 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 31
• mozilla-central: 4
• try: 2

Platform and build breakdown:

• windows11-64-2009-hw-ref: 20
  • opt: 20
• windows11-64-2009-hw-ref-shippable: 16
  • opt: 16
• lint: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1900155&startday=2024-05-27&endday=2024-06-02&tree=all

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/753a18ce6938
do not add requires-gpu for hardware tasks. r=aryx,taskgraph-reviewers,ahal

Comment 6

[Intermittent Failures Robot]

31 failures in 591 pushes (0.052 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-central: 2
• autoland: 28
• try: 1

Platform and build breakdown:

• windows11-64-2009-hw-ref-shippable: 6
  • opt: 6
• windows11-64-2009-hw-ref: 25
  • opt: 25

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1900155&startday=2024-06-03&endday=2024-06-03&tree=all

Comment 7

[Iulian Moraru]

https://hg.mozilla.org/mozilla-central/rev/753a18ce6938

Comment 8

[Intermittent Failures Robot]

31 failures in 3649 pushes (0.008 failures/push) were associated with this bug in the last 7 days.

This is the #43 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-central: 2
• autoland: 28
• try: 1

Platform and build breakdown:

• windows11-64-2009-hw-ref-shippable: 6
  • opt: 6
• windows11-64-2009-hw-ref: 25
  • opt: 25

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1900155&startday=2024-06-03&endday=2024-06-09&tree=all