1782377 - Intermittent Task appears to have failed intermittently - exit code 1073807364 found in task payload.onExitStatus list

Status: RESOLVED INCOMPLETE

(Release Engineering :: General, defect, P5)

Description

[Sandor Molnar[:smolnar]]

Failure log

Failed job

Successfully built simplejson
[task 2022-07-30T21:15:53.034Z] 21:15:53     INFO -  Installing collected packages: simplejson
[task 2022-07-30T21:15:53.281Z] 21:15:53     INFO -  Successfully installed simplejson-3.3.0
[task 2022-07-30T21:15:53.394Z] 21:15:53     INFO - Return code: 0
[task 2022-07-30T21:15:53.394Z] 21:15:53     INFO - Installing None into virtualenv Z:\task_165921240898481\build\venv
[task 2022-07-30T21:15:54.051Z] 21:15:54     INFO - find_links: connection checks passed for https://pypi.pub.build.mozilla.org/pub/, adding.
[task 2022-07-30T21:15:54.052Z] 21:15:54     INFO - retry: Calling run_command with args: [['Z:\\task_165921240898481\\build\\venv\\Scripts\\pip', 'install', '--no-deps', '--timeout', '120', '-r', 'Z:\\task_165921240898481\\build\\tests\\config\\marionette_requirements.txt', '--no-index', '--find-links', 'https://pypi.pub.build.mozilla.org/pub/']], kwargs: {'error_list': [{'substr': 'not found or a compiler error:', 'level': 'warning'}, {'regex': re.compile('\\d+: error: '), 'level': 'error'}, {'regex': re.compile('\\d+: warning: '), 'level': 'warning'}, {'regex': re.compile('Downloading .* \\(.*\\): *([0-9]+%)? *[0-9\\.]+[kmKM]b'), 'level': 'debug'}, {'substr': 'command not found', 'level': 'error'}, {'regex': re.compile('Warning:.*Error: '), 'level': 'warning'}, {'regex': re.compile('package.*> Error:'), 'level': 'error'}, {'substr': 'Traceback (most recent call last)', 'level': 'error'}, {'substr': 'SyntaxError: ', 'level': 'error'}, {'substr': 'TypeError: ', 'level': 'error'}, {'substr': 'NameError: ', 'level': 'error'}, {'substr': 'ZeroDivisionError: ', 'level': 'error'}, {'regex': re.compile('raise \\w*Exception: '), 'level': 'critical'}, {'regex': re.compile('raise \\w*Error: '), 'level': 'critical'}], 'cwd': 'Z:\\task_165921240898481\\build\\tests\\config', 'env': {'ALLUSERSPROFILE': 'C:\\ProgramData', 'APPDATA': 'Z:\\task_165921240898481\\AppData\\Roaming', 'COMMONPROGRAMFILES': 'C:\\Program Files\\Common Files', 'COMMONPROGRAMFILES(X86)': 'C:\\Program Files (x86)\\Common Files', 'COMMONPROGRAMW6432': 'C:\\Program Files\\Common Files', 'COMPUTERNAME': 'VM-RIAFB7MUTEE5', 'COMSPEC': 'C:\\Windows\\system32\\cmd.exe', 'DRIVERDATA': 'C:\\Windows\\System32\\Drivers\\DriverData', 'EXTRA_MOZHARNESS_CONFIG': '{"installer_url": "https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/U3aYhW6XRpu-2__rlRc-iA/artifacts/public/build/target.zip", "test_packages_url": "https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/U3aYhW6XRpu-2__rlRc-iA/artifacts/public/build/target.test_packages.json"}', 'GECKO_HEAD_REPOSITORY': 'https://hg.mozilla.org/releases/mozilla-beta', 'GECKO_HEAD_REV': '9bd2ead1ea37ee1175f3bf72e54f21bc10617e59', 'HG_CACHE': 'y:\\tooltool-cache', 'HOMEDRIVE': 'C:', 'HOMEPATH': '\\Users\\task_165921240898481', 'LOCALAPPDATA': 'C:\\Users\\task_165921240898481\\AppData\\Local', 'LOGONSERVER': '\\\\VM-RIAFB7MUTEE5', 'MOZILLABUILD': 'C:\\mozilla-build', 'MOZ_AUTOMATION': '1', 'MOZ_FETCHES': '[{"artifact": "public/build/fix-stacks.tar.zst", "extract": true, "task": "IC9VYDjGQymQW1rjxXKhCw"}, {"artifact": "public/build/minidump-stackwalk.tar.zst", "extract": true, "task": "MG8bm5pGSGGe2xXjvheyEQ"}]', 'MOZ_FETCHES_DIR': 'Z:/task_165921240898481/fetches', 'MOZ_SCM_LEVEL': '3', 'NUMBER_OF_PROCESSORS': '6', 'OS': 'Windows_NT', 'PATH': 'C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\;C:\\Windows\\System32\\OpenSSH\\;C:\\Program Files\\Puppet Labs\\Puppet\\bin;C:\\Program Files\\Git\\cmd;C:\\Program Files (x86)\\GNU\\GnuPG\\pub;C:\\Program Files (x86)\\Windows Kits\\10\\Windows Performance Toolkit\\;C:\\Program Files\\Mercurial\\;C:\\Program Files\\Mercurial;C:\\mozilla-build\\bin;C:\\mozilla-build\\kdiff;C:\\mozilla-build\\moztools-x64\\bin;C:\\mozilla-build\\mozmake;C:\\mozilla-build\\nsis-3.01;C:\\mozilla-build\\python;C:\\mozilla-build\\python\\Scripts;C:\\mozilla-build\\python3;C:\\mozilla-build\\msys\\bin;C:\\mozilla-build\\msys\\local\\bin;C:\\Users\\task_165921240898481\\AppData\\Local\\Microsoft\\WindowsApps', 'PATHEXT': '.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC', 'PIP_DOWNLOAD_CACHE': 'y:\\pip-cache', 'PROCESSOR_ARCHITECTURE': 'AMD64', 'PROCESSOR_IDENTIFIER': 'Intel64 Family 6 Model 63 Stepping 2, GenuineIntel', 'PROCESSOR_LEVEL': '6', 'PROCESSOR_REVISION': '3f02', 'PROGRAMDATA': 'C:\\ProgramData', 'PROGRAMFILES': 'C:\\Program Files', 'PROGRAMFILES(X86)': 'C:\\Program Files (x86)', 'PROGRAMW6432': 'C:\\Program Files', 'PROMPT': '$P$G', 'PSMODULEPATH': '%ProgramFiles%\\WindowsPowerShell\\Modules;C:\\Windows\\system32\\WindowsPowerShell\\v1.0\\Modules', 'PUBLIC': 'C:\\Users\\Public', 'PYTHON': 'c:\\mozilla-build\\python3\\python3.exe', 'RUN_ID': '5', 'SCCACHE_DISABLE': '1', 'SYSTEMDRIVE': 'C:', 'SYSTEMROOT': 'C:\\Windows', 'TASKCLUSTER_ROOT_URL': 'https://firefox-ci-tc.services.mozilla.com', 'TASK_ID': 'LDNi7-TSQ9SAsLzKV7Vd6g', 'TEMP': 'C:\\Users\\task_165921240898481\\AppData\\Local\\Temp', 'TMP': 'C:\\Users\\task_165921240898481\\AppData\\Local\\Temp', 'TOOLTOOL_CACHE': 'C:\\builds\\tooltool_cache', 'USERDOMAIN': 'VM-RIAFB7MUTEE5', 'USERDOMAIN_ROAMINGPROFILE': 'VM-RIAFB7MUTEE5', 'USERNAME': 'task_165921240898481', 'USERPROFILE': 'C:\\Users\\task_165921240898481', 'WINDIR': 'C:\\Windows'}, 'error_level': 'warning'}, attempt #1
[task 2022-07-30T21:15:54.052Z] 21:15:54     INFO - Running command: ['Z:\\task_165921240898481\\build\\venv\\Scripts\\pip', 'install', '--no-deps', '--timeout', '120', '-r', 'Z:\\task_165921240898481\\build\\tests\\config\\marionette_requirements.txt', '--no-index', '--find-links', 'https://pypi.pub.build.mozilla.org/pub/'] in Z:\task_165921240898481\build\tests\config
[task 2022-07-30T21:15:54.052Z] 21:15:54     INFO - Copy/paste: Z:\task_165921240898481\build\venv\Scripts\pip install --no-deps --timeout 120 -r Z:\task_165921240898481\build\tests\config\marionette_requirements.txt --no-index --find-links https://pypi.pub.build.mozilla.org/pub/
[task 2022-07-30T21:15:54.052Z] 21:15:54     INFO - Using env: (same as previous command)
[task 2022-07-30T21:15:54.842Z] 21:15:54     INFO -  Looking in links: https://pypi.pub.build.mozilla.org/pub/
[task 2022-07-30T21:15:54.842Z] 21:15:54     INFO -  Processing z:\task_165921240898481\build\tests\tools\mozterm
[task 2022-07-30T21:15:54.844Z] 21:15:54     INFO -    DEPRECATION: A future pip version will change local packages to be built in-place without first copying to a temporary directory. We recommend you use --use-feature=in-tree-build to test your packages with this new behavior before it becomes the default.
[task 2022-07-30T21:15:54.844Z] 21:15:54     INFO -     pip 21.3 will remove support for this functionality. You can find discussion regarding this at https://github.com/pypa/pip/issues/7555.
[task 2022-07-30T21:15:55.366Z] 21:15:55     INFO -  Processing z:\task_165921240898481\build\tests\tools\geckoprocesstypes_generator
[task 2022-07-30T21:15:55.863Z] 21:15:55     INFO -  Processing z:\task_165921240898481\build\tests\mozbase\manifestparser
[task 2022-07-30T21:15:56.458Z] 21:15:56     INFO -  Processing z:\task_165921240898481\build\tests\mozbase\mozcrash
[task 2022-07-30T21:15:56.984Z] 21:15:56     INFO -  Processing z:\task_165921240898481\build\tests\mozbase\mozdebug
[task 2022-07-30T21:15:57.505Z] 21:15:57     INFO -  Processing z:\task_165921240898481\build\tests\mozbase\mozdevice
[task 2022-07-30T21:15:58.003Z] 21:15:58     INFO -  Processing z:\task_165921240898481\build\tests\mozbase\mozfile
[task 2022-07-30T21:15:58.501Z] 21:15:58     INFO -  Processing z:\task_165921240898481\build\tests\mozbase\mozgeckoprofiler
[task 2022-07-30T21:15:58.986Z] 21:15:58     INFO -  Processing z:\task_165921240898481\build\tests\mozbase\mozhttpd
[task 2022-07-30T21:15:59.471Z] 21:15:59     INFO -  Processing z:\task_165921240898481\build\tests\mozbase\mozinfo
[task 2022-07-30T21:15:59.977Z] 21:15:59     INFO -  Processing z:\task_165921240898481\build\tests\mozbase\mozinstall
[task 2022-07-30T21:16:00.486Z] 21:16:00     INFO -  Processing z:\task_165921240898481\build\tests\mozbase\mozleak
[task 2022-07-30T21:16:00.952Z] 21:16:00     INFO -  Processing z:\task_165921240898481\build\tests\mozbase\mozlog
[task 2022-07-30T21:16:01.517Z] 21:16:01     INFO -  Processing z:\task_165921240898481\build\tests\mozbase\moznetwork
[taskcluster 2022-07-30T21:16:02.044Z]    Exit Code: 1073807364
[taskcluster 2022-07-30T21:16:02.044Z]    User Time: 15.625ms
[taskcluster 2022-07-30T21:16:02.044Z]  Kernel Time: 15.625ms
[taskcluster 2022-07-30T21:16:02.044Z]    Wall Time: 3m57.2567821s
[taskcluster 2022-07-30T21:16:02.044Z]       Result: FAILED
[taskcluster 2022-07-30T21:16:02.045Z] === Task Finished ===
[taskcluster 2022-07-30T21:16:02.045Z] Task Duration: 3m57.2624327s
[taskcluster 2022-07-30T21:16:02.187Z] Uploading artifact public/logs/localconfig.json from file logs\localconfig.json with content encoding "gzip", mime type "application/json" and expiry 2023-07-30T19:06:39.919Z
[taskcluster 2022-07-30T21:16:02.687Z] Uploading link artifact public/logs/live.log to artifact public/logs/live_backing.log with expiry 2023-07-30T19:06:39.919Z
[taskcluster:error] Task appears to have failed intermittently - exit code 1073807364 found in task payload.onExitStatus list

Comment 1

[Intermittent Failures Robot]

1 failures in 3245 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1

Platform and build breakdown:

• windows10-64-2004-asan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1782377&startday=2022-07-25&endday=2022-07-31&tree=all

Comment 2

[Geoff Brown [:gbrown]]

https://searchfox.org/mozilla-central/rev/c2a2bf5a49626d63c4c0a5be42b6a93838c1b595/taskcluster/gecko_taskgraph/transforms/task.py#705

Comment 3

[Intermittent Failures Robot]

2 failures in 3558 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1
• mozilla-central: 1

Platform and build breakdown:

• windows2012-64: 1
  • debug: 1
• windows2012-64-shippable: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1782377&startday=2022-12-12&endday=2022-12-18&tree=all

Comment 4

[Heitor Neiva [:hneiva]]

Closing this ticket as there hasn't been any recent failures.

Comment 5

[Natalia Csoregi [:nataliaCs]]

Recent job: https://treeherder.mozilla.org/jobs?repo=mozilla-beta&searchStr=windows%2C11%2Cx64%2C22h2%2Casan%2Cwebrender%2Copt%2Creftests%2Cwith%2Cdcomp%2Cpresent%2Cbut%2Cnot%2Coverlays%2Ctest-windows11-64-2009-asan-qr%2Fopt-reftest-wr-dc1-p%2Cr2&revision=ba264d129db8e726266909809edd0025d63131bb&selectedTaskRun=UdgmyKfLTQ6aSo6AUNXXOA.5

Raw log: https://firefox-ci-tc.services.mozilla.com/tasks/UdgmyKfLTQ6aSo6AUNXXOA/runs/0/logs/public/logs/live.log

[task 2024-02-22T16:57:20.268Z] 16:57:20     INFO - REFTEST TEST-START | layout/reftests/margin-collapsing/block-max-height-last-child-3a.html == layout/reftests/margin-collapsing/block-max-height-last-child-3-ref.html
[task 2024-02-22T16:57:20.269Z] 16:57:20     INFO - REFTEST TEST-LOAD | file:///Z:/task_170862036455023/build/tests/reftest/tests/layout/reftests/margin-collapsing/block-max-height-last-child-3a.html | 357 / 716 (49%)
[taskcluster 2024-02-22T16:57:20.441Z]    Exit Code: 1073807364
[taskcluster 2024-02-22T16:57:20.441Z]    User Time: 0s
[taskcluster 2024-02-22T16:57:20.441Z]  Kernel Time: 0s
[taskcluster 2024-02-22T16:57:20.441Z]    Wall Time: 8m46.9316406s
[taskcluster 2024-02-22T16:57:20.441Z]       Result: FAILED
[taskcluster 2024-02-22T16:57:20.442Z] === Task Finished ===
[taskcluster 2024-02-22T16:57:20.442Z] Task Duration: 8m46.9368304s
[taskcluster:error] Uploading error artifact public/logs/localconfig.json from file logs\localconfig.json with message "Could not copy file 'Z:\\task_170862036455023\\logs\\localconfig.json' to temporary location as task user: Failed to copy file Z:\\task_170862036455023\\logs\\localconfig.json to temporary location as task user: exit status 0xc0000142", reason "file-not-readable-on-worker" and expiry 2024-05-22T16:12:18.899Z
[taskcluster:error] TASK FAILURE during artifact upload: file-not-readable-on-worker: Could not copy file 'Z:\task_170862036455023\logs\localconfig.json' to temporary location as task user: Failed to copy file Z:\task_170862036455023\logs\localconfig.json to temporary location as task user: exit status 0xc0000142
[taskcluster:error] Uploading error artifact public/test_info/reftest_errorsummary.log from file build\blobber_upload_dir\reftest_errorsummary.log with message "Could not copy file 'Z:\\task_170862036455023\\build\\blobber_upload_dir\\reftest_errorsummary.log' to temporary location as task user: Failed to copy file Z:\\task_170862036455023\\build\\blobber_upload_dir\\reftest_errorsummary.log to temporary location as task user: exit status 0xc0000142", reason "file-not-readable-on-worker" and expiry 2024-05-22T16:12:18.899Z
[taskcluster:error] TASK FAILURE during artifact upload: file-not-readable-on-worker: Could not copy file 'Z:\task_170862036455023\build\blobber_upload_dir\reftest_errorsummary.log' to temporary location as task user: Failed to copy file Z:\task_170862036455023\build\blobber_upload_dir\reftest_errorsummary.log to temporary location as task user: exit status 0xc0000142
[taskcluster:error] Uploading error artifact public/test_info/system-info.log from file build\blobber_upload_dir\system-info.log with message "Could not copy file 'Z:\\task_170862036455023\\build\\blobber_upload_dir\\system-info.log' to temporary location as task user: Failed to copy file Z:\\task_170862036455023\\build\\blobber_upload_dir\\system-info.log to temporary location as task user: exit status 0xc0000142", reason "file-not-readable-on-worker" and expiry 2024-05-22T16:12:18.899Z
[taskcluster:error] TASK FAILURE during artifact upload: file-not-readable-on-worker: Could not copy file 'Z:\task_170862036455023\build\blobber_upload_dir\system-info.log' to temporary location as task user: Failed to copy file Z:\task_170862036455023\build\blobber_upload_dir\system-info.log to temporary location as task user: exit status 0xc0000142
[taskcluster 2024-02-22T16:57:20.720Z] Uploading link artifact public/logs/live.log to artifact public/logs/live_backing.log with expiry 2024-05-22T16:12:18.899Z
[taskcluster:error] Task appears to have failed intermittently - exit code 1073807364 found in task payload.onExitStatus list

Comment 6

[Intermittent Failures Robot]

3 failures in 3382 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 2
• mozilla-central: 1

Platform and build breakdown:

• windows11-64-2009-qr: 2
  • debug: 2
• windows11-64-2009-asan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1782377&startday=2024-02-19&endday=2024-02-25&tree=all

Comment 7

[Intermittent Failures Robot]

5 failures in 4052 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1
• try: 1
• autoland: 2
• mozilla-central: 1

Platform and build breakdown:

• windows11-32-2009-qr: 2
  • debug: 2
• windows11-64-2009-asan-qr: 2
  • opt: 2
• windows11-64-2009-mingwclang-qr: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1782377&startday=2024-02-26&endday=2024-03-03&tree=all

Comment 8

[Intermittent Failures Robot]

1 failures in 4198 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• windows11-64-2009-qr: 1
  • debug: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1782377&startday=2024-03-04&endday=2024-03-10&tree=all

Comment 9

[Intermittent Failures Robot]

1 failures in 3849 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• windows11-64-2009-asan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1782377&startday=2024-03-11&endday=2024-03-17&tree=all

Comment 10

[BugBot [:suhaib / :marco/ :calixte]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit BugBot documentation.

Comment 11

[Treeherder Bug Filer]

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=473272597&repo=autoland

Comment 12

[Intermittent Failures Robot]

1 failures in 3668 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• android-hw-a51-11-0-aarch64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1782377&startday=2024-09-02&endday=2024-09-08&tree=all

Comment 13

[BugBot [:suhaib / :marco/ :calixte]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit BugBot documentation.