Closed Bug 1503453 Opened Last year Closed 8 months ago

Perma 'mach static-analysis autotest --intree-tool' did not run successfully. Please check log for errors.

Categories

(Firefox Build System :: Source Code Analysis, defect, P1, major)

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: intermittent-bug-filer, Assigned: andi)

References

(Regression)

Details

(Keywords: intermittent-failure, Whiteboard: [stockwell fixed])

Attachments

(2 files)

Filed by: rmaries [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=208737489&repo=autoland

https://queue.taskcluster.net/v1/task/PG8QCrIST_eKSSy9dSSK_A/runs/0/artifacts/public/logs/live_backing.log

https://treeherder.mozilla.org/logviewer.html#?job_id=208737489&repo=autoland&lineNumber=1995

[task 2018-10-30T22:39:01.776Z] ERROR: infer failed to execute gradle [u'autotest:compileInferTestStarvation'].
[task 2018-10-30T22:39:01.777Z] RUNNING: infer check racerd.
[task 2018-10-30T22:39:03.542Z] RUNNING: infer check checkers.
[task 2018-10-30T22:39:11.108Z] 22:39:11  WARNING - setting return code to 2
[task 2018-10-30T22:39:11.109Z] 22:39:11    FATAL - 'mach static-analysis autotest --intree-tool' did not run successfully. Please check log for errors.
[task 2018-10-30T22:39:11.110Z] 22:39:11    FATAL - Running post_fatal callback...
[task 2018-10-30T22:39:11.112Z] 22:39:11    FATAL - Exiting -1
[task 2018-10-30T22:39:11.113Z] 22:39:11     INFO - [mozharness: 2018-10-30 22:39:11.112870Z] Finished static-analysis-autotest step (failed)
[task 2018-10-30T22:39:11.113Z] 22:39:11     INFO - Running post-run listener: _shutdown_sccache
[task 2018-10-30T22:39:11.114Z] 22:39:11     INFO - Running command: ['/builds/worker/workspace/build/src/sccache2/sccache', '--stop-server'] in /builds/worker/workspace/build/src
[task 2018-10-30T22:39:11.115Z] 22:39:11     INFO - Copy/paste: /builds/worker/workspace/build/src/sccache2/sccache --stop-server
[task 2018-10-30T22:39:11.120Z] 22:39:11     INFO -  Stopping sccache server...
[task 2018-10-30T22:39:11.121Z] 22:39:11     INFO -  Compile requests               337
[task 2018-10-30T22:39:11.122Z] 22:39:11     INFO -  Compile requests executed      202
[task 2018-10-30T22:39:11.123Z] 22:39:11     INFO -  Cache hits                     168
[task 2018-10-30T22:39:11.124Z] 22:39:11     INFO -  Cache misses                     0
[task 2018-10-30T22:39:11.125Z] 22:39:11     INFO -  Cache timeouts                   0
[task 2018-10-30T22:39:11.126Z] 22:39:11     INFO -  Cache read errors                0
[task 2018-10-30T22:39:11.126Z] 22:39:11     INFO -  Forced recaches                  0
[task 2018-10-30T22:39:11.128Z] 22:39:11     INFO -  Cache write errors               0
[task 2018-10-30T22:39:11.128Z] 22:39:11     INFO -  Compilation failures             8
[task 2018-10-30T22:39:11.129Z] 22:39:11     INFO -  Cache errors                    26
[task 2018-10-30T22:39:11.130Z] 22:39:11     INFO -  Non-cacheable compilations       0
[task 2018-10-30T22:39:11.131Z] 22:39:11     INFO -  Non-cacheable calls             30
[task 2018-10-30T22:39:11.132Z] 22:39:11     INFO -  Non-compilation calls          105
[task 2018-10-30T22:39:11.133Z] 22:39:11     INFO -  Unsupported compiler calls       0
[task 2018-10-30T22:39:11.134Z] 22:39:11     INFO -  Average cache write          0.000 s
[task 2018-10-30T22:39:11.135Z] 22:39:11     INFO -  Average cache read miss      0.000 s
[task 2018-10-30T22:39:11.136Z] 22:39:11     INFO -  Average cache read hit       0.022 s
[task 2018-10-30T22:39:11.137Z] 22:39:11     INFO -  Cache location             S3, bucket: Bucket(name=taskcluster-level-3-sccache-us-west-1, base_url=http://taskcluster-level-3-sccache-us-west-1.s3.amazonaws.com/)
[task 2018-10-30T22:39:11.138Z] 22:39:11     INFO - Return code: 0
[task 2018-10-30T22:39:11.139Z] 22:39:11     INFO - Running post-run listener: _summarize
[task 2018-10-30T22:39:11.140Z] 22:39:11    ERROR - # TBPL FAILURE #
[task 2018-10-30T22:39:11.140Z] 22:39:11     INFO - [mozharness: 2018-10-30 22:39:11.140704Z] FxDesktopBuild summary:
[task 2018-10-30T22:39:11.141Z] 22:39:11    ERROR - # TBPL FAILURE #
[taskcluster 2018-10-30 22:39:11.898Z] === Task Finished ===
[taskcluster 2018-10-30 22:39:19.174Z] Unsuccessful task run with exit code: 255 completed in 1281.868 seconds
Status: NEW → RESOLVED
Closed: 11 months ago
Resolution: --- → INCOMPLETE

Recent log fsilure: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=237484382&repo=autoland&lineNumber=3949

03:01:37 FATAL - 'mach static-analysis autotest --intree-tool' did not run successfully. Please check log for errors.
03:01:37 FATAL - Running post_fatal callback...
03:01:37 FATAL - Exiting -1
03:01:37 INFO - [mozharness: 2019-04-02 03:01:37.432000Z] Finished static-analysis-autotest step (failed)
03:01:37 INFO - Running post-run listener: _parse_build_tests_ccov
03:01:37 INFO - Running post-run listener: _shutdown_sccache
03:01:37 INFO - Running command: ['z:\build\build\src\sccache2\sccache.exe', '--stop-server'] in z:\build\build\src
03:01:37 INFO - Copy/paste: z:\build\build\src\sccache2\sccache.exe --stop-server
03:01:37 INFO - Stopping sccache server...
03:01:37 INFO - Compile requests 100
03:01:37 INFO - Compile requests executed 63
03:01:37 INFO - Cache hits 12
03:01:37 INFO - Cache misses 22
03:01:37 INFO - Cache timeouts 0
03:01:37 INFO - Cache read errors 0
03:01:37 INFO - Forced recaches 0
03:01:37 INFO - Cache write errors 0
03:01:37 INFO - Compilation failures 0
03:01:37 INFO - Cache errors 29
03:01:37 INFO - Non-cacheable compilations 0
03:01:37 INFO - Non-cacheable calls 12
03:01:37 INFO - Non-compilation calls 25
03:01:37 INFO - Unsupported compiler calls 0
03:01:37 INFO - Successful distributed compilations 0
03:01:37 INFO - Failed distributed compilations 0
03:01:37 INFO - Average cache write 0.044 s
03:01:37 INFO - Average cache read miss 0.055 s
03:01:37 INFO - Average cache read hit 0.030 s
03:01:37 INFO - Non-cacheable reasons:
03:01:37 INFO - unknown source language 4
03:01:37 INFO - multiple input files 8
03:01:37 INFO - Cache location S3, bucket: Bucket(name=taskcluster-level-3-sccache-us-east-1, base_url=http://taskcluster-level-3-sccache-us-east-1.s3.amazonaws.com/)
03:01:37 INFO - Return code: 0
03:01:37 INFO - Running post-run listener: _summarize
03:01:37 ERROR - # TBPL FAILURE #
03:01:37 INFO - [mozharness: 2019-04-02 03:01:37.443000Z] FxDesktopBuild summary:
03:01:37 ERROR - # TBPL FAILURE #
[taskcluster 2019-04-02T03:01:37.460Z] Exit Code: 4294967295
[taskcluster 2019-04-02T03:01:37.460Z] User Time: 0s
[taskcluster 2019-04-02T03:01:37.460Z] Kernel Time: 0s
[taskcluster 2019-04-02T03:01:37.460Z] Wall Time: 5m58.9635791s
[taskcluster 2019-04-02T03:01:37.460Z] Result: FAILED
[taskcluster 2019-04-02T03:01:37.460Z] === Task Finished ===
[taskcluster 2019-04-02T03:01:37.460Z] Task Duration: 31m47.2372323s
[taskcluster 2019-04-02T03:01:37.755Z] Uploading artifact public/logs/localconfig.json from file logs\localconfig.json with content encoding "gzip", mime type "application/octet-stream" and expiry 2020-04-01T01:57:01.389Z
[taskcluster 2019-04-02T03:01:38.117Z] [mounts] Preserving cache: Moving "Z:\task_1554170625\build" to "Z:\caches\USX_OlZbQG6lGQGeooS6xw"
[taskcluster 2019-04-02T03:01:38.727Z] [mounts] Denying task_1554170625 access to 'Z:\caches\USX_OlZbQG6lGQGeooS6xw'
[taskcluster 2019-04-02T03:02:06.901Z] Uploading redirect artifact public/logs/live.log to URL https://queue.taskcluster.net/v1/task/XPFHBOHfR6CvOq6RcgJeHg/runs/0/artifacts/public/logs/live_backing.log with mime type "text/plain; charset=utf-8" and expiry 2020-04-01T01:57:01.389Z
[taskcluster:error] exit status 4294967295

Status: RESOLVED → REOPENED
Resolution: INCOMPLETE → ---
Flags: needinfo?(sledru)
Summary: Intermittent 'mach static-analysis autotest --intree-tool' did not run successfully. Please check log for errors. → Perma 'mach static-analysis autotest --intree-tool' did not run successfully. Please check log for errors.
Assignee: nobody → bpostelnicu

I'm gonna take care of this, so cancelling the need-info for Sylvestre.

Flags: needinfo?(sledru)

I've ran static-analysis-autotest on linux64/debug and the task finished without issues. How did you run your task?

Flags: needinfo?(nbeleuzu)

This makes sense since I think on autoland not all of the patches were in, in the same time. Closing this issue since this just a coincidence on how we've pushed this patches.

Status: REOPENED → RESOLVED
Closed: 11 months ago10 months ago
Flags: needinfo?(bpostelnicu)
Resolution: --- → INVALID

:andi , it's still permafailing and the jobs are getting automatically scheduled

Status: RESOLVED → REOPENED
Flags: needinfo?(bpostelnicu)
Resolution: INVALID → ---
Severity: normal → major
Priority: P5 → P1
Regressed by: 1536790

There are 533 total failures in the last 7 days on linux64 debug and windows2012-64 debug

Recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=238020481&repo=autoland&lineNumber=3170

uppressed 2 warnings (2 with check filters).
02:44:14 WARNING - setting return code to 2
02:44:14 FATAL - 'mach static-analysis autotest --intree-tool' did not run successfully. Please check log for errors.
02:44:14 FATAL - Running post_fatal callback...
02:44:14 FATAL - Exiting -1
02:44:14 INFO - [mozharness: 2019-04-04 02:44:14.903000Z] Finished static-analysis-autotest step (failed)
02:44:14 INFO - Running post-run listener: _parse_build_tests_ccov
02:44:14 INFO - Running post-run listener: _shutdown_sccache
02:44:14 INFO - Running command: ['z:\build\build\src\sccache2\sccache.exe', '--stop-server'] in z:\build\build\src
02:44:14 INFO - Copy/paste: z:\build\build\src\sccache2\sccache.exe --stop-server
02:44:14 INFO - Stopping sccache server...
02:44:14 INFO - Compile requests 99
02:44:14 INFO - Compile requests executed 63
02:44:14 INFO - Cache hits 12
02:44:14 INFO - Cache misses 22
02:44:14 INFO - Cache timeouts 0
02:44:14 INFO - Cache read errors 0
02:44:14 INFO - Forced recaches 0
02:44:14 INFO - Cache write errors 0
02:44:14 INFO - Compilation failures 0
02:44:14 INFO - Cache errors 29
02:44:14 INFO - Non-cacheable compilations 0
02:44:14 INFO - Non-cacheable calls 11
02:44:14 INFO - Non-compilation calls 25
02:44:14 INFO - Unsupported compiler calls 0
02:44:14 INFO - Successful distributed compilations 0
02:44:14 INFO - Failed distributed compilations 0
02:44:14 INFO - Average cache write 0.046 s
02:44:14 INFO - Average cache read miss 0.036 s
02:44:14 INFO - Average cache read hit 0.036 s
02:44:14 INFO - Non-cacheable reasons:
02:44:14 INFO - multiple input files 7
02:44:14 INFO - unknown source language 4
02:44:14 INFO - Cache location S3, bucket: Bucket(name=taskcluster-level-3-sccache-us-east-1, base_url=http://taskcluster-level-3-sccache-us-east-1.s3.amazonaws.com/)
02:44:14 INFO - Return code: 0
02:44:14 INFO - Running post-run listener: _summarize
02:44:14 ERROR - # TBPL FAILURE #
02:44:14 INFO - [mozharness: 2019-04-04 02:44:14.913000Z] FxDesktopBuild summary:
02:44:14 ERROR - # TBPL FAILURE #

Whiteboard: [stockwell disable-recommended] → [stockwell needswork:owner]

There are two issues here:

  • A regression introduced by Bug 1540769.
  • A build failure only on Windows builds.
Flags: needinfo?(bpostelnicu)
Pushed by bpostelnicu@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/a0adda35967e
Fix Perma 'mach static-analysis autotest; caused by Bug 1540769. r=sylvestre
Regressed by: 1540769
No longer regressed by: 1536790
Whiteboard: [stockwell disable-recommended] → [stockwell needswork:owner]
Whiteboard: [stockwell disable-recommended] → [stockwell needswork:owner]
Pushed by archaeopteryx@coole-files.de:
https://hg.mozilla.org/integration/autoland/rev/13b40bbdb45c
Temporarily disable permafailing Windows static analysis builds. a=permafail
Pushed by archaeopteryx@coole-files.de:
https://hg.mozilla.org/integration/autoland/rev/c9d95a544902
Temporarily disable permafailing Windows static analysis builds: Add one more whitespace because the linting gods demand this. a=permafail CLOSED TREE
Whiteboard: [stockwell disable-recommended]
Flags: needinfo?(bpostelnicu)
Whiteboard: [stockwell disable-recommended]
Whiteboard: [stockwell disable-recommended] → [stockwell fixed]
Pushed by sledru@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/2351c4f1d47d
re-enable static-analysis autotest on win64 platform after migrating to clang-tidy 8. r=sylvestre
Keywords: leave-open
Status: REOPENED → RESOLVED
Closed: 10 months ago8 months ago
Resolution: --- → FIXED
See Also: → 1554187
You need to log in before you can comment on or make changes to this bug.