1641711 - Sometimes lints silently fail

Status: RESOLVED FIXED

(GeckoView :: General, defect)

Description

[[ex-Mozilla] Agi Sferro | :agi]

When javadoc fails the build all lints silently fail, this is because |mach lint| ignores when a lint fails before returning output when run with -f treeherder.

Comment 1

[[ex-Mozilla] Agi Sferro | :agi]

Attachment: Bug 1641711 - Don't fail the build if javadoc fails.

|mach lint -f treeherder| expects lint to not fail ever, if a lint fails it's
just ignored. This means that when our javadoc errors out and fails the build
all lints are ignored.

To avoid that we add failOnError = false to the javadoc job so that we can
finish the build and actually report the errors.

This patch also adds gradle output |mach lint| to make it easier to
diagnose errors in the lint.

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2548f39e1751
Don't fail the build if javadoc fails. r=nalexander,snorp

Comment 3

[Bogdan Tara[:bogdan_tara | bogdant]]

Backed out changeset 2548f39e1751 (bug 1641711) for lints.py failure

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&fromchange=2548f39e1751f10df77ccae8c42702d6ef9bbd7d&searchStr=lint&tochange=c9e3fba165b23344e3f0ab121baf649026a05d62&selectedTaskRun=NQEp-MthT0CoM_GdmyJI1g-0

Backout link: https://hg.mozilla.org/integration/autoland/rev/c9e3fba165b23344e3f0ab121baf649026a05d62

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=304234482&repo=autoland&lineNumber=1205

[task 2020-05-29T09:33:53.074Z]   File "/builds/worker/checkouts/gecko/tools/lint/android/lints.py", line 157, in lint
[task 2020-05-29T09:33:53.075Z]     tree = ET.parse(open(path, 'rt'))
[task 2020-05-29T09:33:53.075Z] FileNotFoundError: [Errno 2] No such file or directory: '/builds/worker/checkouts/gecko/obj-firefox/gradle/build/mobile/android/geckoview/reports/lint-results-withGeckoBinariesDebug.xml'
[task 2020-05-29T09:33:53.076Z] Traceback (most recent call last):
[task 2020-05-29T09:33:53.076Z]   File "/builds/worker/checkouts/gecko/python/mozlint/mozlint/roller.py", line 60, in _run_worker
[task 2020-05-29T09:33:53.076Z]     res = func(paths, config, **lintargs) or []
[task 2020-05-29T09:33:53.076Z]   File "/builds/worker/checkouts/gecko/python/mozlint/mozlint/types.py", line 54, in __call__
[task 2020-05-29T09:33:53.076Z]     return self._lint(paths, config, **lintargs)
[task 2020-05-29T09:33:53.076Z]   File "/builds/worker/checkouts/gecko/python/mozlint/mozlint/types.py", line 159, in _lint
[task 2020-05-29T09:33:53.076Z]     return func(config, **lintargs)
[task 2020-05-29T09:33:53.076Z]   File "/builds/worker/checkouts/gecko/tools/lint/android/lints.py", line 87, in api_lint
[task 2020-05-29T09:33:53.076Z]     with open(os.path.join(topobjdir, folder, 'apilint-result.json')) as f:
[task 2020-05-29T09:33:53.076Z] FileNotFoundError: [Errno 2] No such file or directory: '/builds/worker/checkouts/gecko/obj-firefox/gradle/build/mobile/android/geckoview/intermediates/javac/withGeckoBinariesDebug/compileWithGeckoBinariesDebugJavaWithJavac/classes/apilint-result.json'
[task 2020-05-29T09:33:53.076Z] Traceback (most recent call last):
[task 2020-05-29T09:33:53.076Z]   File "/builds/worker/checkouts/gecko/python/mozlint/mozlint/roller.py", line 60, in _run_worker
[task 2020-05-29T09:33:53.076Z]     res = func(paths, config, **lintargs) or []
[task 2020-05-29T09:33:53.076Z]   File "/builds/worker/checkouts/gecko/python/mozlint/mozlint/types.py", line 54, in __call__
[task 2020-05-29T09:33:53.076Z]     return self._lint(paths, config, **lintargs)
[task 2020-05-29T09:33:53.076Z]   File "/builds/worker/checkouts/gecko/python/mozlint/mozlint/types.py", line 159, in _lint
[task 2020-05-29T09:33:53.077Z]     return func(config, **lintargs)
[task 2020-05-29T09:33:53.077Z]   File "/builds/worker/checkouts/gecko/tools/lint/android/lints.py", line 289, in test
[task 2020-05-29T09:33:53.077Z]     report_dir=report_dir))
[task 2020-05-29T09:33:53.077Z]   File "/builds/worker/checkouts/gecko/tools/lint/android/lints.py", line 223, in _parse_android_test_results
[task 2020-05-29T09:33:53.077Z]     raise RuntimeError('No reports found under {}'.format(report_dir))
[task 2020-05-29T09:33:53.077Z] RuntimeError: No reports found under /builds/worker/checkouts/gecko/obj-firefox/gradle/build/mobile/android/geckoview/test-results/testWithGeckoBinariesDebugUnitTest
[task 2020-05-29T09:33:53.077Z] Traceback (most recent call last):
[task 2020-05-29T09:33:53.077Z]   File "/builds/worker/checkouts/gecko/python/mozlint/mozlint/roller.py", line 60, in _run_worker
[task 2020-05-29T09:33:53.077Z]     res = func(paths, config, **lintargs) or []
[task 2020-05-29T09:33:53.077Z]   File "/builds/worker/checkouts/gecko/python/mozlint/mozlint/types.py", line 54, in __call__
[task 2020-05-29T09:33:53.077Z]     return self._lint(paths, config, **lintargs)
[task 2020-05-29T09:33:53.077Z]   File "/builds/worker/checkouts/gecko/python/mozlint/mozlint/types.py", line 159, in _lint
[task 2020-05-29T09:33:53.077Z]     return func(config, **lintargs)
[task 2020-05-29T09:33:53.077Z]   File "/builds/worker/checkouts/gecko/tools/lint/android/lints.py", line 210, in checkstyle
[task 2020-05-29T09:33:53.077Z]     report_path=report_path))
[task 2020-05-29T09:33:53.077Z]   File "/builds/worker/checkouts/gecko/tools/lint/android/lints.py", line 177, in _parse_checkstyle_output
[task 2020-05-29T09:33:53.077Z]     tree = ET.parse(open(report_path, 'rt'))
[task 2020-05-29T09:33:53.077Z] FileNotFoundError: [Errno 2] No such file or directory: '/builds/worker/checkouts/gecko/obj-firefox/gradle/build/mobile/android/geckoview/reports/checkstyle/withGeckoBinariesDebug.xml'
[task 2020-05-29T09:33:53.078Z] No lint issues found.
[fetches 2020-05-29T09:33:53.111Z] removing /builds/worker/fetches
[fetches 2020-05-29T09:33:54.208Z] finished
[taskcluster 2020-05-29 09:33:54.575Z] === Task Finished ===
[taskcluster 2020-05-29 09:33:54.694Z] Artifact "public/app/unittest" not found at "/builds/worker/checkouts/gecko/obj-firefox/gradle/build/mobile/android/app/reports/tests"
[taskcluster 2020-05-29 09:33:54.893Z] Artifact "public/android/geckoview/apilint-result.json" not found at "/builds/worker/checkouts/gecko/obj-firefox/gradle/build/mobile/android/geckoview/intermediates/classes/withGeckoBinaries/debug/apilint-result.json"
[taskcluster 2020-05-29 09:33:54.985Z] Artifact "public/android/geckoview/api.txt" not found at "/builds/worker/checkouts/gecko/obj-firefox/gradle/build/mobile/android/geckoview/intermediates/classes/withGeckoBinaries/debug/api.txt"
[taskcluster 2020-05-29 09:33:55.081Z] Artifact "public/android/checkstyle/checkstyle.xml" not found at "/builds/worker/checkouts/gecko/obj-firefox/gradle/build/mobile/android/app/reports/checkstyle/checkstyle.xml"
[taskcluster 2020-05-29 09:33:55.189Z] Artifact "public/android/lint/lint-results-officialWithoutGeckoBinariesDebug_files" not found at "/builds/worker/checkouts/gecko/obj-firefox/gradle/build/mobile/android/app/reports/lint-results-officialWithoutGeckoBinariesDebug_files"
[taskcluster 2020-05-29 09:33:55.365Z] Artifact "public/android/checkstyle/checkstyle.html" not found at "/builds/worker/checkouts/gecko/obj-firefox/gradle/build/mobile/android/app/reports/checkstyle/checkstyle.html"
[taskcluster 2020-05-29 09:33:55.465Z] Artifact "public/android/lint/lint-results-officialWithoutGeckoBinariesDebug.html" not found at "/builds/worker/checkouts/gecko/obj-firefox/gradle/build/mobile/android/app/reports/lint-results-officialWithoutGeckoBinariesDebug.html"
[taskcluster 2020-05-29 09:33:55.557Z] Artifact "public/geckoview/unittest" not found at "/builds/worker/checkouts/gecko/obj-firefox/gradle/build/mobile/android/geckoview/reports/tests"
[taskcluster 2020-05-29 09:33:55.649Z] Artifact "public/android/lint/lint-results-officialWithoutGeckoBinariesDebug.xml" not found at "/builds/worker/checkouts/gecko/obj-firefox/gradle/build/mobile/android/app/reports/lint-results-officialWithoutGeckoBinariesDebug.xml"
[taskcluster 2020-05-29 09:33:55.813Z] Unsuccessful task run with exit code: 1 completed in 273.859 seconds

https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=304234314&repo=autoland&lineNumber=102

[task 2020-05-29T09:26:04.049Z] 09:26:04.49 flake8 (94) | flake8 version=3.7.5
[task 2020-05-29T09:26:04.050Z] 09:26:04.49 flake8 (92) | Passing the following paths:
[task 2020-05-29T09:26:04.050Z] /builds/worker/checkouts/gecko/docshell  
[task 2020-05-29T09:26:04.050Z] /builds/worker/checkouts/gecko/devtools  
[task 2020-05-29T09:26:04.050Z] /builds/worker/checkouts/gecko/uriloader  
[task 2020-05-29T09:26:04.050Z] /builds/worker/checkouts/gecko/moz.configure  
[task 2020-05-29T09:26:04.050Z] /builds/worker/checkouts/gecko/testing  
[task 2020-05-29T09:26:04.050Z] /builds/worker/checkouts/gecko/dom  
[task 2020-05-29T09:26:04.050Z] /builds/worker/checkouts/gecko/mfbt  
[task 2020-05-29T09:26:04.050Z] /builds/worker/checkouts/gecko/hal  
[task 2020-05-29T09:26:04.050Z] /builds/worker/checkouts/gecko/gradle  
[task 2020-05-29T09:26:04.050Z] /builds/worker/checkouts/gecko/servo  
[task 2020-05-29T09:26:04.050Z] /builds/worker/checkouts/gecko/layout  
[task 2020-05-29T09:26:04.050Z] /builds/worker/checkouts/gecko/image  
[task 2020-05-29T09:26:04.050Z] /builds/worker/checkouts/gecko/configure.py
[task 2020-05-29T09:26:04.051Z] 09:26:04.51 flake8 (94) | Command: --config /builds/worker/checkouts/gecko/.flake8 --output-file /tmp/tmpq5r4tf_z --format {"path":"%(path)s","lineno":%(row)s,"column":%(col)s,"rule":"%(code)s","message":"%(text)s"} --filename *.configure,*.py
[task 2020-05-29T09:26:04.084Z] 09:26:04.84 flake8 (92) | flake8 version=3.7.5
[task 2020-05-29T09:26:04.086Z] 09:26:04.86 flake8 (92) | Command: --config /builds/worker/checkouts/gecko/.flake8 --output-file /tmp/tmpvg2o6lnf --format {"path":"%(path)s","lineno":%(row)s,"column":%(col)s,"rule":"%(code)s","message":"%(text)s"} --filename *.configure,*.py
[task 2020-05-29T09:26:13.446Z] 09:26:13.446 flake8 (93) | Finished in 9.58 seconds
[task 2020-05-29T09:26:40.316Z] 09:26:40.316 flake8 (95) | Finished in 36.45 seconds
[task 2020-05-29T09:27:05.216Z] 09:27:05.216 flake8 (94) | Finished in 61.35 seconds
[task 2020-05-29T09:27:06.320Z] 09:27:06.320 flake8 (92) | Finished in 62.46 seconds
[task 2020-05-29T09:27:06.326Z] TEST-UNEXPECTED-ERROR | /builds/worker/checkouts/gecko/tools/lint/android/lints.py:46:39 | local variable 'devnull' is assigned to but never used (F841)
[taskcluster 2020-05-29 09:27:06.624Z] === Task Finished ===
[taskcluster 2020-05-29 09:27:07.236Z] Unsuccessful task run with exit code: 1 completed in 75.443 seconds

Comment 4

[[ex-Mozilla] Agi Sferro | :agi]

Attachment: Bug 1641711 - Update artifacts for mozlint-android.

Comment 5

[[ex-Mozilla] Agi Sferro | :agi]

Attachment: Bug 1641711 - Make javadoc output parsing more resilient.

As it turns out, javadoc output is not stable across platforms / OSs / versions
(not a big suprise).

E.g. in try this is an example line:

/builds/worker/checkouts/gecko/mobile/android/geckoview/src/main/java/org/mozilla/geckoview/GeckoRuntimeSettings.java:1124: warning - Tag @link: reference not found: Autocomplete.LoginStorageDelegate.onLoginFetch onLoginFetch

on my mac, this is the same error:

/Users/asferro/workspace/mozilla-central/mobile/android/geckoview/src/main/java/org/mozilla/geckoview/GeckoRuntimeSettings.java:1118: error: reference not found
     * {@link Autocomplete.LoginStorageDelegate.onLoginFetch onLoginFetch}.

Notice the - character after warning in the first example and the :
character after the second example.

To be able to parse both this patch simiplifies the regex and also adds a
failsafe that will mark anything that contains the string "error" and "warning"
as such.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f7b9cdfbc377
Don't fail the build if javadoc fails. r=nalexander,snorp,esawin
https://hg.mozilla.org/integration/autoland/rev/4618c9093d0e
Update artifacts for mozlint-android. r=nalexander
https://hg.mozilla.org/integration/autoland/rev/eb654059cd8b
Make javadoc output parsing more resilient. r=snorp

Comment 7

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/f7b9cdfbc377
https://hg.mozilla.org/mozilla-central/rev/4618c9093d0e
https://hg.mozilla.org/mozilla-central/rev/eb654059cd8b