1555255 - Make `mach android archive-geckoview` uses docs package rather than resolving d.android.com online

Status: RESOLVED FIXED

(Firefox Build System :: Android Studio and Gradle Integration, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: csabou [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=248891830&repo=autoland
Full log: https://queue.taskcluster.net/v1/task/UwLLADzXRIS9P1JW6GCTJw/runs/0/artifacts/public/logs/live_backing.log

---

[task 2019-05-29T06:59:27.661Z] 06:59:27 INFO - > Task :geckoview:javadocWithGeckoBinariesDebug
[task 2019-05-29T06:59:27.661Z] 06:59:27 INFO - Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=utf-8
[task 2019-05-29T06:59:35.261Z] 06:59:35 INFO - javadoc: warning - Error fetching URL: https://d.android.com/reference/
[task 2019-05-29T06:59:35.461Z] 06:59:35 INFO - /builds/worker/workspace/build/src/mobile/android/geckoview/src/main/java/org/mozilla/geckoview/GeckoRuntimeSettings.java:301: warning - Tag @link: reference not found: android.app.job.JobScheduler
[task 2019-05-29T06:59:36.061Z] 06:59:36 INFO - 2 warnings
[task 2019-05-29T06:59:36.061Z] 06:59:36 INFO - > Task :geckoview:javadocWithGeckoBinariesDebug FAILED
[task 2019-05-29T06:59:36.061Z] 06:59:36 INFO - FAILURE: Build failed with an exception.
[task 2019-05-29T06:59:36.061Z] 06:59:36 INFO - * Where:
[task 2019-05-29T06:59:36.061Z] 06:59:36 INFO - Build file '/builds/worker/workspace/build/src/mobile/android/geckoview/build.gradle' line: 213
[task 2019-05-29T06:59:36.061Z] 06:59:36 INFO - * What went wrong:
[task 2019-05-29T06:59:36.061Z] 06:59:36 INFO - Execution failed for task ':geckoview:javadocWithGeckoBinariesDebug'.
[task 2019-05-29T06:59:36.061Z] 06:59:36 INFO - > Treating 2 javadoc warning(s) as error(s)
[task 2019-05-29T06:59:36.061Z] 06:59:36 INFO - * Try:
[task 2019-05-29T06:59:36.061Z] 06:59:36 INFO - Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
[task 2019-05-29T06:59:36.061Z] 06:59:36 INFO - * Get more help at https://help.gradle.org
[task 2019-05-29T06:59:36.061Z] 06:59:36 INFO - Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
[task 2019-05-29T06:59:36.061Z] 06:59:36 INFO - Use '--warning-mode all' to show the individual deprecation warnings.
[task 2019-05-29T06:59:36.061Z] 06:59:36 INFO - See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings
[task 2019-05-29T06:59:36.062Z] 06:59:36 INFO - BUILD FAILED in 58s
[task 2019-05-29T06:59:36.062Z] 06:59:36 INFO - 110 actionable tasks: 93 executed, 17 up-to-date
[task 2019-05-29T06:59:36.515Z] 06:59:36 ERROR - Return code: 1
[task 2019-05-29T06:59:36.515Z] 06:59:36 ERROR - 1 not in success codes: [0]
[task 2019-05-29T06:59:36.515Z] 06:59:36 WARNING - setting return code to 2
[task 2019-05-29T06:59:36.515Z] 06:59:36 FATAL - Halting on failure while running ['/builds/worker/workspace/build/src/obj-x86_64-pc-linux-gnu/_virtualenvs/init/bin/python', 'mach', '--log-no-times', 'android', 'archive-geckoview']
[task 2019-05-29T06:59:36.515Z] 06:59:36 FATAL - Running post_fatal callback...
[task 2019-05-29T06:59:36.516Z] 06:59:36 FATAL - Exiting 2
[task 2019-05-29T06:59:36.516Z] 06:59:36 INFO - [mozharness: 2019-05-29 06:59:36.516035Z] Finished build step (failed)
[task 2019-05-29T06:59:36.516Z] 06:59:36 INFO - Running post-run listener: _parse_build_tests_ccov
[task 2019-05-29T06:59:36.516Z] 06:59:36 INFO - Running post-run listener: _shutdown_sccache
[task 2019-05-29T06:59:36.516Z] 06:59:36 INFO - Running command: ['/builds/worker/workspace/build/src/sccache/sccache', '--stop-server'] in /builds/worker/workspace/build/src
[task 2019-05-29T06:59:36.516Z] 06:59:36 INFO - Copy/paste: /builds/worker/workspace/build/src/sccache/sccache --stop-server
[task 2019-05-29T06:59:36.519Z] 06:59:36 INFO - Stopping sccache server...
[task 2019-05-29T06:59:36.519Z] 06:59:36 INFO - Compile requests 3820
[task 2019-05-29T06:59:36.519Z] 06:59:36 INFO - Compile requests executed 3524
[task 2019-05-29T06:59:36.519Z] 06:59:36 INFO - Cache hits 441
[task 2019-05-29T06:59:36.519Z] 06:59:36 INFO - Cache hits (C/C++) 99
[task 2019-05-29T06:59:36.519Z] 06:59:36 INFO - Cache hits (Rust) 342
[task 2019-05-29T06:59:36.519Z] 06:59:36 INFO - Cache misses 3083
[task 2019-05-29T06:59:36.519Z] 06:59:36 INFO - Cache misses (C/C++) 3067
[task 2019-05-29T06:59:36.520Z] 06:59:36 INFO - Cache misses (Rust) 16
[task 2019-05-29T06:59:36.520Z] 06:59:36 INFO - Cache timeouts 0
[task 2019-05-29T06:59:36.520Z] 06:59:36 INFO - Cache read errors 0
[task 2019-05-29T06:59:36.520Z] 06:59:36 INFO - Forced recaches 0
[task 2019-05-29T06:59:36.520Z] 06:59:36 INFO - Cache write errors 0
[task 2019-05-29T06:59:36.520Z] 06:59:36 INFO - Compilation failures 0
[task 2019-05-29T06:59:36.520Z] 06:59:36 INFO - Cache errors 0
[task 2019-05-29T06:59:36.520Z] 06:59:36 INFO - Non-cacheable compilations 0
[task 2019-05-29T06:59:36.520Z] 06:59:36 INFO - Non-cacheable calls 125
[task 2019-05-29T06:59:36.520Z] 06:59:36 INFO - Non-compilation calls 171
[task 2019-05-29T06:59:36.520Z] 06:59:36 INFO - Unsupported compiler calls 0
[task 2019-05-29T06:59:36.520Z] 06:59:36 INFO - Successful distributed compilations 0
[task 2019-05-29T06:59:36.520Z] 06:59:36 INFO - Failed distributed compilations 0
[task 2019-05-29T06:59:36.520Z] 06:59:36 INFO - Average cache write 0.086 s
[task 2019-05-29T06:59:36.520Z] 06:59:36 INFO - Average cache read miss 3.302 s
[task 2019-05-29T06:59:36.520Z] 06:59:36 INFO - Average cache read hit 0.094 s
[task 2019-05-29T06:59:36.520Z] 06:59:36 INFO - Non-cacheable reasons:
[task 2019-05-29T06:59:36.520Z] 06:59:36 INFO - crate-type 66
[task 2019-05-29T06:59:36.521Z] 06:59:36 INFO - -fsyntax-only 37
[task 2019-05-29T06:59:36.521Z] 06:59:36 INFO - - 10
[task 2019-05-29T06:59:36.521Z] 06:59:36 INFO - unknown source language 10
[task 2019-05-29T06:59:36.521Z] 06:59:36 INFO - -E 2
[task 2019-05-29T06:59:36.521Z] 06:59:36 INFO - Cache location S3, bucket: Bucket(name=taskcluster-level-3-sccache-us-west-2, base_url=http://taskcluster-level-3-sccache-us-west-2.s3.amazonaws.com/)
[task 2019-05-29T06:59:36.521Z] 06:59:36 INFO - Return code: 0
[task 2019-05-29T06:59:36.521Z] 06:59:36 INFO - Running post-run listener: _summarize
[task 2019-05-29T06:59:36.521Z] 06:59:36 ERROR - # TBPL FAILURE #
[task 2019-05-29T06:59:36.521Z] 06:59:36 INFO - [mozharness: 2019-05-29 06:59:36.521497Z] FxDesktopBuild summary:
[task 2019-05-29T06:59:36.521Z] 06:59:36 ERROR - # TBPL FAILURE #
[taskcluster 2019-05-29 06:59:37.228Z] === Task Finished ===

Comment 1

[Intermittent Failures Robot]

1 failures in 4337 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform breakdown:

• android-5-0-x86_64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1555255&startday=2019-05-27&endday=2019-06-02&tree=all

Comment 2

[Intermittent Failures Robot]

2 failures in 4753 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1
• mozilla-inbound: 1

Platform breakdown:

• android-5-0-x86_64: 1
• android-4-2-x86: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1555255&startday=2019-06-10&endday=2019-06-16&tree=all

Comment 3

[Intermittent Failures Robot]

24 failures in 2581 pushes (0.009 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 19
• mozilla-central: 1
• try: 4

Platform breakdown:

• android-5-0-x86_64: 6
• android-5-0-aarch64: 7
• android-4-0-geckoview-fat-aar: 2
• android-4-0-armv7-api16: 6
• android-4-2-x86: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1555255&startday=2019-06-17&endday=2019-06-23&tree=all

Comment 4

[Andrew Erickson [:aerickson]]

I think these failures are just due to https://d.android.com/reference/ not returning, resulting in an undefined @link document element.

[task 2019-05-29T06:59:35.261Z] 06:59:35 INFO - javadoc: warning - Error fetching URL: https://d.android.com/reference/
[task 2019-05-29T06:59:35.461Z] 06:59:35 INFO - /builds/worker/workspace/build/src/mobile/android/geckoview/src/main/java/org/mozilla/geckoview/GeckoRuntimeSettings.java:301: warning - Tag @link: reference not found: android.app.job.JobScheduler

Not sure if we can do anything but retry in this case.

:nalexander, can timeouts be tweaked for this task (javadocWithGeckoBinariesDebug in mobile/android/geckoview/build.gradle)?

Comment 5

[Intermittent Failures Robot]

44 failures in 625 pushes (0.07 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 30
• mozilla-inbound: 5
• mozilla-central: 9

Platform breakdown:

• android-5-0-x86_64: 9
• android-5-0-aarch64: 9
• android-4-0-geckoview-fat-aar: 3
• android-4-0-armv7-api16: 15
• android-4-2-x86: 8

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1555255&startday=2019-06-24&endday=2019-06-24&tree=all

Comment 6

[Nick Alexander :nalexander [he/him]]

Attachment: Bug 1555255 - Don't require a network connection to produce Javadoc. r?#geckoview-reviewers

See https://stackoverflow.com/a/24026735.

Adding the docs package requirement is not ideal, but it's not worth
the effort to install it only in automation (or in the relevant task),
and it's not that large: 1.0G on my macOS installation.

Comment 7

[Nick Alexander :nalexander [he/him]]

Try build, where red is actually green: https://treeherder.mozilla.org/#/jobs?repo=try&revision=29edabede5bb4e2a730f28dbeaa34b128b26e125&selectedJob=253394151.

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/44a166a4efdb
Don't require a network connection to produce Javadoc. r=geckoview-reviewers,agi

Comment 9

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/44a166a4efdb

Comment 10

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/fa342b3889f3

Comment 11

[Intermittent Failures Robot]

46 failures in 4409 pushes (0.01 failures/push) were associated with this bug in the last 7 days.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 31
• mozilla-inbound: 5
• mozilla-central: 9
• try: 1

Platform breakdown:

• android-5-0-x86_64: 10
• android-4-0-geckoview-fat-aar: 3
• android-5-0-aarch64: 9
• android-4-0-armv7-api16: 16
• android-4-2-x86: 8

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1555255&startday=2019-06-24&endday=2019-06-30&tree=all

Comment 12

[Nick Alexander :nalexander [he/him]]

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1555255&startday=2019-06-24&endday=2019-06-30&tree=all

This graph looks great: failures drop to 0 on the 25th, when the patch landed, suggesting this is an appropriate fix.

Comment 13

[Natalia Csoregi [:nataliaCs]]

Backed out from integration as it has caused android-sdk-linux bustage because the docs package doesn't exist anymore

backout: https://hg.mozilla.org/integration/autoland/rev/d11588e43453f9cb0c4db74742a6e3cbad299682

push where the failures were seen: https://treeherder.mozilla.org/#/jobs?repo=autoland&group_state=expanded&revision=9d6acea5d4321927ef8cda0b9be085d85919721c&selectedTaskRun=ZlwkYoWzR9usHYWnfy9bNw-1&searchStr=Toolchains%2Copt%2CToolchain%2Cbuilds%2Cfor%2CLinux%2C64-bits%2Ctoolchain-linux64-android-sdk-linux-repack%2CTL%28android-sdk-linux%29

Comment 14

[Mike Hommey [:glandium]]

Here's a log with bug 1635664 fixed, that shows the full sdkmanager log: https://firefoxci.taskcluster-artifacts.net/KLq3c0Z3QdyTbyL7cgQvUg/0/public/logs/live_backing.log