1452589 - Intermittent subprocess.CalledProcessError: Command '[u'/builds/worker/workspace/build/src/mach', 'android', 'assemble-app']' returned non-zero exit status 1

Status: REOPENED

(Firefox Build System :: Android Studio and Gradle Integration, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: ccoroiu [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=172643128&repo=mozilla-central

https://queue.taskcluster.net/v1/task/G6-919R5QeyA2nNkOsM6kg/runs/0/artifacts/public/logs/live_backing.log

After rerun this job, it failed again. 

task 2018-04-09T10:46:12.905Z] 10:46:12  WARNING -  /builds/worker/workspace/build/src/mobile/android/thirdparty/ch/boye/httpclientandroidlib/conn/HttpHostConnectException.java:56: warning: non-varargs call of varargs method with inexact argument type for last parameter;
[task 2018-04-09T10:46:12.905Z] 10:46:12     INFO -          this(cause, host, null);
[task 2018-04-09T10:46:12.905Z] 10:46:12     INFO -                            ^
[task 2018-04-09T10:46:12.905Z] 10:46:12     INFO -    cast to InetAddress for a varargs call
[task 2018-04-09T10:46:12.905Z] 10:46:12     INFO -    cast to InetAddress[] for a non-varargs call and to suppress this warning
[task 2018-04-09T10:46:12.905Z] 10:46:12     INFO -  :app:processOfficialWithoutGeckoBinariesNoMinApiPhotonDebugAndroidTestJavaRes NO-SOURCE
[task 2018-04-09T10:46:12.913Z] 10:46:12     INFO -  :app:mergeOfficialWithoutGeckoBinariesNoMinApiPhotonDebugAndroidTestJniLibFolders
[task 2018-04-09T10:46:12.914Z] 10:46:12     INFO -  :app:validateSigningOfficialWithoutGeckoBinariesNoMinApiPhotonDebugAndroidTest
[task 2018-04-09T10:46:14.123Z] 10:46:14     INFO -  Note: Some input files use or override a deprecated API.
[task 2018-04-09T10:46:14.123Z] 10:46:14     INFO -  Note: Recompile with -Xlint:deprecation for details.
[task 2018-04-09T10:46:14.124Z] 10:46:14     INFO -  Note: Some input files use unchecked or unsafe operations.
[task 2018-04-09T10:46:14.124Z] 10:46:14     INFO -  Note: Recompile with -Xlint:unchecked for details.
[task 2018-04-09T10:46:14.124Z] 10:46:14     INFO -  1 warning
[task 2018-04-09T10:46:14.164Z] 10:46:14     INFO -  Note: Some input files use or override a deprecated API.
[task 2018-04-09T10:46:14.164Z] 10:46:14     INFO -  Note: Recompile with -Xlint:deprecation for details.
[task 2018-04-09T10:46:14.164Z] 10:46:14     INFO -  Note: /builds/worker/workspace/build/src/mobile/android/geckoview/src/main/java/org/mozilla/gecko/gfx/PanZoomController.java uses unchecked or unsafe operations.
[task 2018-04-09T10:46:14.164Z] 10:46:14     INFO -  Note: Recompile with -Xlint:unchecked for details.
[task 2018-04-09T10:46:14.823Z] 10:46:14     INFO -  :geckoview:transformClassesAndResourcesWithPrepareIntermediateJarsForOfficialWithoutGeckoBinariesNoMinApiDebug
[task 2018-04-09T10:46:14.994Z] 10:46:14     INFO -  :thirdparty:transformClassesAndResourcesWithPrepareIntermediateJarsForDebug
[task 2018-04-09T10:46:14.995Z] 10:46:14     INFO -  :app:javaPreCompileOfficialWithoutGeckoBinariesNoMinApiPhotonDebug
[task 2018-04-09T10:46:14.995Z] 10:46:14     INFO -  :thirdparty:transformNativeLibsWithMergeJniLibsForDebug
[task 2018-04-09T10:46:15.005Z] 10:46:15     INFO -  :thirdparty:transformNativeLibsWithIntermediateJniLibsForDebug
[task 2018-04-09T10:46:15.258Z] 10:46:15     INFO -  :app:transformNativeLibsWithMergeJniLibsForOfficialWithoutGeckoBinariesNoMinApiPhotonDebug
[task 2018-04-09T10:46:15.280Z] 10:46:15     INFO -  :app:transformNativeLibsWithMergeJniLibsForOfficialWithoutGeckoBinariesNoMinApiPhotonDebugAndroidTest
[task 2018-04-09T10:46:15.290Z] 10:46:15     INFO -  FAILURE: Build failed with an exception.
[task 2018-04-09T10:46:15.290Z] 10:46:15     INFO -  * Where:
[task 2018-04-09T10:46:15.290Z] 10:46:15     INFO -  Build file '/builds/worker/workspace/build/src/mobile/android/app/build.gradle' line: 285
[task 2018-04-09T10:46:15.290Z] 10:46:15     INFO -  * What went wrong:
[task 2018-04-09T10:46:15.290Z] 10:46:15     INFO -  Execution failed for task ':app:syncPreprocessedResForOfficialWithoutGeckoBinariesNoMinApiPhotonDebug'.
[task 2018-04-09T10:46:15.291Z] 10:46:15     INFO -  > Could not copy file '/builds/worker/workspace/build/src/obj-firefox/mobile/android/base/res/values/strings.xml' to '/builds/worker/workspace/build/src/obj-firefox/gradle/build/mobile/android/app/moz.build/src/officialWithoutGeckoBinariesNoMinApiPhotonDebug/res/values/strings.xml'.
[task 2018-04-09T10:46:15.291Z] 10:46:15     INFO -  * Try:
[task 2018-04-09T10:46:15.291Z] 10:46:15     INFO -  Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
[task 2018-04-09T10:46:15.291Z] 10:46:15     INFO -  * Get more help at https://help.gradle.org
[task 2018-04-09T10:46:15.291Z] 10:46:15     INFO -  BUILD FAILED in 16s
[task 2018-04-09T10:46:15.291Z] 10:46:15     INFO -  77 actionable tasks: 77 executed
[task 2018-04-09T10:46:15.881Z] 10:46:15    ERROR -  Traceback (most recent call last):
[task 2018-04-09T10:46:15.881Z] 10:46:15     INFO -    File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
[task 2018-04-09T10:46:15.882Z] 10:46:15     INFO -      "__main__", fname, loader, pkg_name)
[task 2018-04-09T10:46:15.882Z] 10:46:15     INFO -    File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
[task 2018-04-09T10:46:15.882Z] 10:46:15     INFO -      exec code in run_globals
[task 2018-04-09T10:46:15.882Z] 10:46:15     INFO -    File "/builds/worker/workspace/build/src/python/mozbuild/mozbuild/action/file_generate.py", line 117, in <module>
[task 2018-04-09T10:46:15.882Z] 10:46:15     INFO -      sys.exit(main(sys.argv[1:]))
[task 2018-04-09T10:46:15.882Z] 10:46:15     INFO -    File "/builds/worker/workspace/build/src/python/mozbuild/mozbuild/action/file_generate.py", line 68, in main
[task 2018-04-09T10:46:15.883Z] 10:46:15     INFO -      ret = module.__dict__[method](output, *args.additional_arguments, **kwargs)
[task 2018-04-09T10:46:15.884Z] 10:46:15     INFO -    File "/builds/worker/workspace/build/src/mobile/android/gradle.py", line 39, in assemble_app
[task 2018-04-09T10:46:15.884Z] 10:46:15     INFO -      return android('assemble-app')
[task 2018-04-09T10:46:15.884Z] 10:46:15     INFO -    File "/builds/worker/workspace/build/src/mobile/android/gradle.py", line 31, in android
[task 2018-04-09T10:46:15.884Z] 10:46:15     INFO -      subprocess.check_call(cmd)
[task 2018-04-09T10:46:15.884Z] 10:46:15     INFO -    File "/usr/lib/python2.7/subprocess.py", line 186, in check_call
[task 2018-04-09T10:46:15.884Z] 10:46:15     INFO -      raise CalledProcessError(retcode, cmd)
[task 2018-04-09T10:46:15.884Z] 10:46:15     INFO -  subprocess.CalledProcessError: Command '[u'/builds/worker/workspace/build/src/mach', 'android', 'assemble-app']' returned non-zero exit status 1
[task 2018-04-09T10:46:15.889Z] 10:46:15     INFO -  backend.mk:40: recipe for target 'android_apks' failed
[task 2018-04-09T10:46:15.889Z] 10:46:15    ERROR -  make[3]: *** [android_apks] Error 1
[task 2018-04-09T10:46:15.890Z] 10:46:15     INFO -  make[3]: *** Deleting file 'android_apks'
[task 2018-04-09T10:46:15.890Z] 10:46:15     INFO -  make[3]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox/mobile/android/base'
[task 2018-04-09T10:46:15.890Z] 10:46:15     INFO -  /builds/worker/workspace/build/src/config/rules.mk:418: recipe for target 'default' failed
[task 2018-04-09T10:46:15.890Z] 10:46:15    ERROR -  make[2]: *** [default] Error 2
[task 2018-04-09T10:46:15.890Z] 10:46:15     INFO -  make[2]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox/mobile/android/base'
[task 2018-04-09T10:46:15.891Z] 10:46:15     INFO -  Makefile:53: recipe for target 'libs-ach' failed
[task 2018-04-09T10:46:15.891Z] 10:46:15    ERROR -  make[1]: *** [libs-ach] Error 2
[task 2018-04-09T10:46:15.891Z] 10:46:15     INFO -  make[1]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox/mobile/android/locales'
[task 2018-04-09T10:46:15.891Z] 10:46:15     INFO -  Makefile:81: recipe for target 'installers-ach' failed
[task 2018-04-09T10:46:15.891Z] 10:46:15     INFO -  make: *** [installers-ach] Error 2
[task 2018-04-09T10:46:15.891Z] 10:46:15    ERROR - Return code: 2
[task 2018-04-09T10:46:15.892Z] 10:46:15     INFO - Setting buildbot property ach_failure to ach failed in make installers-ach!
[task 2018-04-09T10:46:15.892Z] 10:46:15     INFO - mkdir: /builds/worker/properties
[task 2018-04-09T10:46:15.892Z] 10:46:15     INFO - Writing buildbot properties ['ach_failure'] to /builds/worker/properties/ach_failure
[task 2018-04-09T10:46:15.892Z] 10:46:15     INFO - Writing to file /builds/worker/properties/ach_failure
[task 2018-04-09T10:46:15.892Z] 10:46:15     INFO - Contents:
[task 2018-04-09T10:46:15.892Z] 10:46:15     INFO -  ach_failure:ach failed in make installers-ach!
[task 2018-04-09T10:46:15.892Z] 10:46:15    ERROR - ach failed in make installers-ach!
[task 2018-04-09T10:46:15.893Z] 10:46:15  WARNING - setting return code to 1

Comment 1

[Nick Alexander :nalexander [he/him]]

The issue is not clear from the logs, but this looks like a legitimate issue:

[task 2018-04-09T10:46:09.794Z] 10:46:09     INFO -  :app:syncPreprocessedResForOfficialWithoutGeckoBinariesNoMinApiPhotonDebug[Fatal Error] :1:24: The entity "brandShorterName" was referenced, but not declared.

I think that entity is browser-only, based on

https://searchfox.org/mozilla-central/search?q=brandShorterName&case=false&regexp=false&path=

Now, it's not clear if this is an error in the ach locale, or if we should support brandShorterName in mobile/android as well as in browser/.  For that, we need an l10n voice -- which I think should be Delphine in this context.

Comment 2

[Delphine Lebédel [:delphine - Use needinfo]]

Hi - seems like this was an error introduced by localizers. I've fixed it in Pontoon a couple minutes ago: https://pontoon.mozilla.org/ach/firefox-for-android/mobile/android/base/android_strings.dtd/?search=Ikom+%26brandShorterName%3B&string=72095

Comment 3

[Intermittent Failures Robot]

3 failures in 792 pushes (0.004 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-central: 2
* try: 1

Platform breakdown:
* android-4-0-armv7-api16: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1452589&startday=2018-04-09&endday=2018-04-15&tree=trunk

Comment 4

[Firefox Bug Husbandry Bot]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup

Comment 5

[Andrei Ciure[:aciure]]

This has failed again:

https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&group_state=expanded&filter-resultStatus=testfailed&filter-resultStatus=busted&filter-resultStatus=exception&filter-classifiedState=unclassified&selectedJob=183348878&filter-searchStr=Android+4.0+API16%2B+opt+build-android-api-16%2Fopt+%28B%29

https://treeherder.mozilla.org/logviewer.html#?job_id=183348878&repo=mozilla-inbound

Comment 6

[Intermittent Failures Robot]

1 failures in 408 pushes (0.002 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-inbound: 1

Platform breakdown:
* android-4-0-armv7-api16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1452589&startday=2018-06-11&endday=2018-06-17&tree=trunk

Comment 7

[Firefox Bug Husbandry Bot]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup

Comment 8

[Andreea Pavel [:apavel]]

[task 2018-07-24T20:50:10.277Z] 20:50:10     INFO -  BUILD FAILED in 3s
[task 2018-07-24T20:50:10.277Z] 20:50:10     INFO -  Traceback (most recent call last):
[task 2018-07-24T20:50:10.277Z] 20:50:10     INFO -    File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
[task 2018-07-24T20:50:10.277Z] 20:50:10     INFO -      "__main__", fname, loader, pkg_name)
[task 2018-07-24T20:50:10.277Z] 20:50:10     INFO -    File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
[task 2018-07-24T20:50:10.278Z] 20:50:10     INFO -      exec code in run_globals
[task 2018-07-24T20:50:10.278Z] 20:50:10     INFO -    File "/builds/worker/workspace/build/src/python/mozbuild/mozbuild/action/file_generate.py", line 110, in <module>
[task 2018-07-24T20:50:10.278Z] 20:50:10     INFO -      sys.exit(main(sys.argv[1:]))
[task 2018-07-24T20:50:10.278Z] 20:50:10     INFO -    File "/builds/worker/workspace/build/src/python/mozbuild/mozbuild/action/file_generate.py", line 70, in main
[task 2018-07-24T20:50:10.278Z] 20:50:10     INFO -      ret = module.__dict__[method](output, *args.additional_arguments, **kwargs)
[task 2018-07-24T20:50:10.278Z] 20:50:10     INFO -    File "/builds/worker/workspace/build/src/mobile/android/gradle.py", line 39, in assemble_app
[task 2018-07-24T20:50:10.278Z] 20:50:10     INFO -      return android('assemble-app')
[task 2018-07-24T20:50:10.278Z] 20:50:10     INFO -    File "/builds/worker/workspace/build/src/mobile/android/gradle.py", line 31, in android
[task 2018-07-24T20:50:10.279Z] 20:50:10     INFO -      subprocess.check_call(cmd)
[task 2018-07-24T20:50:10.279Z] 20:50:10     INFO -    File "/usr/lib/python2.7/subprocess.py", line 186, in check_call
[task 2018-07-24T20:50:10.279Z] 20:50:10     INFO -      raise CalledProcessError(retcode, cmd)
[task 2018-07-24T20:50:10.279Z] 20:50:10     INFO -  subprocess.CalledProcessError: Command '[u'/builds/worker/workspace/build/src/mach', 'android', 'assemble-app']' returned non-zero exit status 1
[task 2018-07-24T20:50:10.279Z] 20:50:10     INFO -  backend.mk:54: recipe for target '.deps/android_apks.stub' failed
[task 2018-07-24T20:50:10.279Z] 20:50:10     INFO -  make[4]: *** [.deps/android_apks.stub] Error 1
[task 2018-07-24T20:50:10.279Z] 20:50:10     INFO -  make[4]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox/mobile/android/base'
[task 2018-07-24T20:50:10.279Z] 20:50:10     INFO -  /builds/worker/workspace/build/src/config/recurse.mk:101: recipe for target 'mobile/android/base/export' failed
[task 2018-07-24T20:50:10.280Z] 20:50:10     INFO -  make[3]: *** [mobile/android/base/export] Error 2
[task 2018-07-24T20:50:10.280Z] 20:50:10     INFO -  make[3]: *** Waiting for unfinished jobs....
[task 2018-07-24T20:50:10.661Z] 20:50:10     INFO -  make[4]: Entering directory '/builds/worker/workspace/build/src/obj-firefox/netwerk/dns'
[task 2018-07-24T20:50:10.661Z] 20:50:10     INFO -  etld_data.inc.stub
[task 2018-07-24T20:50:10.662Z] 20:50:10     INFO -  /builds/worker/workspace/build/src/obj-firefox/_virtualenvs/init/bin/python -m mozbuild.action.file_generate /builds/worker/workspace/build/src/netwerk/dns/prepare_tlds.py main etld_data.inc .deps/etld_data.inc.pp .deps/etld_data.inc.stub /builds/worker/workspace/build/src/netwerk/dns/effective_tld_names.dat
[task 2018-07-24T20:50:10.662Z] 20:50:10     INFO -  make[4]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox/netwerk/dns'
[task 2018-07-24T20:50:21.474Z] 20:50:21     INFO -  make[4]: Entering directory '/builds/worker/workspace/build/src/obj-firefox/security/manager/ssl'
[task 2018-07-24T20:50:21.474Z] 20:50:21     INFO -  nsSTSPreloadList.h.stub

Comment 9

[Nick Alexander :nalexander [he/him]]

I'm pretty confident that these are localization errors that are not well reported.  I'll open a ticket to try to make that better.

Comment 10

[Francesco Lodolo [:flod]]

It's unclear if comment 8 indicates a bug in one of our localizations. A pointer to the log of a failing build?

Comment 11

[Firefox Bug Husbandry Bot]

Re-triaging per https://bugzilla.mozilla.org/show_bug.cgi?id=1473195

Needinfo :susheel if you think this bug should be re-triaged.

Comment 12

[Intermittent Failures Robot]

1 failures in 760 pushes (0.001 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-inbound: 1

Platform breakdown:
* android-5-0-aarch64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1452589&startday=2018-07-23&endday=2018-07-29&tree=trunk

Comment 13

[Firefox Bug Husbandry Bot]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup

Comment 14

[Intermittent Failures Robot]

1 failures in 2672 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-central: 1

Platform breakdown:
* android-4-0-armv7-api16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1452589&startday=2018-08-13&endday=2018-08-19&tree=all

Comment 15

[Eliza Balazs [:ebalazs_]]

Recent failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=196409269&repo=mozilla-central&lineNumber=7658

Comment 16

[Intermittent Failures Robot]

2 failures in 2545 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-central: 2

Platform breakdown:
* android-4-0-armv7-api16: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1452589&startday=2018-08-27&endday=2018-09-02&tree=all

Comment 17

[Intermittent Failures Robot]

1 failures in 2684 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* autoland: 1

Platform breakdown:
* android-4-0-armv7-api16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1452589&startday=2018-09-10&endday=2018-09-16&tree=all

Comment 18

[Intermittent Failures Robot]

25 failures in 553 pushes (0.045 failures/push) were associated with this bug yesterday.

Repository breakdown:
* try: 9
* mozilla-central: 16

Platform breakdown:
* linux64: 2
* macosx64-qr: 1
* linux32: 1
* android-4-0-armv7-api16: 21

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1452589&startday=2018-10-04&endday=2018-10-04&tree=all

Comment 19

[Natalia Csoregi [:nataliaCs]]

Update: There have been 30 failures within the last 7 days:
- 29 on Android 4.0 API16+ opt
- 1 on Linux x64 debug 

Failure log example: https://treeherder.mozilla.org/logviewer.html#?job_id=203185596&repo=autoland&lineNumber=9814

[task 2018-10-03T19:07:24.884Z] 19:07:24     INFO -  :app:validateSigningOfficialWithoutGeckoBinariesNoMinApiPhotonDebugAndroidTest UP-TO-DATE
[task 2018-10-03T19:07:24.894Z] 19:07:24     INFO -  FAILURE: Build failed with an exception.
[task 2018-10-03T19:07:24.894Z] 19:07:24     INFO -  * Where:
[task 2018-10-03T19:07:24.895Z] 19:07:24     INFO -  Build file '/builds/worker/workspace/build/src/mobile/android/app/build.gradle' line: 295
[task 2018-10-03T19:07:24.895Z] 19:07:24     INFO -  * What went wrong:
[task 2018-10-03T19:07:24.895Z] 19:07:24     INFO -  Execution failed for task ':app:syncPreprocessedResForOfficialWithoutGeckoBinariesNoMinApiPhotonDebug'.
[task 2018-10-03T19:07:24.896Z] 19:07:24     INFO -  > Could not copy file '/builds/worker/workspace/build/src/obj-firefox/mobile/android/base/res/values/strings.xml' to '/builds/worker/workspace/build/src/obj-firefox/gradle/build/mobile/android/app/moz.build/src/officialWithoutGeckoBinariesNoMinApiPhotonDebug/res/values/strings.xml'.
[task 2018-10-03T19:07:24.896Z] 19:07:24     INFO -  * Try:
[task 2018-10-03T19:07:24.896Z] 19:07:24     INFO -  Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
[task 2018-10-03T19:07:24.897Z] 19:07:24     INFO -  * Get more help at https://help.gradle.org
[task 2018-10-03T19:07:24.897Z] 19:07:24     INFO -  BUILD FAILED in 1s
[task 2018-10-03T19:07:24.897Z] 19:07:24     INFO -  79 actionable tasks: 1 executed, 78 up-to-date
[task 2018-10-03T19:07:25.294Z] 19:07:25    ERROR -  Traceback (most recent call last):
[task 2018-10-03T19:07:25.294Z] 19:07:25     INFO -    File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
[task 2018-10-03T19:07:25.294Z] 19:07:25     INFO -      "__main__", fname, loader, pkg_name)
[task 2018-10-03T19:07:25.294Z] 19:07:25     INFO -    File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
[task 2018-10-03T19:07:25.295Z] 19:07:25     INFO -      exec code in run_globals
[task 2018-10-03T19:07:25.295Z] 19:07:25     INFO -    File "/builds/worker/workspace/build/src/python/mozbuild/mozbuild/action/file_generate.py", line 110, in <module>
[task 2018-10-03T19:07:25.295Z] 19:07:25     INFO -      sys.exit(main(sys.argv[1:]))
[task 2018-10-03T19:07:25.296Z] 19:07:25     INFO -    File "/builds/worker/workspace/build/src/python/mozbuild/mozbuild/action/file_generate.py", line 70, in main
[task 2018-10-03T19:07:25.296Z] 19:07:25     INFO -      ret = module.__dict__[method](output, *args.additional_arguments, **kwargs)
[task 2018-10-03T19:07:25.296Z] 19:07:25     INFO -    File "/builds/worker/workspace/build/src/mobile/android/gradle.py", line 39, in assemble_app
[task 2018-10-03T19:07:25.297Z] 19:07:25     INFO -      return android('assemble-app')
[task 2018-10-03T19:07:25.297Z] 19:07:25     INFO -    File "/builds/worker/workspace/build/src/mobile/android/gradle.py", line 31, in android
[task 2018-10-03T19:07:25.297Z] 19:07:25     INFO -      subprocess.check_call(cmd)
[task 2018-10-03T19:07:25.297Z] 19:07:25     INFO -    File "/usr/lib/python2.7/subprocess.py", line 186, in check_call
[task 2018-10-03T19:07:25.298Z] 19:07:25     INFO -      raise CalledProcessError(retcode, cmd)
[task 2018-10-03T19:07:25.298Z] 19:07:25     INFO -  subprocess.CalledProcessError: Command '[u'/builds/worker/workspace/build/src/mach', 'android', 'assemble-app']' returned non-zero exit status 1
[task 2018-10-03T19:07:25.301Z] 19:07:25     INFO -  backend.mk:64: recipe for target '.deps/android_apks.stub' failed
[task 2018-10-03T19:07:25.301Z] 19:07:25    ERROR -  make[3]: *** [.deps/android_apks.stub] Error 1
[task 2018-10-03T19:07:25.302Z] 19:07:25     INFO -  make[3]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox/mobile/android/base'
[task 2018-10-03T19:07:25.302Z] 19:07:25     INFO -  /builds/worker/workspace/build/src/config/rules.mk:429: recipe for target 'default' failed
[task 2018-10-03T19:07:25.302Z] 19:07:25    ERROR -  make[2]: *** [default] Error 2
[task 2018-10-03T19:07:25.303Z] 19:07:25     INFO -  make[2]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox/mobile/android/base'
[task 2018-10-03T19:07:25.303Z] 19:07:25     INFO -  Makefile:53: recipe for target 'libs-ja' failed
[task 2018-10-03T19:07:25.303Z] 19:07:25    ERROR -  make[1]: *** [libs-ja] Error 2
[task 2018-10-03T19:07:25.304Z] 19:07:25     INFO -  make[1]: Leaving directory '/builds/worker/workspace/build/src/obj-firefox/mobile/android/locales'
[task 2018-10-03T19:07:25.304Z] 19:07:25     INFO -  Makefile:81: recipe for target 'installers-ja' failed
[task 2018-10-03T19:07:25.304Z] 19:07:25     INFO -  make: *** [installers-ja] Error 2
[task 2018-10-03T19:07:25.304Z] 19:07:25    ERROR - Return code: 2
[task 2018-10-03T19:07:25.305Z] 19:07:25    ERROR - ja failed in make installers-ja!
[task 2018-10-03T19:07:25.305Z] 19:07:25    ERROR - # TBPL FAILURE #
[task 2018-10-03T19:07:25.305Z] 19:07:25  WARNING - setting return code to 2
[task 2018-10-03T19:07:25.305Z] 19:07:25    ERROR - Repacked 3 of 4 binaries successfully.

Comment 20

[Nick Thomas [:nthomas] (UTC+12)]

A little earlier in the log there is 

[task 2018-10-07T12:09:22.272Z] 12:09:22     INFO -  error: /builds/worker/workspace/build/src/obj-firefox/mobile/android/base/res/values/strings.xml:1052:75: The entity "fxaccount_status_choose_what" was referenced, but not declared.

which looks like it might be a newish string that a locale doesn't have yet, from https://hg.mozilla.org/mozilla-central/rev/9c622588634e on Sep 17. I don't know if we intend to merge in en-US strings on Android and this is failing, or we don't support that. flod, do you know ?

Comment 21

[Francesco Lodolo [:flod]]

There's an ongoing discussion with needinfo for Axel in https://bugzilla.mozilla.org/show_bug.cgi?id=1496293#c2

Comment 22

[Intermittent Failures Robot]

46 failures in 2708 pushes (0.017 failures/push) were associated with this bug in the last 7 days.

This is the #43 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. ** 

Repository breakdown:
* autoland: 7
* mozilla-central: 20
* oak: 1
* try: 15
* mozilla-inbound: 3

Platform breakdown:
* linux32: 1
* android-4-0-armv7-api16: 41
* macosx64-qr: 1
* linux64: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1452589&startday=2018-10-01&endday=2018-10-07&tree=all

Comment 23

[Intermittent Failures Robot]

19 failures in 2740 pushes (0.007 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* try: 3
* mozilla-central: 16

Platform breakdown:
* android-4-0-armv7-api16: 19

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1452589&startday=2018-10-08&endday=2018-10-14&tree=all

Comment 24

[Intermittent Failures Robot]

5 failures in 2978 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* autoland: 5

Platform breakdown:
* android-4-2-x86: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1452589&startday=2018-10-29&endday=2018-11-04&tree=all

Comment 25

[Cosmin Sabou [:CosminS]]

Hi! This bug turned into permafail after this merge: https://hg.mozilla.org/mozilla-central/pushloghtml?changeset=8540104bb0bd811f9fd45b160dbe38a67c9aa5cd

Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=213062576&repo=mozilla-central&lineNumber=2114

Treeherder: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&searchStr=android-4-0-armv7-api16-ccov%2Cdebug%2Cbuild-android-api-16-ccov%2Fdebug%2C%28b%29&fromchange=8991d660f20e3eea652e060c30e17670b45a9257&tochange=8540104bb0bd811f9fd45b160dbe38a67c9aa5cd&selectedJob=213062576

Could this be related to the changes in Bug 1508547? Thank you.

Comment 26

[Nick Alexander :nalexander [he/him]]

Cosmin, that's a reasonable guess, but I think it's not actually that.  snorp is seeing the same issue for https://bugzilla.mozilla.org/show_bug.cgi?id=1486524; there's something else that's impacting PG and it's not likely to be anything to do with clang (so not likely to be Bug 1508547).

We're investigating.

Comment 27

[Nick Alexander :nalexander [he/him]]

(In reply to Nick Alexander :nalexander [he/him] from comment #26)
> Cosmin, that's a reasonable guess, but I think it's not actually that. 
> snorp is seeing the same issue for
> https://bugzilla.mozilla.org/show_bug.cgi?id=1486524; there's something else
> that's impacting PG and it's not likely to be anything to do with clang (so
> not likely to be Bug 1508547).
> 
> We're investigating.

I've filed https://bugzilla.mozilla.org/show_bug.cgi?id=1509146 to track the permafailing issue.

Comment 28

[Cristina Coroiu [:ccoroiu]]

Today was merged the fix from 1509146, there is no new failure.
https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&selectedJob=213653365&searchStr=android-4-0-armv7-api16-ccov%2Cdebug%2Cbuild-android-api-16-ccov%2Fdebug%2C%28b%29

Comment 29

[Intermittent Failures Robot]

24 failures in 2387 pushes (0.01 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-central: 24

Platform breakdown:
* android-4-0-armv7-api16-ccov: 24

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1452589&startday=2018-11-19&endday=2018-11-25&tree=all

Comment 30

[Intermittent Failures Robot]

3 failures in 2628 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 2
• mozilla-central: 1

Platform breakdown:

• android-4-0-armv7-api16: 2
• android-4-2-x86: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1452589&startday=2019-01-28&endday=2019-02-03&tree=all

Comment 31

[Intermittent Failures Robot]

3 failures in 2561 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 3

Platform breakdown:

• android-4-0-armv7-api16-ccov: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1452589&startday=2019-03-18&endday=2019-03-24&tree=all

Comment 32

[Intermittent Failures Robot]

1 failures in 3910 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform breakdown:

• android-4-0-armv7-api16-ccov: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1452589&startday=2019-03-25&endday=2019-03-31&tree=all

Comment 33

[Intermittent Failures Robot]

1 failures in 3964 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform breakdown:

• linux64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1452589&startday=2019-10-14&endday=2019-10-20&tree=all