1283438 - Android Nightly Trunk Builds fail with No rule to make target `/builds/slave/m-cen-and-api-15-l10n_2-000000/build/mozilla-central/obj-l10n/dist/fennec-50.0a1.en-US.android-arm.apk', needed by `/builds/slave/m-cen-and-api-15-l10n_2-000000/build/mozilla-cen

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Carsten Book [:Tomcat]]

04:48:17 ERROR - make: *** No rule to make target `/builds/slave/m-cen-and-api-15-l10n_2-000000/build/mozilla-central/obj-l10n/dist/fennec-50.0a1.en-US.android-arm.apk', needed by `/builds/slave/m-cen-and-api-15-l10n_2-000000/build/mozilla-central/obj-l10n/dist/l10n-stage/fennec'. Stop.

https://treeherder.mozilla.org/logviewer.html#?job_id=4194500&repo=mozilla-central#L4412

seems several android nightlys failed on m-c today.

Comment 1

[Justin Wood (:Callek)]

Attachment: Bug 1283438 - Set AUTOCLOBBER to empty for l10n .

Needed because buildbot clones/checks out from the repo head (of default)
and then updates to the rev for the nightly we're pulling, which can cause
CLOBBER file changes to initiate an unwanted clobber of the object directory
where we just pulled the nightly binary from. Even when CLOBBER hasn't actually
been touched in the changeset range we're looking at between nightlies.

Review commit: https://reviewboard.mozilla.org/r/61610/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/61610/

Comment 2

[Justin Wood (:Callek)]

After chatting on IRC today, we found the problem, which was the CLOBBER file in a changeset closer to HEAD than the nightlies themselves had changed.

Because we configure then do the wget, then pull then update then configure again it caused us to clobber the object directory over again and thus fail.

This is a stop-gap to making us just use the rev the nightly is on to begin with.... So we can not fail (as much) :-)

Comment 3

[Axel Hecht [:Pike]]

I think it's worth adding that buildbot does check out mozilla-central from the version of nightly. mozharness then checks out mozilla-central from the default branch.

Comment 4

[Justin Wood (:Callek)]

(In reply to Axel Hecht [:Pike] from comment #3)
> I think it's worth adding that buildbot does check out mozilla-central from
> the version of nightly. mozharness then checks out mozilla-central from the
> default branch.

Not quite, buildbot uses archiver to grab the mozharness scripts from the version of the nightly... and mozharness does the checkout to default.

Comment 5

[Mike Hommey [:glandium]]

Comment on attachment 8766892 [details]
Bug 1283438 - Set AUTOCLOBBER to empty for l10n .

https://reviewboard.mozilla.org/r/61610/#review58500

::: browser/config/mozconfigs/linux32/l10n-mozconfig:18
(Diff revision 1)
>  
>  ac_add_options --disable-stdcxx-compat
>  
> +# Don't autoclobber l10n, as this can lead to missing binaries and broken builds
> +# Bug 1283438
> +mk_add_options AUTOCLOBBER=

One concern is if this means those builds are never going to be clobbered. OTOH, the CLOBBER file is not involved in the things that could cause problems by not clobbering those builds...

Comment 6

[Intermittent Failures Robot]

7 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-central: 7

Platform breakdown:
* android-4-0-armv7-api15: 7

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1283438&startday=2016-06-27&endday=2016-07-03&tree=all

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/e0e56c2e1881
Set AUTOCLOBBER to empty for l10n r=glandium.

Comment 8

[Justin Wood (:Callek)]

(In reply to Mike Hommey [:glandium] from comment #5)
> Comment on attachment 8766892 [details]
> Bug 1283438 - Set AUTOCLOBBER to empty for l10n .
> 
> 
> One concern is if this means those builds are never going to be clobbered.
> OTOH, the CLOBBER file is not involved in the things that could cause
> problems by not clobbering those builds...

We do have clobberer that can still clobber them if we need to.

Comment 9

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/e0e56c2e1881