1003359 - Removing a resource file causes build failures in geckoview_library (Bug 958889 required a clobber)

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Michael Comella (:mcomella) [NI reported issues only: ex-Mozilla]]

After pushing parts 1 and 2. This was likely due to the creation of a new package (`mobile/android/base/tabspanel`) with existing files from `mobile/android/base` being moved there.

The original push is:
remote:   https://hg.mozilla.org/integration/fx-team/rev/e68c18a91f3d
remote:   https://hg.mozilla.org/integration/fx-team/rev/c29c34aea5b0

The clobber push is:
https://hg.mozilla.org/integration/fx-team/rev/01acb9c03509

Comment 1

[Nick Alexander :nalexander [he/him]]

There is no evidence of a clobber needed on TBPL; the builds were killed before they finished because mcomella saw bad behaviour locally.

I pushed the two patches to an fx-team branch locally and incrementally built after each one.  My resulting tree looked like:

changeset:   203259:da4e52666704
tag:         958889.2.patch
tag:         qtip
tag:         tip
user:        Michael Comella <[e-mail]>
date:        Tue Apr 29 11:21:38 2014 -0700
summary:     Bug 958889 - Part 2: Miscellaneous code clean up. r=bnicholson

changeset:   203258:5552bb76a5dd
tag:         958889.1.patch
tag:         qbase
user:        Michael Comella <[e-mail]>
date:        Tue Apr 29 11:20:59 2014 -0700
summary:     Bug 958889 - Part 1: Move tabspanel files to new tabspanel package. r=bnicholson

changeset:   203257:3b77d1ed52af
bookmark:    fx-team/default
tag:         fx-team/default
tag:         qparent
user:        Jared Wein <[e-mail]>
date:        Mon Apr 28 14:30:48 2014 -0400
summary:     Bug 923156 - [OSX] Moving the bookmarks toolbar items into the navbar causes weird vertical alignment issues with the navbar and some buttons. r=Gijs

I built initially with |mach build| and subsequently with |./mach build mobile/android/base -v &> out.N|.  I've attached those logs, which look healthy.

My guess is that mcomella managed to get a build to start without mach noticing that |mach build-backend| was necessary.

Comment 2

[Nick Alexander :nalexander [he/him]]

Attachment: out.1

Comment 3

[Nick Alexander :nalexander [he/him]]

Attachment: out.2

Comment 4

[Michael Comella (:mcomella) [NI reported issues only: ex-Mozilla]]

I had a similar scenario occur just now after pushing bug 1000149 where it complained about various resource files. Perhaps the issue is that I've been pushing and popping patches for several bugs which all add/remove files. I imagine the build system should be capable of handling this, but it's likely not reproducable. WORKSFORME wfm.

Comment 5

[Nick Alexander :nalexander [he/him]]

(In reply to Michael Comella (:mcomella) from comment #4)
> I had a similar scenario occur just now after pushing bug 1000149 where it
> complained about various resource files. Perhaps the issue is that I've been
> pushing and popping patches for several bugs which all add/remove files. I
> imagine the build system should be capable of handling this, but it's likely
> not reproducable. WORKSFORME wfm.

Logs, please.  And STR.  My guess is that we don't notice when the Java deps change, and that |mach build-backend| is needed.  Next time you see this, do |mach build-backend && mach build mobile/android/base| and see if that fixes the issue.  If yes, we can try to understand what deps are actually failing.

Comment 6

[Michael Comella (:mcomella) [NI reported issues only: ex-Mozilla]]

(In reply to Nick Alexander :nalexander from comment #5)
> Logs, please.  And STR.

I no longer have them so I'll be sure to do it next time.

Comment 7

[Michael Comella (:mcomella) [NI reported issues only: ex-Mozilla]]

Attachment: `mach package` failure

I got another failure. I added a resource file (layout/remote_tabs_setup_inner.xml), popped the patch of which remote_tabs_setup_inner is a member (but forgot to add it meaning it hangs around), pulled and built fx-team (which I believe auto-clobbered), had the build fail because remote_tabs_setup_inner was still around, added it, qnew`d it, popped it, rebuilt, and had a failure on package. I don't think geckoview_library is properly removing resources.

Surprising to me, `rm -rf obj-android/mobile/android/geckoview_android && mach build-backend && mach build && mach package` does not fix the issue. `mach clobber && mach build...` does.

needinfo'ing Nick so he is aware of this update.

Comment 8

[Nick Alexander :nalexander [he/him]]

Attachment: Delete existing zip before zipping resources. r=mcomella

Great STR!  I can confirm that a deleted resource could hang around in
the geckoview_resources.zip.  That could definitely cause a resource
ID mismatch during packaging.  To start really clean with the Android
part of the build, try

|rm -rf $OBJDIR/mobile/android && mach build-backend && mach build mobile/android|.

Comment 9

[Nick Alexander :nalexander [he/him]]

> Surprising to me, `rm -rf obj-android/mobile/android/geckoview_android &&
> mach build-backend && mach build && mach package` does not fix the issue.
> `mach clobber && mach build...` does.

There's a typo here, but I assume you did the right thing on your machine.  The deps between mobile/android/* are contorted: mobile/android/geckoview_library references mobile/android/base/geckoview_resources.zip.

In any case, this is an issue, and we should fix it.  Great STR and log!

Comment 10

[Michael Comella (:mcomella) [NI reported issues only: ex-Mozilla]]

Comment on attachment 8420207 [details] [diff] [review]
Delete existing zip before zipping resources. r=mcomella

Review of attachment 8420207 [details] [diff] [review]:
-----------------------------------------------------------------

I would consider adding a comment saying why we delete the zip.

Comment 11

[Nick Alexander :nalexander [he/him]]

https://hg.mozilla.org/integration/fx-team/rev/3271168dd6bf

Comment 12

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/3271168dd6bf