313939 - use a single build ID throughout our build process

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Chase Phillips]

From bug 313923, apparently build IDs are created at least twice in our build process.  If the first build ID is generated in one hour and the second build ID is generated in a following hour, they will differ.  This only came to light recently as a problem when as it turns out software update in the client uses the second build ID while Talkback (and due to various sundry reasons) the software update infrastructure itself uses the first build ID.

Fix our build config to create just one build ID.

An example of a bad build can be seen at:
http://tinderbox.mozilla.org/showlog.cgi?log=Mozilla1.8/1130275020.2564.gz&fulltext=1

Comment 1

[Benjamin Smedberg]

The makefile BUILD_ID comes from http://lxr.mozilla.org/mozilla/source/config/config.mk#873 which is taken from config/build_number

Where is the other one coming from? Or are we updating build_number at some point during the build process...?

Comment 2

[cls]

From the log, Firefox was initially built with BUILD_ID=2005102514; as was Talkback.  Then, when Firefox was packaged using 'make install', the build id was bumped to 2005102515.  Since MOZ_OFFICIAL (or BUILD_OFFICIAL) is set, the build_number will be bumped everytime we build in config/.  

We should probably start generating the build_number upon cvs checkout in client.mk to avoid this and the liblayout/gecko-date problems.

Comment 3

[Benjamin Smedberg]

Hacking this in client.mk will not work well for people who build from tarballs/RPMs/thingslikethat. Why can't we hack it into configure?

Comment 4

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

Is one of these two IDs the same as the directory name in ftp.mozilla.org/.../nightly/, or is that a third?

It would be good if the ID for all were based on the time of the start of the build or (preferably, if pulling by date) the cvs checkout date so that people have a better idea of what source the build was based on.

Comment 5

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

(In reply to comment #3)
> Hacking this in client.mk will not work well for people who build from
> tarballs/RPMs/thingslikethat. Why can't we hack it into configure?

It will if client.mk generates a file that's present in the tarball.  (That file could also later be extended to contain branch information and then the whole thing could go in about:buildconfig.)

Comment 6

[cls]

At some point, the source used by the rpms & tarballs would have been grabbed using the checkout or fast-update target from client.mk.  If they are using some other mechanism, then they're on their own.

If we generate the build id at configure time, then trees that are built from the same source tree could have different build ids.  Granted, that's the current behavior but it's non-intuitive and something we should fix since we're revisiting this issue.

Comment 7

[Chase Phillips]

(In reply to comment #4)
> Is one of these two IDs the same as the directory name in
> ftp.mozilla.org/.../nightly/, or is that a third?

A third, which based on my understanding of post-mozilla-rel.pl is loosely "what datetime is it when we are about to upload files?" :/

> It would be good if the ID for all were based on the time of the start of the
> build or (preferably, if pulling by date) the cvs checkout date so that people
> have a better idea of what source the build was based on.

Agreed.

(In reply to comment #6)
> If we generate the build id at configure time, then trees that are built from
> the same source tree could have different build ids.  Granted, that's the
> current behavior but it's non-intuitive and something we should fix since we're
> revisiting this issue.

A compromise, then?  If we have the pull-by-date checkout time (we should if they grabbed with client.mk), then write that as the build ID.  If we don't have it by the time configure runs, then write the current datetime from configure as the build ID.

The only problem I could see with my proposal is if we assume that build IDs are associated with the time the source was pulled and some builds exist where the pull-by-date time wasn't available.  If we view it as a large enough problem, we could also store the checkout date when we have it in an about:buildconfig variable "CVS Checkout Date" and leave that undefined when we don't have it.

Comment 8

[Benjamin Smedberg]

Perhaps we should go back and decide what the buildid is supposed to represent. We're using it in the following places:

1) in appstartup, to determine when we need to remove and reregister compreg/xpti
2) in update client and server, to identify builds for updating
3) in talkback, to identify and match stacks with binaries
4) in the user agent, whatever purpose that serves
5) for non-final-release builds of "the GRE" (seamonkey or xulrunner), it is used as the GRE identifier in the windows registry/unix .conf files

We seem to have several options for where/when to get a buildid:

A) a time representing when the build was done
B) a time representing when the source was checked out
C) an external source (tinderbox scripts passing in a generated build ID to idenfity a particular build in preparation for staging); this is perhaps impractical

Option A is better for appstartup, because it changes every time you do a build. It does not need to be compiled-in

Option C is probably best for update/talkback, because it ensures uniqueness across branches etc.

I don't see any situation in which option B is best. What good does it do us if everyone who builds Firefox 1.5 from source tarballs has the same buildid? Or if a builder does a build, patches something locally, and then rebuilds and has the same buildid...

Comment 9

[Benjamin Smedberg]

By option C), I mean that the tinderbox scripts would pass in an option to the build system setting buildid = "atlantia-2005102804-ffox-1.8branch" or some other unique identifier.

Comment 10

[Chase Phillips]

(In reply to comment #8)
> Perhaps we should go back and decide what the buildid is supposed to represent.
> We're using it in the following places:
> 
> 1) in appstartup, to determine when we need to remove and reregister
> compreg/xpti
> 2) in update client and server, to identify builds for updating
> 3) in talkback, to identify and match stacks with binaries
> 4) in the user agent, whatever purpose that serves
> 5) for non-final-release builds of "the GRE" (seamonkey or xulrunner), it is
> used as the GRE identifier in the windows registry/unix .conf files
> 
> We seem to have several options for where/when to get a buildid:
> 
> A) a time representing when the build was done
> B) a time representing when the source was checked out
> C) an external source (tinderbox scripts passing in a generated build ID to
> idenfity a particular build in preparation for staging); this is perhaps
> impractical
> 
> Option A is better for appstartup, because it changes every time you do a
> build. It does not need to be compiled-in
> 
> Option C is probably best for update/talkback, because it ensures uniqueness
> across branches etc.

"uniqueness across branches" has been solved in update by using the version number the app reports back to the update server.  It was solved in Talkback by coding the version number in the Talkback client build system.  The build ID doesn't need to convey that information.

> I don't see any situation in which option B is best. What good does it do us if
> everyone who builds Firefox 1.5 from source tarballs has the same buildid? Or
> if a builder does a build, patches something locally, and then rebuilds and has
> the same buildid...

This bug has real effects for us on the ground and solving it doesn't require opening this quagmire "let's rethink build IDs" quagmire.  We have a simple problem that the build ID is reset during the build.  Let's fix that first.

Comment 11

[Benjamin Smedberg]

What target is building config/ the second time? AFAIK we don't run "make install" over the entire tree, and the second call to "config/Makefile export" happens right after (or during) talkback symbol packaging.

Comment 12

[Benjamin Smedberg]

OK, I've got this diagnosed: we're building the entire tree again at http://lxr.mozilla.org/mozilla/source/tools/tinderbox/post-mozilla-rel.pl#196 because mac_bundle_path is not set in tinder-config.

Comment 13

[Chase Phillips]

(In reply to comment #12)
> OK, I've got this diagnosed: we're building the entire tree again at
> http://lxr.mozilla.org/mozilla/source/tools/tinderbox/post-mozilla-rel.pl#196
> because mac_bundle_path is not set in tinder-config.

I'm updating it now, it will be in for rc2.  Adding code to post-mozilla-rel.pl that asserts that these variables are set properly and kills the build otherwise.