26798 - Build IDs of non-official builds are misleading

Status: RESOLVED FIXED

(SeaMonkey :: Build Config, enhancement, P3)

Description

[Sean Richardson]

As it stands, it appears that all builds of Mozilla display a Build ID string
at the bottom right of the browser window even when it is wrong and misleading.
Other than offical builds, the current date and time is not used, because
if it were, the ID could be newer than the code that was build (see bug 24353
for that). 

On the other hand, as it is *any* non-official build, including those that
may get wide distribution, like the Debian-frozen M12 binary, which had the
nonsensical Build ID of 1999082316 (see bug 24799 and bug 22604), display
*some* Build ID.

The danger is that another non-official build might get wide distribution,
with an out-of-date but believable Build ID a week or two old, wreaking havoc 
on the ability of bug screeners to tell a regression from a DUP of a fixed
bug, and for that matter a Milestone release from a nightly binary.

In general, though, it's a bad idea to be displaying inaccurate Build IDs,
as both the Bug Writing Guidelines at 
http://www.mozilla.org/quality/bug-writing-guidelines.html
and the Bugzilla Helper at http://www.mozilla.org/quality/help/bug-form.html
ask for the Build ID string, and asking for the identity of the build in more 
than one way is unlikely to be helpful for getting good info from beginning
bug reporters.

The better solution is probably along the lines of having a file deep in the
SeaMonkey tree overwritten with the Build ID string each hour by at least
two cron jobs running on at least two machines, and then setting up the non-
official build process to interpolate the contents of that file rather than
using the build time for the build ID. That way, if a day or two go by before
the build, the ID will still reflect the age of the code, but not ever be
older than the code.

A less preferable alternative would be to drop the Build ID display from 
builds that are more than "n" days more recent than the code built. Since the
Build ID is a number that gets used elsewhere too, it couldn't be surpressed
itself, so this would mean conditional compilation of the code that displays 
it, complicating things. 

That's still better than collecting Build IDs in bug reports that are just 
plain *wrong*, though, from people who would never know the difference.

Comment 1

[Chris Yeh]

Well, there are two problems here.

First, when I turned on build number generation all the time, what ended up 
happening was that engineers would change the build number in their own 
development builds, and then check it in with their chrome changes. Duh.

We can fix this by resetting the checked in number to 0000000000, but I can't 
guarentee how long it'll stay there.

Cronjobs are also unsuitable as a solution, because of the way the build numbers 
are structured: YYYYMMDDHH. That means 24 updates to a cvs file per day, which 
equals 8670 revisions a year. The RCS file would just get to be too large and 
unwieldy.

The right thing to do is change the perl scripts to always insert 0000000000 
into a development build. That way it would be clear to people that it's a 
development version, and not a official release.

Comment 2

[Sean Richardson]

Understood; this leaves the problem of widely redistributed non-official builds
of the Milestone releases, though. The only solution I see there is better
documentation where it would get seen along the lines of, "If you are going to
redistribute, please get the Build ID to match the official build ID for the
Milestone." I don't know why a separate Debian Linux M12 was created, but
if it was a matter of library linking, it could happen again. Is it possible
to add such a warning, or would it just be a waste of time?

Comment 3

[Daniel (Leaf) Nunes]

Endico is working on a document describing how to produce distributable bits
(such that the build sets the Buildid). Note, the buildid for each milestone
build can be different; the windows, mac, and linux buildids are rarely the
same, due to different build times. That's not a problem, though.

Comment 4

[sarnold]

I have a thought you guys may or may not like..

Why not make the build id a bit .. more informative? I know debian will, and 
redhat and suse will probably, build their own .deb or .rpm from source. I 
imagine some others will wish the same.

So, why not make the build id be the most recently updated bit of code (perhaps 
that cron-job? :) *in addition to the uid@host info from whoever actually built 
the thing*. This way, redhat .rpms will be visibly different from the 
other .rpms, and provide useful info on how old the code is.

Something like: yyyymmddhhmm:user@host.

:)

Comment 5

[sarnold]

Any interested in this bug might also be interested in bug 24353.

Comment 6

[cls]

I guess we're either waiting for documentation from Dawn or we're following bug
24353's example.

Comment 7

[Dawn Endico]

http://www.mozilla.org/build/distribution.html

The build docs are here. An "Official" build means a build that gets
put on the ftp site and distributed and that many people could report
bugs on. "Offical" does not mean, "build created by leaf".

The build IDs are for identifying which build a person used, not for
identifying the milestone. If there's more than one build for a given
platform, you can tell them apart using the build ID.

I like cyeh's suggestion of changing the perl scripts to always insert
0000000000 into development builds. Marking as an enhancement request,
adding helpwanted keyword, and reassigning back to cls to close if he 
doesn't like this idea or to let sit around as a helpwanted bug.

Comment 8

[cls]

Attachment: Always set buildid to 00000000 unless MOZILLA_OFFICIAL is defined

Comment 9

[cls]

I don't think we want to use the buildid to "brand" a Mozilla build but I sorta
like the idea of adding user@host to the buildid.

Comment 10

[Dawn Endico]

yeah, adding user@host seems like it might make it easier to keep track of 
which builds people are using when looking at bug reports.

Comment 11

[Sean Richardson]

Ok, I understand not wanting to brand with build dates, but if a CVS build
will get 0000000000 to make sure that its build date is not invalid (more
recent or earlier than the code), then the same reasoning would suggest that 
Official builds for milestones should be branded with the Milestone number of
the frozen code that it is built from, because the actual build date and time 
will be unreliable for identifying the age of the code.

Quoting from bug 24353, "Are Build IDs on CVS-pull builds out of date?"
  > --- Additional Comments From [e-mail]  2000-01-19 10:37 ---
  > ...
  > Buildid only tells us when it was built, which doesn't necessarily imply 
  > when the source tree was pulled which is what we really need to know.

The solution here could be to add a string, updated once per Milestone and
used only for Official builds, like " (M14)", to be appended to the Build ID.

The other half would be to amend the instructions at 
http://www.mozilla.org/quality/bug-writing-guidelines.html and
http://www.mozilla.org/quality/help/bug-form.html
to ask for ask for the "M" number if available, so that bug screeners, QA 
people, and engineers can tell a tardy Official build from a nightly binary.

Comment 12

[cls]

Well, I just hit a snag.  I modified mozilla/config/aboutime.pl to add
":user@host" to the buildid. No problem.  However, it appears as though
xpinstall actually uses the build id to store in the registry.  And of course,
they treat the build id as a int32.  I can probably change the code to ignore
everything after the ":" but how do we really want to handle this?  Is the
buildid meant to be merely informative (date) or do we actually want to have a
visible official branding there?

Comment 13

[cls]

Attachment: Add prelim support for strings in buildid

Comment 14

[cls]

Ok, I bypassed the xpinstall problem by just using a separate define in
nsBuildId.h for the string.  So by default, the build id string will be set to
user@host.  If MOZILLA_OFFICIAL_STRING is set, the build id will be set to that.
We should probably get someone on the xpinstall in on this to see if it's
possible to add the build string to the registry as well.  Or do we even want to
bother with it?

Comment 15

[Daniel Veditz [:dveditz]]

We also need some sort of a "Pull ID". The mostly-agreed-on user agent string 
is supposed to contain a Gecko/sourcedate token, where the sourcedate indicates 
when the files were pulled.  As I understood it, anyway--this should be checked 
with [e-mail] (or [e-mail]). That way if some 
Mozilla-derivative comes out after two months of polishing the "Gecko" date 
will jibe with what the mozilla community expects from Gecko than the actual 
product's build date.

userAgent is orthogonal to this discussion, but I raise it because as a 
developer (who doesn't plan to release anything) the "pull date" is what I'm 
concerned with when I'm trying to track down bugs and it would be nice if 
*that's* what showed up for my builds.

I posted patches to do this for windows, but cyeh didn't like it because of the 
"checking in modified files" problem. He was pushing for a solution that used 
the checked-in files as a template but didn't modify them directly. 

Comment 16

[cls]

*** Bug 29424 has been marked as a duplicate of this bug. ***

Comment 17

[cls]

After talking to cbegle on irc, it's fairly clear that the buildid is mostly
irrelevant and UserAgent is what we want to concentrate on.  I still would like
to set the default to 0, unless MOZILLA_OFFICIAL is set, just to alleviate user
confusion.

cyeh: Is there any reason why build_number is checked in at all?  It seems to me
that this file should just be generated for the local build.  Currently, to
force the build id to 0, I have to use the FORCE rule that's already in place. 
This means that anything that depends upon build_number or nsBuildId.h will be
recompiled on each pass of the tree.

dveditz:  Can you attach those patches to this bug?  I'd like to come up with
the equivalent for the unix builds.

Comment 18

[Asa Dotzler [:asa]]

*** Bug 30329 has been marked as a duplicate of this bug. ***

Comment 19

[Chris Yeh]

mass re-assign of all bugs where i was listed as the qa contact

Comment 20

[Andreas Franke (gone)]

This problem (misleading build ID) happened to 
	mozilla-i686-pc-linux-gnu-M16-mathml-svg.tar.gz
which is likely to become fairly widespread.
Also I came across at least one bug report recently where someone had specified
the (wrong, misleading) build ID although it was a recent CVS pull, which caused
some irritation.

Since there seems to be a patch available to clear the build ID,
can someone please check it in? It's not the perfect solution, but
at least better than nothing.

Comment 21

[Daniel (Leaf) Nunes]

I'll take a look at fixing this next week; potentially by removing the checked
in file.

Comment 22

[cls]

Checked in a fix for the unix builds which causes build_number to always be set
to 00000000 unless MOZILLA_OFFICIAL is defined. 

Comment 23

[Daniel Roberts]

Yeah--bug 42138 is an excellent example of the kind of irritation with false
Build ID's that Andreas Franke mentioned above.

Comment 24

[R.K.Aa.]

Just d/l'ed a build from official ftp site - mozilla-i686-pc-linux-gnu.tar.gz -
and it has build ID 0000000000
Which in the long run could become a little misleading as well?

Comment 25

[cls]

Not really.  If someone wants to report a bug, the BuildId is not primary means
by which to identify which build they are using.  Hence the setting of all
non-release buildids to 000000000.  A full path to the tarball you downloaded
would help more than a BuildId.  And a pull_id as suggested by dveditz would
probably help more but I'm not sure how we'd do that.

Comment 26

[R.K.Aa.]

And what if full path is ftp.mozilla.org/latest ?
There's a good chance this will cause more confusion than it removes..

Comment 27

[cls]

IMO, the problem with identifying builds from the ftp should be solved by
putting a identifying string in the name of the tarball.  Or just getting rid of
the latest directory.

I don't see how the current situation is more confusing that the previous one. 
Instead of a non-sensical buildid caused by lazy engineers checking in code they
shouldn't have, you always have a build id of 0000000 unless it's an "official"
build. Of course, this  doesn't alieviate the problem of multiple vendor's
"official" builds, mind you.  For that, I'll probably have to resurrect the
NS_BUILD_ID_STR patch.

Comment 28

[R.K.Aa.]

The vast majority of downloaders download the precompiled mozilla from ftp.
This now has a 000.. string. Since M13 i have not ONCE seen a binary there with
wrong build ID, and i downloaded them pretty much all.

When attempting to help out with bugs, i have found the build ID a great help as
a point of reference. Now I'll have to ask people if they remember the directory
they downloaded from - or *when* they downloaded it from the ../latest dir.
Which will generate a lot more discussion back and forth, checking "visited
link" on the ftp site etc etc - lots more work in other words. Which I don't
need.

This must be the most idotic idea i've seen implemented in project till now.

Goodbye Bugzilla - I'm outta here.

Comment 29

[Jason Eager]

Why don't we just have the build id be automatically generated by client.mk (on
the client side) at the end of every
CVS pull? Then the Build ID will become the last time that the person pulled
from the tree. Just
have it be the date and time according to GMT on the person's machine (or PDT if
necessary).

That way, the ftp downloads will have the date that the tree was last pulled
automatically.

this (0000000000) stuff is really bad.

Comment 30

[David Krause]

The mozilla nightly builds should not have a 0000000000 build id.  Whoever
builds the nightlies (leaf?) needs to set the MOZILLA_OFFICIAL variable.  This
could get really confusing if not.  It's not like the nightlies are going to
have the wrong build id, because they are build from the development branch
everyday. I do like the idea of having cvs builds be 0000000000 unless the
variable is set though, but whoever does the nightlies needs to set it on their box.

Comment 31

[timeless]

mozbot is reliant on /latest/ if you make a change that harms it I'm assigning 
the bug to you.

Comment 32

[cls]

Wow, they're really coming out of the woodwork for this one.  Maybe I should up
the severity. I suspect the nightly build script only sets BUILD_OFFICIAL so I
replaced the MOZILLA_OFFICIAL ifdef to one that checks for both of them.  

Comment 33

[Daniel Roberts]

There seems to be a lot of frustration on both sides of this issue.  Let's face
it folks, this is a lose-lose situation, no matter WHAT you do.  If the Build ID
is set in client.mk everytime Mozilla is pulled/built, then it takes forever to
build, since it has to re-build EVERYTHING that has the Build ID in it.  On the
other hand, if we do the 0000000 thing as is being done right now, then people
start complaining that the Build ID is useless.  This is really a stupid
quarrel.  I vote that we leave it the way it is now, and get this over with.

Comment 34

[R.K.Aa.]

There was another "misleading build ID" bug reported, regarding the 000.. string.
(bug 44987). Since the build ID's now seem to be back on the nightlies, perhaps
someone should set that as WFM - or even "FIXED".
Btw.. the 000 string was commented upon in several other bugs during the short
hours it lasted. Seemed to cause frustration amongst bug-reporters as well.

Glad it's back. Now crawling back into the woodwork to snuggle wiff da bugs ;)

Comment 35

[Daniel (Leaf) Nunes]

chris, you can mark this fixed. your solution is 100% appropriate, only released
builds should have ids.

Comment 36

[Jason Eager]

Wait a minute. How many files actually rely on the build id? I couldn't imagine
it would be
more then a few, especially since we've just proved that you can stick any old
number
in there and not screw up mozilla.

This change is *REALLY* annoying some of our best bug finders.

Comment 37

[cls]

Leaf, unfortunately, I cannot.  Someone (read not me) still has to do win & mac.

jce, the build id is back in nightlies. The number of files affected directly
numbers 3.  2 of which are chrome.  Then there are the depends of nsBuildId.h
which are at least 2 files in 2 different libraries.  

If people are still being bugged by the 0...0, then they should
1) use a nightly, which gives us a better chance of narrowing down the problem
anyways
or
2) create their own "official" build using the instructions at
http://www.mozilla.org/build/distribution.html , which doesn't really help us
any more than no buildid because we still don't know exactly what went into that
build.

Reassigning to leaf for win32 and CC'ing a couple of mac gurus.

Comment 38

[Dawn Endico]

Please let me know what http://www.mozilla.org/build/distribution.html
should say. It currently says to set both BUILD_OFFICIAL and MOZILLA_OFFICIAL.
I think one of these is unneeded and is only used by netscape build automation
or leaf's staging scripts?

Comment 39

[Daniel (Leaf) Nunes]

accepting for windows; dawn, it looks like setting both MOZILLA_OFFICIAL and
BUILD_OFFICIAL are required, we should get rid of one or the other, though,
since they mean the same thing.

Comment 40

[Jason Eager]

I'll take a look at the link about making my build "official" and hack something
into my local copy
of client.mk to automatically generate a build id for my cvs builds. I asked
about the number
of files that it affects mainly to see if the complaint about it taking forever
to rebuild is true. With 8
files and 2 libraries (unless one of those libraries is libraptorhtml.so), I
don't think that it takes too
long to rebuild.

Comment 41

[Daniel Roberts]

*ANYTHING* that goes above and beyond meaningful changes takes too long, IMHO.
If I make a minor change to something in my own tree, for example, then I don't
want to have to re-build those extra 8 files/2 libraries just to test my
1-liners all the time.  I want 1-liners to take a few seconds, not an extra minute.

Not only this, but as someone else pointed out, making your own build "Official"
will simply degrade the meaning for QA folks, since there is no way to guarantee
that people's own personal builds have the prescribed stuff in them.  Only
builds *officially* built by mozilla.org have this guarantee (and are therefore
worthy of a Build ID, IMHO).

Comment 42

[Jason Eager]

If you're talking about testing your own "one line" changes, then I don't think
you'd be doing a CVS pull. I would only have the client.mk file generate a 
new build id when actually pulling from CVS. If you're pulling from CVS, you're
usually going to have at least 8 files changed, if not more.

I just want some simple way of remembering when I last did a cvs pull. The Build
ID would serve that purpose for me if I changed it locally.

Comment 43

[Daniel (Leaf) Nunes]

having a buildid for a cvs pulled tree, based on the build time, doesn't make
sense, it doesn't matter how much time it adds or does not add to the build.

The idea of a buildid is that a particular build from a particular machine is
identified by it. I'm going to mark this fixed when i get windows to show
00000000, and when mac shows that as well.

Comment 44

[cls]

Attachment: Add support for zero'd build id to windows. switch to using template files and update perl scripts

Comment 45

[cls]

Attachment: template for build.dtd

Comment 46

[cls]

Attachment: template for nsBuildID.h

Comment 47

[cls]

Attachment: proposed build changes for mac

Comment 48

[cls]

So, the not-so-current plan is to set all of the build ids to zero if
MOZILLA_OFFICIAL isn't set (in one fashion or another).  The latest set of
patches makes that possible on windows & mac by moving the logic that was in the
unix makefile into the perl script used by all platforms.  As part of this
patchset, we switch to using template files for nsBuildID.h & build.dtd so that
we avoid modifying files that are checked into the tree (and therefore avoid the
accidental checkins).  build.dtd comes from the extraction of the actual buildId
entity from navigator.dtd.

nsBuildId.h.in needs to be placed in config/
build.dtd.in needs to be placed in xpfe/global/

Although previously the buildId was part of navigator.dtd which was in
en-US.jar, I decided to place it in toolkit.jar (under content/global) since it
really isn't locale dependent.  If anyone has any better ideas for placing it,
let us know.
 
Thanks to Peter "jag" Annema for his help on this.

Comment 49

[Simon Fraser [no longer active]]

patch for Mac looks ok, but I'd much prefer we use a .pm for this. system(perl) 
on Mac is evil.

Comment 50

[cls]

Attachment: Updated patch containing changes for all platforms to use .pm

Comment 51

[cls]

Attachment: config/mozBDate.pm

Comment 52

[cls]

Attachment: Make sure to add config/ to @INC in necessary mac build scripts

Comment 53

[Scott Collins]

I have applied all the 10/09 mac specific patches to my trunk, and am building
both debug and optimized.  I'll post the results as soon as I have them.

Comment 54

[Scott Collins]

# Undefined subroutine &main::cwd called.
File 'Firefly:Trees:MozillaTrunk:Tree:mozilla:build:mac:BuildNGLayoutDebug.pl';
Line 27
# BEGIN failed--compilation aborted.
File 'Firefly:Trees:MozillaTrunk:Tree:mozilla:build:mac:BuildNGLayoutDebug.pl';
Line 30

I'll try moving the |use cwd;| line to before the |BEGIN|.

Comment 55

[Scott Collins]

hmmm

Usage: Firefly:Trees:MozillaTrunk:Tree:mozilla:build:mac:BuildNGLayoutDebug.pl
{gen [outfile official] | sub outfile numfile [infile]}


OK, let's move on and try an optimized build (with the |Cwd| fix).

Comment 56

[Scott Collins]

Same problem.  Hmmm.  Chris, help me figure out what's going wrong here.

Comment 57

[cls]

It looks like I screwed up the Mac portion of the patch.  The BEGIN{} section
needs to come after the 'use' clauses.  Wrt "{gen [outfile official] | sub
outfile numfile [infile]}", it looks like the patch to build/mac/Moz.pm (from
attach 16600) wasn't applied. 

Comment 58

[Scott Collins]

Attachment: Here are the updated patches for the Mac specific files in Chris' earlier attachments.

Comment 59

[Scott Collins]

Note also that the unconditionally executed portion of his new module must be
removed on Mac ... Chris: will you post a new version that satisfies all clients?

Comment 60

[cls]

Attachment: Updated mozDBate.pm file without a main section

Comment 61

[cls]

Attachment: Updated patches for unix & win32

Comment 62

[Daniel (Leaf) Nunes]

r=leaf for patchset 17228

Comment 63

[Axel Hecht]

Hi,
the following was tested on both solaris and windows, I don't know about the
mac (someone?)
perl -e"use POSIX qw(strftime);print strftime('%Y%m%d%H',gmtime());print
qq(\n);"
Two things, first, it doesn't use the localtime, which is worth a shot. IMHO.
Having local timestamps is quite confusing to people on my side of the ocean.

I have cut down the formatting of the time to a oneliner, using the POSIX module
and the strftime function. This should be available on the mac, too.
Could someone test this?
Comments?

Axel

Comment 64

[Daniel (Leaf) Nunes]

the problem is that *not* having local timestamps would be misleading to the
majority of current testers. The *right* solution is to have the buildid include
timezone information, but i'm not sure if that's a good idea (not sure what uses
the buildid as a number, rather than a string).

Comment 65

[cls]

Waiting for confirmation from jj that the mac automation scripts have been
updated to set MOZILLA_OFFICIAL=1.

Comment 66

[Peter "jag" Annema]

You forgot hiddenWindow.xul, which also needs the build.dtd lines added. It's
basically a mini version of navigator.xul, used on Mac for when all windows are
closed (but the app is still open).

Comment 67

[Jon Granrose]

this just broke the netscape 6 trunk build.

Comment 68

[cls]

The fix for the netscape trunk build was checked in on Friday.  If it's not
evident, the landing didn't go as smoothly as hoped.  Currently, we have a copy
of mozBDate.pm in both config/ & build/mac/ because the mac tinderbox scripts
only pull build/mac/ in order to pull the rest of the tree (equiv of
client.m[a]k).  sfraser's working on the new mac build script which will fix
this problem.

Here's the patch for hiddenWindow.xul

Index: ./xpfe/global/resources/content/hiddenWindow.xul
===================================================================
RCS file: /cvsroot/mozilla/xpfe/global/resources/content/hiddenWindow.xul,v
retrieving revision 1.29
diff -u -r1.29 hiddenWindow.xul
--- hiddenWindow.xul    2000/06/17 03:09:36     1.29
+++ hiddenWindow.xul    2000/10/22 19:46:39
@@ -39,6 +39,8 @@
 <!DOCTYPE window [
 <!ENTITY % brandDTD SYSTEM "chrome://global/locale/brand.dtd" >
 %brandDTD;
+<!ENTITY % buildDTD SYSTEM "chrome://global/content/build.dtd" >
+%buildDTD;
 <!ENTITY % navigatorDTD SYSTEM "chrome://navigator/locale/navigator.dtd" >
 %navigatorDTD;
 ]>

Comment 69

[cls]

The hidden window patch has been checked in.