474610 - Hourly/Nightly builds should have some way to see which changeset was used

Status: VERIFIED FIXED

(Firefox Build System :: General, defect, P3)

Description

[Johnathan Nightingale [:johnath]]

We drop the builds from various tinderbox machines into directories like:

http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-central-win32/1232535077/

which include zips/installers/whatever depending on platform.

These are valuable for finding regression ranges, and we have people like Jim Jeffrey who, when we find a bug, will do a binary search through them to track down the problem, which is an immensely valuable thing for them to do.

Unfortunately, right now they can't see which changeset a given build is using until they install it.  It would ease their life and get us faster regression range results if there was a way to tell by inspection whether a given hourly matters.

One thing I proposed to Ben was that we just drop a 0-byte file in the directories with the changeset ID as a name.  Of course, we could just change the naming structure of the builds or the directories, but I suspect there are parts of our infra that make assumptions about those names, which is why I suggested the extra file drop - it seems less likely that we make an assumption that would break.

How can we make their lives easier, given that our time investment here gets greatly multiplied by their efforts.

Comment 1

[[e-mail] (:bhearsum)]

This shouldn't be too tough to do. Here's what would have to done:
* MercurialBuildFactory will need a step added that creates this file
* MozillaStageUpload needs to know how to upload it.

I'm tempted to block this on https://bugzilla.mozilla.org/show_bug.cgi?id=474297, because MozillaStageUpload doesn't have a way to say 'upload this random file too', but I may not get to that bug for a week or two.

Here's the relevant code, in any case:
http://mxr.mozilla.org/mozilla/source/tools/buildbotcustom/process/factory.py#627
http://mxr.mozilla.org/mozilla/source/tools/buildbotcustom/steps/transfer.py#7

Comment 2

[Axel Hecht [:Pike]]

I'd be curious on the actual question, with a shadow of my build-database hat on, as this bug request is somewhat tainted by the proposed solution.

In particular, do you want to see the source stamp for a build, or the build for a source stamp?

Comment 3

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

(In reply to comment #0)
> We drop the builds from various tinderbox machines into directories like:
> 
> http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-central-win32/1232535077/
> 
> One thing I proposed to Ben was that we just drop a 0-byte file in the
> directories with the changeset ID as a name.  
Yep, that would work - nice suggestion. 

We will need to make sure this file doesn't accidentally get bundled up and shipped to users, though!


> Of course, we could just change
> the naming structure of the builds or the directories, but I suspect there are
> parts of our infra that make assumptions about those names, 
Yep, exactly. We did investigate using changesets for dirname, but we needed something that was chronological, and also something that worked equally with the cvs-based systems that we're still running.

> How can we make their lives easier, given that our time investment here gets
> greatly multiplied by their efforts.
Triaging to Future until someone has time for this, and to check the installers. If there's other suggestions that would help make things more efficient, please file separate bugs for them too, ok?

Comment 4

[Johnathan Nightingale [:johnath]]

Came up in conversation today with Mike, cc'ng him in

Comment 5

[Mike Shaver (:shaver emeritus)]

Why not ship the file to users?  It's 0 byte!

This would help our regression-range army of awesome work more effectively, which makes me smile.  Don't you want to see me smile? :-(

Comment 6

[Axel Hecht [:Pike]]

We do have the changesets in application.ini, and in the case of fennec, the changeset of gecko on platform.ini, too. We're using those for about:buildconfig, and increasingly for l10n builds. Note, about:buildconfig only returns the app revision, IIRC, not the toolkit one.

This bug is about having the changeset of the build without having to actually download it.

Comment 7

[Jim Jeffery not reading bug-mail 1/2/11]

I don't know how its being done, but someone is creating an hourly archive available here, that uses zip builds, but some of us testers prefer to install via the installer... This shows the changeset ID and build-times 

http://hourly-archive.localgho.st/hourly-archive2/

Comment 9

[Henrik Skupin [:whimboo][⌚️UTC+2]]

As given in my duped bug we should also do that for nightly builds. It will help us a lot for spotting the changeset id of a build when doing a regression test. Sometimes you will get a day range from the reporter and I do not want to always install/unzip the builds. Updating summary.

Comment 10

[Chris Cooper [:coop] (he/him)]

Mass move of bugs from Release Engineering:Future -> Release Engineering. See
http://coop.deadsquid.com/2010/02/kiss-the-future-goodbye/ for more details.

Comment 11

[(not currently active) Ted Mielczarek]

Attachment: generate a text file alongside application packages that includes the build ID and source changeset, and upload it with the build package [Checkin: Comment 13 & 31]

This patch makes us generate a $(package).txt file (like firefox-3.7a2pre.en-US.linux-x86_64.txt) alongside the build package when "make package" is invoked, and uploads it along with the build with "make upload". The text file contains just "$BUILDID $CHANGESET".

From reading post-upload.py, it looks like just uploading this file should make it wind up in all the right directories on FTP.

Comment 12

[[e-mail] (:bhearsum)]

Comment on attachment 428739 [details] [diff] [review]
generate a text file alongside application packages that includes the build ID and source changeset, and upload it with the build package
[Checkin: Comment 13 & 31]

Looks good to me. We should probably unify the release automation with this at some point -- we generate txt files for each build through the Buildbot factory currently. As it stands now, we'll have to throw *these* ones away. I filed bug 548549 on this follow-up.

Comment 13

[(not currently active) Ted Mielczarek]

Pushed to m-c:
http://hg.mozilla.org/mozilla-central/rev/42c7d789364e

These should start showing up in hourlies after my push, and nightlies starting tomorrow. (Assuming my reading of post-upload.py is correct.)

Comment 14

[Serge Gautherie (:sgautherie)]

Attachment: (Bv1-CC) Copy (the useful part of) it to comm-central

Comment 15

[Serge Gautherie (:sgautherie)]

Just in case, would this fix obsolete the following '?=' too?
http://mxr.mozilla.org/comm-central/search?string=MOZ_SOURCE_STAMP+%5C%3F%5C%3D&regexp=1&case=1&find=%2FMakefile.in

Comment 16

[Serge Gautherie (:sgautherie)]

Attachment: (Bv2-CC) Copy (the useful part of) it to comm-central (1.9.2+) [Checkin: See comment 24+28 & 32]

Bv1-CC, keeping m-1.9.2 support c-c has.

Comment 17

[Justin Wood (:Callek)]

Comment on attachment 428972 [details] [diff] [review]
(Bv2-CC) Copy (the useful part of) it to comm-central (1.9.2+)
[Checkin: See comment 24+28 & 32]

I suspect we may want a slightly different solution for c-c here (package a "list" of changesets perhaps)

Comment 18

[Serge Gautherie (:sgautherie)]

(In reply to comment #17)

Probably, but at least it's a start in the meantime.
Then we could file a follow-up bug.

Comment 19

[Johnathan Nightingale [:johnath]]

Ria, Jim - do these do the trick for you when finding regression windows?

(Thanks, Ted!)

Comment 20

[Jim Jeffery not reading bug-mail 1/2/11]

(In reply to comment #19)
> Ria, Jim - do these do the trick for you when finding regression windows?
> 
> (Thanks, Ted!)

Yes, its going to work out very well.  So far not had to go 'hunting', but it will be really nice down the road, at least from my standpoint.
Thanks again to Ted.

Comment 21

[Ria Klaassen (not reading all bugmail)]

(In reply to comment #19)
> Ria, Jim - do these do the trick for you when finding regression windows?
> 
> (Thanks, Ted!)

Yes, good idea, thanks for the notification.

Comment 22

[Henrik Skupin [:whimboo][⌚️UTC+2]]

This is pretty helpful and looks fantastic. Just checked the mozilla-central folder for nightly builds.

Can we get this for 1.9.2 and 1.9.1 builds too?

Comment 23

[(not currently active) Ted Mielczarek]

Comment on attachment 428739 [details] [diff] [review]
generate a text file alongside application packages that includes the build ID and source changeset, and upload it with the build package
[Checkin: Comment 13 & 31]

Can't hurt to ask.

Drivers: this is a small build-config only change that helps nightly testers by providing a text file with Build ID and changeset alongside each build.

Comment 24

[Serge Gautherie (:sgautherie)]

Comment on attachment 428972 [details] [diff] [review]
(Bv2-CC) Copy (the useful part of) it to comm-central (1.9.2+)
[Checkin: See comment 24+28 & 32]


http://hg.mozilla.org/comm-central/rev/83078dc64965

Comment 25

[Serge Gautherie (:sgautherie)]

(In reply to comment #24)
> http://hg.mozilla.org/comm-central/rev/83078dc64965

Argh! Until bug 478221 is ported in bug 496236:
http://hg.mozilla.org/comm-central/rev/6804e330ba06
(Cv1-CC) Restore removed line on "1.9.3" ftb

Comment 26

[Serge Gautherie (:sgautherie)]

(In reply to comment #25)
> (Cv1-CC) Restore removed line on "1.9.3" ftb

Ftr, not all c-c builds were affected: odd!?

Comment 27

[Serge Gautherie (:sgautherie)]

(In reply to comment #17)
> I suspect we may want a slightly different solution for c-c here (package a
> "list" of changesets perhaps)

I eventually filed bug 549958.

Comment 28

[Serge Gautherie (:sgautherie)]

Comment on attachment 428972 [details] [diff] [review]
(Bv2-CC) Copy (the useful part of) it to comm-central (1.9.2+)
[Checkin: See comment 24+28 & 32]


(In reply to comment #25)
> http://hg.mozilla.org/comm-central/rev/6804e330ba06
> (Cv1-CC) Restore removed line on "1.9.3" ftb

http://hg.mozilla.org/comm-central/rev/450e89a3fdd4
Revert patch Cv1-CC after bug 496236 patch Qv1 landing, Improve a comment.

Comment 29

[Mike Beltzner [:beltzner, not reading bugmail]]

Comment on attachment 428739 [details] [diff] [review]
generate a text file alongside application packages that includes the build ID and source changeset, and upload it with the build package
[Checkin: Comment 13 & 31]

a1919,1922=beltzner

Comment 30

[Sierk Bornemann]

Is it possible to clone this bug also for Camino and Camino nightly builds?
Camino nightlies are available here: http://caminobrowser.org/download/releases/nightly/, http://ftp.mozilla.org//pub/mozilla.org/camino/nightly/ .
These resources are also lacking a simple text file or hint, which indicates remotely the latest BuildID and changeset of the builds.

Instead of filing a new bug for Camino, wouldn't it be smarter to clone or extend this bug also for the product Camino (and platform: MacOSX) instead of limiting it to Core?

Comment 31

[(not currently active) Ted Mielczarek]

AFAIK Camino is building off of really old branches, so any fix we do here isn't going to affect those builds. You can file a new bug on Camino if you want them to pick up a similar fix. If they moved to building off of a newer branch, it's quite likely they'd get this for free. (Which is why we fix things in the Core platform!)

Pushed to 1.9.2:
http://hg.mozilla.org/releases/mozilla-1.9.2/rev/e99f405b50c1

Pushed to 1.9.1:
http://hg.mozilla.org/releases/mozilla-1.9.1/rev/790e999a3798

Comment 32

[Serge Gautherie (:sgautherie)]

Comment on attachment 428972 [details] [diff] [review]
(Bv2-CC) Copy (the useful part of) it to comm-central (1.9.2+)
[Checkin: See comment 24+28 & 32]


http://hg.mozilla.org/comm-central/rev/e2198795f0f7
(Cv1-CC) Update c-c after m-1.9.2 landing

Comment 33

[Serge Gautherie (:sgautherie)]

(In reply to comment #15)
> Just in case, would this fix obsolete the following '?=' too?

I guess it doesn't when building just these directories.

Comment 34

[Al Billings [:abillings - ex-MoCo]]

Verified for 1.9.2 and 1.9.1 by looking at the directories.