Closed Bug 584178 Opened 14 years ago Closed 14 years ago

list hourly tinderbox builds by changeset on ftp.mozilla.org

Categories

(Release Engineering :: General, defect, P5)

Product:
Release Engineering
Component:
General
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED WONTFIX

People

(Reporter: beltzner, Assigned: joduinn)

References

Details

(Whiteboard: [buildbot][ftp])

Right now hourly builds are really hard to match up against their changesets. That's because they're filed in ftp.mozilla.org by OS and tree first, then by UNIX time. I submit that nobody reasons about how many seconds it's been since 1969. :) Instead, I propose that we create the following structure: - categorize builds by changeset, then tree & OS - also symlink to latest-hourly by tree & OS So /tinderbox-builds /by_changeset /d271cah2 /... /mozilla-central-macosx-debug [builds here] /mozilla-central-macosx [builds here] /... /...etc... /latest-hourly /... /mozilla-central-macosx-debug [symlinks-to-builds here] /mozilla-central-macosx [symlinks-to-builds here] /... Added bonus: this gives Tinderbox and Tinderboxpushlog a clear target to link to from each changeset.
We need to handle multiple builds per push ID/platform, so not sure if that requires another level of build1/build2/etc. +1 otherwise
Summary: list hourly tinderbox builds by push ID on ftp.mozilla.org → list hourly tinderbox builds by changeset on ftp.mozilla.org
Why do we need to keep those other builds? Wouldn't they be bitwise identical?
(In reply to comment #2) > Why do we need to keep those other builds? Wouldn't they be bitwise identical? Not necessarily. Two situations come to mind: - Nightly builds with different mozconfigs, but same revision. Not sure off the top of my head if nightlies also show up in /tinderbox-builds or not - Builds triggered as a result of changing something to do with build process. e.g. changing the mozconfig, or compiler. While we're at it, can we call the top level something other than 'tinderbox-builds'? Tinderbox has nothing to do with producing the builds any more.
how about builds/ ? :)
(In reply to comment #3) > (In reply to comment #2) > > Why do we need to keep those other builds? Wouldn't they be bitwise identical? > > Not necessarily. Two situations come to mind: > - Nightly builds with different mozconfigs, but same revision. Not sure off > the top of my head if nightlies also show up in /tinderbox-builds or not They don't.
(In reply to comment #5) > (In reply to comment #3) > > (In reply to comment #2) > > > Why do we need to keep those other builds? Wouldn't they be bitwise identical? > > > > Not necessarily. Two situations come to mind: > > - Nightly builds with different mozconfigs, but same revision. Not sure off > > the top of my head if nightlies also show up in /tinderbox-builds or not > > They don't. I lied, they do.
Now that we're past bhearsum's lies, my thoughts go two ways: - we should probably be making an hg commit if we change build configs to track that something changes, this would change the changeset - if not, I'm still not sure there's additional value to keeping the old builds around, we should probably just clobber with the latest builds
(In reply to comment #7) > Now that we're past bhearsum's lies, my thoughts go two ways: > > - we should probably be making an hg commit if we change build configs to > track that something changes, this would change the changeset Few of the build configs are in the same repo as the source, so this wouldn't happen automatically. > - if not, I'm still not sure there's additional value to keeping the old > builds around, we should probably just clobber with the latest builds I disagree. If you allow overwriting builds, then test results suddenly become much trickier to associate with a particular build run, since the test run could have downloaded any of the available builds depending on exact timing and caching behaviour.
OK, that makes sense, I guess; /buildN it is, but perhaps only in cases where needed?
Priority: -- → P5
Whiteboard: [buildbot][ftp]
Grabbing, because I already have a bunch of other similar bugs.
Assignee: nobody → joduinn
(In reply to comment #11) > I submit that nobody reasons about how many seconds it's been since 1969. :) Agreed 100% :-) > Instead, I propose that we create the following structure: > - categorize builds by changeset, then tree & OS > - also symlink to latest-hourly by tree & OS > > So > > /tinderbox-builds > /by_changeset > /d271cah2 > /... > /mozilla-central-macosx-debug > [builds here] > /mozilla-central-macosx > [builds here] > /... > /...etc... > /latest-hourly > /... > /mozilla-central-macosx-debug > [symlinks-to-builds here] > /mozilla-central-macosx > [symlinks-to-builds here] > /... Thats certainly one option - some other alternatives are in: bug#449607 change dated dirs on ftp.m.o to use new longer BuildID bug#538540 stop putting hour number in nightly directories > Added bonus: this gives Tinderbox and Tinderboxpushlog a clear target to link > to from each changeset. Yep.
I'm untangling these 4 overlapping bugs: https://bugzilla.mozilla.org/show_bug.cgi?id=449607 https://bugzilla.mozilla.org/show_bug.cgi?id=487036 https://bugzilla.mozilla.org/show_bug.cgi?id=538540 https://bugzilla.mozilla.org/show_bug.cgi?id=584178 After some study, I believe these are about two interwoven but orthogonal issues: 1) improve regression hunting ============================= There are a few ways to do this: a) beltzner's tagging proposal. b) to look on ftp, at the txt file containing changeset which RelEng already post in a txt file alongside the build. c) to create a website reporting from buildbot db and present that in a public UI. 2) organize builds on ftp.m.o so that each build is in a unique and consistent location ======================================================================================= This means build systems can programmaticly calculate locations of builds and their log files, which is needed to remove a dependency on TinderboxServer. This needs to handle rebuilds of the same changeset (like for respins of nightlies). Using changesets in the dirname is unique, but does not sort chronologically, and does not handle respins. However, using the full BuildID does handle both these usage cases. I propose the following: * morph https://bugzilla.mozilla.org/show_bug.cgi?id=487036 into "write tool to read buildbot db for BuildID+changesets of nightlies, and then construct URL to feed to hg pushlog". For example, buildbot status db contains a list of changesets, and buildids. RelEng would write a dashboard which queries the db and generate a URL to send to hgpushlog. For example, this URL shows all changes between Tues nightly (early morning of 14dec) and Wed nightly (early morning of 15dec): http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=66036625795f&tochange=f11f7ed625ba * FIX https://bugzilla.mozilla.org/show_bug.cgi?id=449607 "change dated dirs on ftp.m.o to use new longer BuildID", using the full BuildID as outlined above. * WONTFIX https://bugzilla.mozilla.org/show_bug.cgi?id=538540 "stop putting hour number in nightly directories" because of bug#449607 * WONTFIX https://bugzilla.mozilla.org/show_bug.cgi?id=584178 "list hourly tinderbox builds by changeset on ftp.mozilla.org" because of bug#449607 If I've missed anything in all this detangling, please let me know.
See Also: → 449607, 487036, 538540
(In reply to comment #12) ... > * WONTFIX https://bugzilla.mozilla.org/show_bug.cgi?id=584178 "list hourly > tinderbox builds by changeset on ftp.mozilla.org" because of bug#449607
Status: NEW → RESOLVED
Closed: 14 years ago
Resolution: --- → WONTFIX
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.