Closed Bug 538540 Opened 15 years ago Closed 14 years ago

stop putting hour number in nightly directories

Categories

(Release Engineering :: General, defect, P3)

Product:
Release Engineering
Component:
General
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED WONTFIX

People

(Reporter: vlad, Assigned: joduinn)

References

Details

(Whiteboard: [automation]) 

Right now we end up with

.../2009-12-22-01-mozilla-central/
.../2009-12-22-03-mozilla-central/
.../2009-12-22-05-mozilla-central/

Each with a different set of platforms depending on what completed during that hour.  Can we just get rid of the hour bit?  Would make it much easier to find builds from the past, as well as making the directory listing much simpler.
Cleaning up the directory listing makes sense to me. One blocking issue here is what to do with respun nightlies. If we just went off and made this change they'd end up overwriting the previous build for the day, which is bad to begin with, but also will break partial updates for that build.
I think we want to keep the hour in the build filename; just not in the directory name.
Yep, what bz said -- though only problem is that we don't currently have the hour in the build filename, just the version.  Can make the scripts a little smarter, and have them see if there's an existing file of the same name as it's trying to move in, and if so then make new -build2 -build3 -build4 etc. directory?
(In reply to comment #3)
> Yep, what bz said -- though only problem is that we don't currently have the
> hour in the build filename, just the version.  Can make the scripts a little
> smarter, and have them see if there's an existing file of the same name as it's
> trying to move in, and if so then make new -build2 -build3 -build4 etc.
> directory?

We probably could; but this ends up touching a lot more systems than just changing the directory name. Makefiles in all repositories, Buildbot scripts, nightly updates, and probably more.
Ben, could we teach post_upload.py to use the YYYY-MM-DD-branch style for the first nightly, and YYYY-MM-DD-N-branch (where N>=2) for subsequent ones ? Would that avoid touching a lot of other systems ?
(In reply to comment #5)
> Ben, could we teach post_upload.py to use the YYYY-MM-DD-branch style for the
> first nightly, and YYYY-MM-DD-N-branch (where N>=2) for subsequent ones ? Would
> that avoid touching a lot of other systems ?

That sounds like it would head off a bunch of problems. Would that work in the nightly respinning case, though? (Eg, would the nightly update system be able to generate updates.)
I guess that depends on how we write the update snippet on the slave, in particular if we use the property we get back from the upload.
(In reply to comment #7)
> I guess that depends on how we write the update snippet on the slave, in
> particular if we use the property we get back from the upload.

I was thinking more about the nightly update generator - it has to parse the directory names in nightly/, doesn't it?
If we're prepared to limit the scope of this bug to builds-from-hg then I think we're OK.

The update generator has a rewrite for the change in layout on the ftp server, eg from firefox/nightly/YYYY-MM-DD-HH-branch to firefox/nightly/YYYY/MM/YYYY-MM-DD-HH-branch (with the former just being a symlink these days). That's still needed for Fx3.0 and Tb2.0 builds as they're produced by tinderbox, but we don't want to touch the upload logic there. The rewrite doesn't apply to hg builds because we already use the YYYY/MM/YY... form there.

I don't _think_ it cares otherwise because buildid's are used to determine ordering. Does that sound right to you coop ?
(In reply to comment #9)
> I don't _think_ it cares otherwise because buildid's are used to determine
> ordering. Does that sound right to you coop ?

Yes, that jives with my understanding of things.
Nobody's stepped up to take this on, and due to the wide array of testing needed it's beyond trivial. I think this belongs in Future.
Component: Release Engineering → Release Engineering: Future
Mass move of bugs from Release Engineering:Future -> Release Engineering. See
http://coop.deadsquid.com/2010/02/kiss-the-future-goodbye/ for more details.
Component: Release Engineering: Future → Release Engineering
Priority: -- → P3
Depends on: 431905
Whiteboard: [automation]
(In reply to comment #9)
> If we're prepared to limit the scope of this bug to builds-from-hg then I think
> we're OK.
Yes, totally prepared to limit scope to builds-from-hg. We're actively tracking powering off builds-from-cvs in other bugs, so no point in wasted effort.
Depends on: 554226
grabbing this, while I reconcile this with bug#431905 and bug#449607.

On first read, it seems like this bug is really about "for a given night, put all nightly builds for a branch, across all OS, in the same directory"... but let me dig some more before I morph this bug.
Assignee: nobody → joduinn
I vote for DUP to bug 584178.  If we move everything to changeset based directories, then we won't have hour numbers in the directories either!
I don't think we should move nightlies to being by changeset...  That would make them really annoying to use for their primary use case (manual bisection on dates).
Ok, how about having both date and changeset?

nightlies/2010-11-24-mozilla-central-6d28edbb9871/

Any nightlies built off that revision would end up in that directory.  If we re-build nightlies from the same revision, they get overwritten there, but would still be in tinderbox-builds.
That worskforme, I think, though it makes it a bit of a PITA to write a script to download nightlies.  Why do we need the changeset id in the directory name for the nightly archive, exactly?
I'm untangling these 4 overlapping bugs:

https://bugzilla.mozilla.org/show_bug.cgi?id=449607
https://bugzilla.mozilla.org/show_bug.cgi?id=487036
https://bugzilla.mozilla.org/show_bug.cgi?id=538540
https://bugzilla.mozilla.org/show_bug.cgi?id=584178


After some study, I believe these are about two interwoven but orthogonal issues:

1) improve regression hunting
=============================
There are a few ways to do this:
a) beltzner's tagging proposal. 
b) to look on ftp, at the txt file containing changeset which RelEng already post in a txt file alongside the build. 
c) to create a website reporting from buildbot db and present that in a public UI.

2) organize builds on ftp.m.o so that each build is in a unique and consistent location
=======================================================================================
This means build systems can programmaticly calculate locations of builds and their log files, which is needed to remove a dependency on TinderboxServer. This needs to handle rebuilds of the same changeset (like for respins of nightlies). Using changesets in the dirname is unique, but does not sort chronologically, and does not handle respins. However, using the full BuildID does handle both these usage cases.



I propose the following:

* morph https://bugzilla.mozilla.org/show_bug.cgi?id=487036 into "write tool to read buildbot db for BuildID+changesets of nightlies, and then construct URL to feed to hg pushlog". For example, buildbot status db contains a list of changesets, and buildids. RelEng would write a dashboard which queries the db and generate a URL to send to hgpushlog. For example, this URL shows all changes between Tues nightly (early morning of 14dec) and Wed nightly (early morning of 15dec): http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=66036625795f&tochange=f11f7ed625ba

* FIX https://bugzilla.mozilla.org/show_bug.cgi?id=449607 "change dated dirs on ftp.m.o to use new longer BuildID", using the full BuildID as outlined above.

* WONTFIX https://bugzilla.mozilla.org/show_bug.cgi?id=538540 "stop putting hour number in nightly directories" because of bug#449607

* WONTFIX https://bugzilla.mozilla.org/show_bug.cgi?id=584178 "list hourly tinderbox builds by changeset on ftp.mozilla.org" because of bug#449607


If I've missed anything in all this detangling, please let me know.
See Also: → 449607, 487036, 584178
So what will the resulting build locations look like on the ftp server?

And what's the plan for the thing this bug is about, which is making it easier to _find_ the nightly for a given OS for a given day?
(In reply to comment #20)
> So what will the resulting build locations look like on the ftp server?

Details are in https://bugzilla.mozilla.org/show_bug.cgi?id=449607#c0. 


> And what's the plan for the thing this bug is about, which is making it easier
> to _find_ the nightly for a given OS for a given day?

Bug#570814 has already changed nightly builds so that, for a given branch, the nightly builds for all OS all use the same revision and BuildID, and end up in the same directory. I believe this addresses Vlad's original concern, while also still allowing us to deal with respin-a-nightly scenarios.

Let me know if I missed anything.
(In reply to comment #19)
...
> * WONTFIX https://bugzilla.mozilla.org/show_bug.cgi?id=538540 "stop putting
> hour number in nightly directories" because of bug#449607
Status: NEW → RESOLVED
Closed: 14 years ago
Resolution: --- → WONTFIX
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.