Closed Bug 412454 Opened 12 years ago Closed 12 years ago

the Bootstrap Makefile should be more generic

Categories

(Release Engineering :: General, defect, P2)

defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: bhearsum, Assigned: bhearsum)

Details

Attachments

(3 files, 5 obsolete files)

Right now it _only_ works for 1.8 staging and requires bumping whenever the version changes. It would be nice to just be able to say "make clean_stage 3.0b3", or something like that. We could probably do something clever in the BuildStep that calls it so it would read the version string in from bootstrap.cfg to get rid of the bumping annoyance.
Priority: P3 → P2
This patch to the Makefile will remove all hardcoded versions, products, and tags. It now reads in all of this information from bootstrap.cfg. I thought about having it passed in from Buildbot, but after looking at it I really don't see what difference it would make -- other than making the master.cfg uglier ;). We depend on bootstrap.cfg for every other part of Bootstrap, so it shouldn't hurt to depend on it here, too.

I've done a quick local test here and 'make stage' works as intended.

The only thing I'm unsure about here is in the 'tag -d' part. It's obviously necessary to clobber the tag for the current rc. But what happens in cases where the current rc is > 1? Should we be clobbering every tag for a specific version?

In any case, a master.cfg patch is upcoming...
Attachment #297365 - Flags: review?(rhelmer)
At some point we should move the 'sync cvsmirror' logic to the Makefile. It's not dependent on bumping, though, so it's fine for now.
Attachment #297371 - Flags: review?
Attachment #297371 - Flags: review? → review?(rhelmer)
(In reply to comment #1)
> Created an attachment (id=297365) [details]
> support different versions, products in Bootstrap Makefile
> 
> This patch to the Makefile will remove all hardcoded versions, products, and
> tags. It now reads in all of this information from bootstrap.cfg. I thought
> about having it passed in from Buildbot, but after looking at it I really don't
> see what difference it would make -- other than making the master.cfg uglier
> ;). We depend on bootstrap.cfg for every other part of Bootstrap, so it
> shouldn't hurt to depend on it here, too.
> 
> I've done a quick local test here and 'make stage' works as intended.
> 
> The only thing I'm unsure about here is in the 'tag -d' part. It's obviously
> necessary to clobber the tag for the current rc. But what happens in cases
> where the current rc is > 1? Should we be clobbering every tag for a specific
> version?


Yeah, we have just hardcoded this in the Makefile, as-needed. Couldn't you use a for loop in the makefile to support this?
(In reply to comment #3)
> (In reply to comment #1)
> > Created an attachment (id=297365) [details] [details]
> > support different versions, products in Bootstrap Makefile
> > 
> > This patch to the Makefile will remove all hardcoded versions, products, and
> > tags. It now reads in all of this information from bootstrap.cfg. I thought
> > about having it passed in from Buildbot, but after looking at it I really don't
> > see what difference it would make -- other than making the master.cfg uglier
> > ;). We depend on bootstrap.cfg for every other part of Bootstrap, so it
> > shouldn't hurt to depend on it here, too.
> > 
> > I've done a quick local test here and 'make stage' works as intended.
> > 
> > The only thing I'm unsure about here is in the 'tag -d' part. It's obviously
> > necessary to clobber the tag for the current rc. But what happens in cases
> > where the current rc is > 1? Should we be clobbering every tag for a specific
> > version?
> 
> 
> Yeah, we have just hardcoded this in the Makefile, as-needed. Couldn't you use
> a for loop in the makefile to support this?
> 

Yeah, I'm sure I can. I didn't want to look at Makefile looping unless it was necessary though...;-).
Attachment #297371 - Flags: review?(rhelmer) → review+
Note the definition change of PRODUCT_RC, from, eg, 'rc3' -> '3'.

This is really ugly :(.
Attachment #297365 - Attachment is obsolete: true
Attachment #297382 - Flags: review?(rhelmer)
Attachment #297365 - Flags: review?(rhelmer)
Attachment #297382 - Flags: review?(rhelmer) → review+
I've switched this to ftp://ftp.mozilla.org/... since there's http forwards for 3.0b2. They'll be one in place for 3.0b3 too, I bet, so better to catch this now.
Attachment #297382 - Attachment is obsolete: true
Attachment #297407 - Flags: review?
Attachment #297407 - Flags: review? → review?(rhelmer)
Forgot to checkout Bootstrap on 1.9 staging.
Attachment #297371 - Attachment is obsolete: true
Attachment #297409 - Flags: review?(rhelmer)
I've got a run going on staging-trunk-automation right now, the prestage factory has completed, so I'm reasonably sure that things are OK.
Comment on attachment 297407 [details] [diff] [review]
fix typo, use ftp:// instead of http:// for ftp.m.o

should tke out "-e robots=off" too.
Attachment #297407 - Flags: review?(rhelmer) → review+
Attached patch remove robots option (obsolete) — Splinter Review
Attachment #297407 - Attachment is obsolete: true
Attachment #297409 - Flags: review?(rhelmer) → review+
This patch appears to have worked fine on staging-trunk-automation, I think it's good for check-in. There was a failure on win32 updateverify, but I believe it is unrelated.
Attachment #297412 - Attachment is obsolete: true
Attachment #297564 - Flags: review?(rhelmer)
Attachment #297564 - Flags: review?(rhelmer) → review+
Comment on attachment 297564 [details] [diff] [review]
[checked in] add robots option back into http download

Checking in Makefile;
/cvsroot/mozilla/tools/release/Makefile,v  <--  Makefile
new revision: 1.20; previous revision: 1.19
done
Attachment #297564 - Attachment description: add robots option back into http download → [checked in] add robots option back into http download
Comment on attachment 297409 [details] [diff] [review]
[checked in] checkout bootstrap on 1.9 staging

Checking in staging/master.cfg;
/cvsroot/mozilla/tools/buildbot-configs/automation/staging/master.cfg,v  <--  master.cfg
new revision: 1.17; previous revision: 1.16
done
Checking in staging-1.9/master.cfg;
/cvsroot/mozilla/tools/buildbot-configs/automation/staging-1.9/master.cfg,v  <--  master.cfg
new revision: 1.14; previous revision: 1.13
done
Attachment #297409 - Attachment description: checkout bootstrap on 1.9 staging → [checked in] checkout bootstrap on 1.9 staging
Comment on attachment 297564 [details] [diff] [review]
[checked in] add robots option back into http download

>+	# download old builds, for l10n verify
>+	cd /data && wget -nv --cut-dirs=3 -np -r ftp://ftp.mozilla.org/pub/mozilla.org/${PRODUCT}/nightly/${OLD_VERSION}-candidates/rc1/ && mv ftp.mozilla.org/nightly/* /home/ftp/pub/${PRODUCT}/nightly/ && rm -rfv ftp.mozilla.org

Actually, for FTP the URL needs to have * at the end (just testing on staging-1.9-master) or updateverify fails, because *info.txt files are not present (should add that updateverify depends on this too in the comment).

This is needed for l10nverify, but the verify steps do not do a clean checkout, so it succeeded when really it should have failed. I am going to file a separate bug on asserting that the verify dirs are empty before beginning.
Attachment #297564 - Attachment is obsolete: true
Attachment #298259 - Flags: review?(rhelmer)
Attachment #297564 - Attachment is obsolete: false
Attachment #298259 - Flags: review?(rhelmer) → review+
Comment on attachment 298259 [details] [diff] [review]
[checked in] fix the ftp URL

Checking in Makefile;
/cvsroot/mozilla/tools/release/Makefile,v  <--  Makefile
new revision: 1.21; previous revision: 1.20
done
Attachment #298259 - Attachment description: fix the ftp URL → [checked in] fix the ftp URL
Status: ASSIGNED → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.