449208 - move Bootstrap's version bumping logic to an external script

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P2)

Description

[[e-mail] (:bhearsum)]

Bootstrap should be modified to use this script to avoid duplicating this logic anywhere.

The new script should live in http://hg.mozilla.org/build/tools.

Comment 1

[[e-mail] (:bhearsum)]

Attachment: version bump logic, in a stand-alone script

This is a straight tear-out of Bootstrap's version bump logic into this external script. Bump() is essentially the same as Bootstrap::Step::Tag::Bump. Upcoming is a diff of that logic vs. this scripts.

Comment 2

[[e-mail] (:bhearsum)]

Attachment: diff of Bootstrap's bump logic vs. external scripts

Comment 3

[[e-mail] (:bhearsum)]

Attachment: have Bootstrap call the version bump script

Pretty straightforward, I think. The downside here is going around and installing Mercurial on all of our 1.8/1.9 release machines.

Comment 4

[Chris Cooper [:coop] (he/him)]

Comment on attachment 332369 [details] [diff] [review]
version bump logic, in a stand-alone script

Three nits:

1) Need to do some input checking on the command line options. We should fail early if we're missing something.

2) If we ever expect concurrent access to the IN-/OUTFILE, please unbuffer the output, i.e. add "$|++;" at the start of the script. Probably a good idea to do this anyway.

3) Please fix the usage message to explain how @bumpFiles are specified. They're coming from @ARGV in Bump(), but that's not explained anywhere.

Comment 5

[[e-mail] (:bhearsum)]

Attachment: [checked in] add usage message, address coop's other comments

Comment 6

[Nick Thomas [:nthomas] (UTC+12)]

(In reply to comment #5)
> Created an attachment (id=332412) [details]
> add usage message, address coop's other comments

The %searchReplace definition for the client.mk case can go ? 

Comment 7

[Nick Thomas [:nthomas] (UTC+12)]

n/m, forgot we need to use this for CVS based releases too.

Comment 8

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 332376 [details] [diff] [review]
have Bootstrap call the version bump script

>Index: Bootstrap/Step/Tag/Bump.pm
>===================================================================
...
>+    $this->Shell(
>+      cmd => 'perl',
>+      cmdArgs => [catfile($buildTagDir, 'tools', 'release', 'bin',
>+                          'version-bump.pl'),

The ../bin/.. seems superfluous, is it just to match the existing dir in CVS ?

>+                  '-w', catfile($cvsrootTagDir, 'mozilla'),
>+                  '-t', $releaseTag,
>+                  '-a', $appName,
>+                  '-v', $appVersion,
>+                  '-m', $milestone,
>+                  @bumpFiles]

Please add a 
   logFile => catfile($logDir, 'tag-bump_files.log'),
to this Shell call.

r+ with that. Are you planning to run 1.8 staging to test these sort of changes as you go or at the end ?

Comment 9

[[e-mail] (:bhearsum)]

(In reply to comment #8)
> (From update of attachment 332376 [details] [diff] [review])
> >Index: Bootstrap/Step/Tag/Bump.pm
> >===================================================================
> ...
> >+    $this->Shell(
> >+      cmd => 'perl',
> >+      cmdArgs => [catfile($buildTagDir, 'tools', 'release', 'bin',
> >+                          'version-bump.pl'),
> 
> The ../bin/.. seems superfluous, is it just to match the existing dir in CVS ?
> 

Yep, I can remove that if it's unnecessary.

> >+                  '-w', catfile($cvsrootTagDir, 'mozilla'),
> >+                  '-t', $releaseTag,
> >+                  '-a', $appName,
> >+                  '-v', $appVersion,
> >+                  '-m', $milestone,
> >+                  @bumpFiles]
> 
> Please add a 
>    logFile => catfile($logDir, 'tag-bump_files.log'),
> to this Shell call.
> 

Yeah, good point.

> r+ with that. Are you planning to run 1.8 staging to test these sort of changes
> as you go or at the end ?
> 

I'm going to do a test-run once version-bump.pl is checked in. It would be awful if we discovered a problem with this in 2.0.0.17/3.0.2.

Comment 10

[[e-mail] (:bhearsum)]

Attachment: [checked in] Bump.pm, addresses Nick's comments

Going to land this today to get a test-run in before 2.0.0.17/3.0.2.

Comment 11

[[e-mail] (:bhearsum)]

Comment on attachment 332412 [details] [diff] [review]
[checked in] add usage message, address coop's other comments

changeset:   3:64d70b3564f9

Comment 12

[[e-mail] (:bhearsum)]

Comment on attachment 333400 [details] [diff] [review]
[checked in] Bump.pm, addresses Nick's comments

Checking in Step/Tag/Bump.pm;
/cvsroot/mozilla/tools/release/Bootstrap/Step/Tag/Bump.pm,v  <--  Bump.pm
new revision: 1.17; previous revision: 1.16
done

Comment 13

[[e-mail] (:bhearsum)]

Starting a test run on 1.8 staging

Comment 14

[[e-mail] (:bhearsum)]

I add to add hgToolsRepo to the staging config to do a test-run:
diff -u -r1.29 fx-moz18-staging-bootstrap.cfg
--- fx-moz18-staging-bootstrap.cfg      25 Jul 2008 13:31:07 -0000      1.29
+++ fx-moz18-staging-bootstrap.cfg      12 Aug 2008 16:20:10 -0000
@@ -80,3 +80,4 @@
 symbolServerKey  = /home/cltbld/.ssh/id_rsa
 useTarGz        = 1
 useBetaChannel  = 1
+hgToolsRepo     = http://hg.mozilla.org/build/tools
ca-245:configs bhearsum$ cvs commit -m "bug 449208: add hgToolsRepo to staging
1.8 bootstrap.cfg so we can do a test run of the new Bump.pm code."
cvs commit: Examining .
Checking in fx-moz18-staging-bootstrap.cfg;
/cvsroot/mozilla/tools/release/configs/fx-moz18-staging-bootstrap.cfg,v  <-- 
fx-moz18-staging-bootstrap.cfg
new revision: 1.30; previous revision: 1.29
done

Comment 15

[[e-mail] (:bhearsum)]

Ok, I got a good test-run out staging 1.8 today. Here's a log snippet:
log: Ending time is 07:32:27 08/14/08
log: Running shell command:
log:   arg0: perl
log:   arg1: /builds/tags/FIREFOX_2_0_0_17_BUILD1/tools/release/version-bump.pl
log:   arg2: -w
log:   arg3: /builds/tags/FIREFOX_2_0_0_17_BUILD1/cvsroot/mozilla
log:   arg4: -t
log:   arg5: FIREFOX_2_0_0_17_RELEASE
log:   arg6: -a
log:   arg7: browser
log:   arg8: -v
log:   arg9: 2.0.0.17
log:   arg10: -m
log:   arg11: 1.8.1.17
log:   arg12: client.mk
log:   arg13: browser/app/module.ver
log:   arg14: browser/config/version.txt
log:   arg15: config/milestone.txt
log: Starting time is 07:32:27 08/14/08
log: Logging output to /builds/logs/tag-bump_files.log
log: Timeout: 3600
/builds/tags/FIREFOX_2_0_0_17_BUILD1/cvsroot/mozilla/client.mk: ^#?MOZ_CO_TAG\s+=.+$ found
/builds/tags/FIREFOX_2_0_0_17_BUILD1/cvsroot/mozilla/client.mk: ^#?MOZ_CO_TAG\s+=.+$ replaced with MOZ_CO_TAG           = FIREFOX_2_0_0_17_RELEASE
/builds/tags/FIREFOX_2_0_0_17_BUILD1/cvsroot/mozilla/client.mk: ^NSPR_CO_TAG\s+=\s+\w* found
/builds/tags/FIREFOX_2_0_0_17_BUILD1/cvsroot/mozilla/client.mk: ^NSPR_CO_TAG\s+=\s+\w* replaced with NSPR_CO_TAG          = FIREFOX_2_0_0_17_RELEASE
/builds/tags/FIREFOX_2_0_0_17_BUILD1/cvsroot/mozilla/client.mk: ^NSS_CO_TAG\s+=\s+\w* found
/builds/tags/FIREFOX_2_0_0_17_BUILD1/cvsroot/mozilla/client.mk: ^NSS_CO_TAG\s+=\s+\w* replaced with NSS_CO_TAG           = FIREFOX_2_0_0_17_RELEASE
/builds/tags/FIREFOX_2_0_0_17_BUILD1/cvsroot/mozilla/client.mk: ^LDAPCSDK_CO_TAG\s+=.*$ found
/builds/tags/FIREFOX_2_0_0_17_BUILD1/cvsroot/mozilla/client.mk: ^LDAPCSDK_CO_TAG\s+=.*$ replaced with LDAPCSDK_CO_TAG      = FIREFOX_2_0_0_17_RELEASE
/builds/tags/FIREFOX_2_0_0_17_BUILD1/cvsroot/mozilla/client.mk: ^LOCALES_CO_TAG\s+=.*$ found
/builds/tags/FIREFOX_2_0_0_17_BUILD1/cvsroot/mozilla/client.mk: ^LOCALES_CO_TAG\s+=.*$ replaced with LOCALES_CO_TAG       = FIREFOX_2_0_0_17_RELEASE
/builds/tags/FIREFOX_2_0_0_17_BUILD1/cvsroot/mozilla/browser/app/module.ver: ^WIN32_MODULE_PRODUCTVERSION_STRING=2.0.0.17pre$ found
/builds/tags/FIREFOX_2_0_0_17_BUILD1/cvsroot/mozilla/browser/app/module.ver: ^WIN32_MODULE_PRODUCTVERSION_STRING=2.0.0.17pre$ replaced with WIN32_MODULE_PRODUCTVERSION_STRING=2.0.0.17
/builds/tags/FIREFOX_2_0_0_17_BUILD1/cvsroot/mozilla/browser/config/version.txt: ^2.0.0.17pre$ found
/builds/tags/FIREFOX_2_0_0_17_BUILD1/cvsroot/mozilla/browser/config/version.txt: ^2.0.0.17pre$ replaced with 2.0.0.17
/builds/tags/FIREFOX_2_0_0_17_BUILD1/cvsroot/mozilla/config/milestone.txt: ^1.8.1.17pre$ found
/builds/tags/FIREFOX_2_0_0_17_BUILD1/cvsroot/mozilla/config/milestone.txt: ^1.8.1.17pre$ replaced with 1.8.1.17
log: Ending time is 07:32:28 08/14/08

Here's a log snippet from 2.0.0.16, production, running with the old Bump.pm:
log: ^#?MOZ_CO_TAG\s+=.+$ found
log: ^#?MOZ_CO_TAG\s+=.+$ replaced with MOZ_CO_TAG           = FIREFOX_2_0_0_16_RELEASE
log: ^NSPR_CO_TAG\s+=\s+\w* found
log: ^NSPR_CO_TAG\s+=\s+\w* replaced with NSPR_CO_TAG          = FIREFOX_2_0_0_16_RELEASE
log: ^NSS_CO_TAG\s+=\s+\w* found
log: ^NSS_CO_TAG\s+=\s+\w* replaced with NSS_CO_TAG           = FIREFOX_2_0_0_16_RELEASE
log: ^LDAPCSDK_CO_TAG\s+=.*$ found
log: ^LDAPCSDK_CO_TAG\s+=.*$ replaced with LDAPCSDK_CO_TAG      = FIREFOX_2_0_0_16_RELEASE
log: ^LOCALES_CO_TAG\s+=.*$ found
log: ^LOCALES_CO_TAG\s+=.*$ replaced with LOCALES_CO_TAG       = FIREFOX_2_0_0_16_RELEASE
log: ^WIN32_MODULE_PRODUCTVERSION_STRING=2.0.0.16pre$ found
log: ^WIN32_MODULE_PRODUCTVERSION_STRING=2.0.0.16pre$ replaced with WIN32_MODULE_PRODUCTVERSION_STRING=2.0.0.16
log: ^2.0.0.16pre$ found
log: ^2.0.0.16pre$ replaced with 2.0.0.16
log: ^1.8.1.16pre$ found
log: ^1.8.1.16pre$ replaced with 1.8.1.16


Looks like everything is in order here. Does it look okay to you, Nick? We should create a new RELEASE_AUTOMATION tag if it does.

Comment 16

[[e-mail] (:bhearsum)]

Attachment: [checked in] add 'hgToolsRepo' to all release bootstrap configs w/ https

Comment 17

[Nick Thomas [:nthomas] (UTC+12)]

(In reply to comment #15)
> Looks like everything is in order here. Does it look okay to you, Nick? We
> should create a new RELEASE_AUTOMATION tag if it does.

Looks OK to me too. Is this the only change we'd make before Fx2.0.0.17/3.0.2 etc ? If not, perhaps we should tag closer to those releases.

Comment 18

[[e-mail] (:bhearsum)]

We're staring on Monday, I believe, so we could wait till then. I suspect no other changes will come in before that, though.

Comment 19

[Chris Cooper [:coop] (he/him)]

Comment on attachment 333788 [details] [diff] [review]
[checked in] add 'hgToolsRepo' to all release bootstrap configs w/ https

https:// maybe, given bug 450645? Basic patch is fine though.

Comment 20

[[e-mail] (:bhearsum)]

Yeah, I'll land it with https://

Comment 21

[[e-mail] (:bhearsum)]

Comment on attachment 333788 [details] [diff] [review]
[checked in] add 'hgToolsRepo' to all release bootstrap configs w/ https

landed w/ https:
Checking in fx-moz18-bootstrap.cfg;
/cvsroot/mozilla/tools/release/configs/fx-moz18-bootstrap.cfg,v  <--  fx-moz18-bootstrap.cfg
new revision: 1.49; previous revision: 1.48
done
Checking in fx-moz18-staging-bootstrap.cfg;
/cvsroot/mozilla/tools/release/configs/fx-moz18-staging-bootstrap.cfg,v  <--  fx-moz18-staging-bootstrap.cfg
new revision: 1.31; previous revision: 1.30
done
Checking in fx-moz19-bootstrap.cfg;
/cvsroot/mozilla/tools/release/configs/fx-moz19-bootstrap.cfg,v  <--  fx-moz19-bootstrap.cfg
new revision: 1.34; previous revision: 1.33
done
Checking in fx-moz19-staging-bootstrap.cfg;
/cvsroot/mozilla/tools/release/configs/fx-moz19-staging-bootstrap.cfg,v  <--  fx-moz19-staging-bootstrap.cfg
new revision: 1.33; previous revision: 1.32
done
Checking in tb-moz18-bootstrap.cfg;
/cvsroot/mozilla/tools/release/configs/tb-moz18-bootstrap.cfg,v  <--  tb-moz18-bootstrap.cfg
new revision: 1.27; previous revision: 1.26
done
Checking in tb-moz18-staging-bootstrap.cfg;
/cvsroot/mozilla/tools/release/configs/tb-moz18-staging-bootstrap.cfg,v  <--  tb-moz18-staging-bootstrap.cfg
new revision: 1.8; previous revision: 1.7
done
Checking in tb-moz19-bootstrap.cfg;
/cvsroot/mozilla/tools/release/configs/tb-moz19-bootstrap.cfg,v  <--  tb-moz19-bootstrap.cfg
new revision: 1.6; previous revision: 1.5
done
Checking in tb-moz19-staging-bootstrap.cfg;
/cvsroot/mozilla/tools/release/configs/tb-moz19-staging-bootstrap.cfg,v  <--  tb-moz19-staging-bootstrap.cfg
new revision: 1.3; previous revision: 1.2
done

Comment 22

[[e-mail] (:bhearsum)]

Comment on attachment 333400 [details] [diff] [review]
[checked in] Bump.pm, addresses Nick's comments

3.0.2 was bumped with this code without issue.

Comment 23

[[e-mail] (:bhearsum)]

I think we're all done here.