392969 - implement Sign step for Bootstrap

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P2)

Description

[Robert Helmer [:rhelmer]]

Bootstrap currently has an unused "Sign" step; I am going to move this between "Repack" and "Updates" and use it to watch for win32 signed bits (with signing still done manually elsewhere).

Current process is that unsigned win32 files go in e.g.:
/firefox/nightly/2.0.0.6-candidates/rc2/unsigned/

We pull, sign and push back to:

/firefox/nightly/2.0.0.6-candidates/rc2/

Couple ways we could go here. I am thinking that the Sign step will just sit in a look and wait for something to happen, and then it'll exit so the process can go on.

One way to do this is to look for a file, that must be copied last (e.g. the signing log file). This seems like the simplest way to go, for now.

Comment 1

[Robert Helmer [:rhelmer]]

Attachment: simplest way I can think of

Comment 2

[Robert Helmer [:rhelmer]]

Attachment: sec not millis, better logfile name, grab product/version for announce

Now with 100% more testing!

Comment 3

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 277497 [details] [diff] [review]
sec not millis, better logfile name, grab product/version for announce

>Index: Bootstrap/Step/Sign.pm
>===================================================================
...
>+    my $signedDir = $config->GetFtpCandidateDir(bitsUnsigned => 0);
>+    my $unsignedDir = $config->GetFtpCandidateDir(bitsUnsigned => 1);
>+
>+    while (! -f catfile($unsignedDir . '/win32_signing.log')) {
>+        sleep(10);
>+    }

I think you want $signedDir in the -f test, unless you're proposing we stash the log in unsigned/ instead ? That'd save on removing it in the Stage step but you have to remember to tweak the target when rsyncing it up. 

Also, I had a look on the signing server and the recent trend seems to be to use rc$rc in the log name, otherwise you end up clobbering logs when signing later rc's. So maybe 
    catfile($signedDir, 'win32_signing_rc' . $rc . '.log') 
instead.

>+sub Announce {

This seems a bit weird for a manual step, since you know you did the signing. Is it for completeness, or as a cross check that you pushed up the log ? Would a message saying "Win32 builds done, please go sign them" be a useful prod when Sign starts ? (or modify the announce for the win32 Repack)

More generally, do you have thoughts on how we finish off the Stage step ? Last I checked we'd need a second Sign step to create the detached sigs & signed win32 installers, and do the final merging of the updates with everything else. I was wondering if we want to have Sign-Win32 and Sign-All steps, or we just add more code to Stage. Might want to rename this step in the first case.

Comment 4

[Robert Helmer [:rhelmer]]

(In reply to comment #3)
> (From update of attachment 277497 [details] [diff] [review])
> >Index: Bootstrap/Step/Sign.pm
> >===================================================================
> ...
> >+    my $signedDir = $config->GetFtpCandidateDir(bitsUnsigned => 0);
> >+    my $unsignedDir = $config->GetFtpCandidateDir(bitsUnsigned => 1);
> >+
> >+    while (! -f catfile($unsignedDir . '/win32_signing.log')) {
> >+        sleep(10);
> >+    }
> 
> I think you want $signedDir in the -f test, unless you're proposing we stash
> the log in unsigned/ instead ? That'd save on removing it in the Stage step but
> you have to remember to tweak the target when rsyncing it up. 

Ok I understand from talking with cf in irc now.. this only returns a signed dir if you're on win32. This step just runs on stage.

Let's just do: 
my $unsignedDir = $config->GetFtpCandidateDir(bitsUnsigned => 1);

> Also, I had a look on the signing server and the recent trend seems to be to
> use rc$rc in the log name, otherwise you end up clobbering logs when signing
> later rc's. So maybe 
>     catfile($signedDir, 'win32_signing_rc' . $rc . '.log') 
> instead.


Seems kind of redundant because it's in an "rc" dir already.. maybe this makes more sense on the other side? I'll go ahead and do this.


> >+sub Announce {
> 
> This seems a bit weird for a manual step, since you know you did the signing.
> Is it for completeness, or as a cross check that you pushed up the log ? Would
> a message saying "Win32 builds done, please go sign them" be a useful prod when
> Sign starts ? (or modify the announce for the win32 Repack)


It's more for QA's benefit.


> More generally, do you have thoughts on how we finish off the Stage step ? Last
> I checked we'd need a second Sign step to create the detached sigs & signed
> win32 installers, and do the final merging of the updates with everything else.
> I was wondering if we want to have Sign-Win32 and Sign-All steps, or we just
> add more code to Stage. Might want to rename this step in the first case.


Personally, I think we should do all signing at this stage, but I am vigorously ignoring installer signing at this point :)

The Stage step gets to the point where we have a stage-signed/ dir ready for signing, and I think we'll have to take over by hand from there for now. 

Comment 5

[Robert Helmer [:rhelmer]]

Attachment: rc in the logfilename

Comment 6

[Robert Helmer [:rhelmer]]

(In reply to comment #5)
> Created an attachment (id=277569) [details]
> rc in the logfilename


Also cleans up the GetFTPCandidates() usage

Comment 7

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 277569 [details] [diff] [review]
rc in the logfilename

r+ with some small changes ...

>Index: Bootstrap/Step/Sign.pm
...
>     my $logDir = $config->Get(sysvar => 'logDir');

Can be removed as not used any more.

>+    exit(1);

Left over from testing ?

>+    while (! -f catfile($signedDir . '/win32_signing_rc' . $rc . '.log')) {

Not really using catfile here, just concatenation. How about 
   catfile($signedDir, 'win32_signing_rc' . $rc . '.log')

Comment 8

[Robert Helmer [:rhelmer]]

Attachment: patch as landed

Checking in README;
/cvsroot/mozilla/tools/release/README,v  <--  README
new revision: 1.5; previous revision: 1.4
done
Checking in release;
/cvsroot/mozilla/tools/release/release,v  <--  release
new revision: 1.13; previous revision: 1.12
done
Checking in Bootstrap/Step/Sign.pm;
/cvsroot/mozilla/tools/release/Bootstrap/Step/Sign.pm,v  <--  Sign.pm
new revision: 1.7; previous revision: 1.6
done