Automate signing and publishing of XULRunner builds

RESOLVED FIXED

Status

P2
normal
RESOLVED FIXED
8 years ago
5 years ago

People

(Reporter: nthomas, Assigned: rail)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [release][automation])

Attachments

(4 attachments, 5 obsolete attachments)

(Reporter)

Description

8 years ago
We do these steps manually
 https://wiki.mozilla.org/Releases/Firefox_3.6.13/BuildNotes#XULRunner_Signing
to create a XULRunner release to match each Firefox release. We should automate it too.
(Reporter)

Comment 1

8 years ago
With this, bug 565882, and a push to mirrors builder, we would pretty much have automated XULRunner builds.
The signing portion of this is fixed by bug 509158, I think?
Assignee: nobody → rail
Priority: P5 → P3
Depends on: 732962
Priority: P3 → P2
Created attachment 602901 [details] [diff] [review]
tools

* generate checksums: allow override product, ssh user name/key
* push to mirrors:  allow override product, ssh user name/key; shouldn't exclude zip files for xulrunner

to be tested
Component: Release Engineering → Release Engineering: Releases
QA Contact: release → bhearsum
Created attachment 602910 [details] [diff] [review]
buildbotcustom

* Uploads files to win32 instead of unsigned/win32 (post_upload.py --signed ...)
* Generates MD5/SHA1/SHA512SUMS files automatically
* Pushes to the internal mirrors just after checksums generation
Duplicate of this bug: 565882
Component: Release Engineering: Releases → Release Engineering: Automation
QA Contact: bhearsum → catlee
Created attachment 605931 [details] [diff] [review]
tools

Comments incoming
Attachment #602901 - Attachment is obsolete: true
Attachment #605931 - Flags: review?(bhearsum)
Comment on attachment 605931 [details] [diff] [review]
tools

Review of attachment 605931 [details] [diff] [review]:
-----------------------------------------------------------------

::: scripts/release/generate-sums.py
@@ +96,5 @@
> +    if productName == 'firefox':
> +        cmd = 'mkdir -v -m 2775 %s/contrib %s/contrib-localized' % \
> +            (candidatesDir, candidatesDir)
> +        run_remote_cmd(cmd, server=stageServer, username=stageUsername,
> +                       sshKey=stageSshKey)

candidates are applicable to firefox only.

::: scripts/release/push-to-mirrors.py
@@ -29,5 @@
>  DEFAULT_RSYNC_EXCLUDES = ['--exclude=*tests*',
>                            '--exclude=*crashreporter*',
>                            '--exclude=*.log',
>                            '--exclude=*.txt',
> -                          '--exclude=*.zip',

Xulrunner needs *.zip files (SDK)
Created attachment 605942 [details] [diff] [review]
buildbotcustom

This enables signing on demand for Xulrunner. The patch shouldn't be landed unless we are sure that there is no 11.0 chemspill or we need to land the build system patches to m-r.
Attachment #602910 - Attachment is obsolete: true
Attachment #605942 - Flags: review?(bhearsum)
Comment on attachment 605931 [details] [diff] [review]
tools

Review of attachment 605931 [details] [diff] [review]:
-----------------------------------------------------------------

::: scripts/release/generate-sums.py
@@ +96,5 @@
> +    if productName == 'firefox':
> +        cmd = 'mkdir -v -m 2775 %s/contrib %s/contrib-localized' % \
> +            (candidatesDir, candidatesDir)
> +        run_remote_cmd(cmd, server=stageServer, username=stageUsername,
> +                       sshKey=stageSshKey)

Can we find a better way of doing this than special casing the product, please? I think it's better to pass an explicit --create-contrib option (or maybe --no-contrib) from release.py since we're already forced into special casing things there.

::: scripts/release/push-to-mirrors.py
@@ +165,5 @@
> +    stageSshKey = options.ssh_key or branchConfig["stage_ssh_key"]
> +    stageSshKey = path.join(os.path.expanduser("~"), ".ssh", stageSshKey)
> +
> +    if productName != 'xulrunner':
> +        DEFAULT_RSYNC_EXCLUDES.append('--exclude=*.zip')

Similar thing here. I'm not sure if it's better to have an --exclude-zip option or --additional-exclude=*.zip though.
Attachment #605931 - Flags: review?(bhearsum) → review-
Attachment #605942 - Flags: review?(bhearsum)
Created attachment 607148 [details] [diff] [review]
tools

To be tested
Attachment #605931 - Attachment is obsolete: true
Attachment #607148 - Flags: feedback?(bhearsum)
Created attachment 607149 [details] [diff] [review]
buildbotcustom
Attachment #605942 - Attachment is obsolete: true
Attachment #607148 - Flags: feedback?(bhearsum) → feedback+
Created attachment 608726 [details] [diff] [review]
builbotcustom
Attachment #607149 - Attachment is obsolete: true
Attachment #608726 - Flags: review?(bhearsum)
Comment on attachment 607148 [details] [diff] [review]
tools

worked fine in staging
Attachment #607148 - Flags: review?(bhearsum)
Attachment #608726 - Flags: review?(bhearsum) → review+
Attachment #607148 - Flags: review?(bhearsum) → review+
Status: NEW → RESOLVED
Last Resolved: 7 years ago
Resolution: --- → FIXED
bustage found, thanks to jhford:
AssertionError: <buildbotcustom.scheduler.AggregatingScheduler instance at 0x13eb2320> uses unknown builder release-mozilla-1.9.2-xulrunner_checksums

patch incoming
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Created attachment 609564 [details] [diff] [review]
extra checks

Added extra checks. git diff -b (ignore blanks) looks like https://gist.github.com/2211305
Attachment #609564 - Flags: review?(jhford)
Comment on attachment 609564 [details] [diff] [review]
extra checks

looks good to me
Attachment #609564 - Flags: review?(jhford) → review+
Bulk move of bugs to Release Automation component.
Component: Release Engineering: Automation (General) → Release Engineering: Automation (Release Automation)
Flags: checked-in+
No longer blocks: 714371
Mass move of bugs to Release Automation component.
Blocks: 714371
No longer blocks: 714371
this was deployed during a reconfig today
Created attachment 610132 [details] [diff] [review]
missing post_upload param

Somehow, the attached hunk hasn't been applied to the final push. :/
win32 build went to unsigned in 12.0b3
Attachment #610132 - Flags: review?(bhearsum)
Comment on attachment 610132 [details] [diff] [review]
missing post_upload param

Review of attachment 610132 [details] [diff] [review]:
-----------------------------------------------------------------

d'oh!
Attachment #610132 - Flags: review?(bhearsum) → review+
Attachment #610132 - Flags: checked-in+
Attachment #607148 - Flags: checked-in+
Attachment #609564 - Flags: checked-in+
Attachment #608726 - Flags: checked-in+
(In reply to Rail Aliiev [:rail] from comment #23)
> Created attachment 610132 [details] [diff] [review]
> missing post_upload param
> 
> Somehow, the attached hunk hasn't been applied to the final push. :/
> win32 build went to unsigned in 12.0b3

Actually it was removed by another patch http://hg.mozilla.org/build/buildbotcustom/rev/8aff61059e92#l3.563

Comment 26

7 years ago
These changes went live around 11:55AM PDT.
Status: REOPENED → RESOLVED
Last Resolved: 7 years ago7 years ago
Resolution: --- → FIXED
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.