Closed Bug 627182 Opened 9 years ago Closed 8 years ago

Automate signing and publishing of XULRunner builds

Categories

(Release Engineering :: Release Automation: Other, defect, P2)

x86
All
defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: nthomas, Assigned: rail)

References

Details

(Whiteboard: [release][automation])

Attachments

(4 files, 5 obsolete files)

We do these steps manually
 https://wiki.mozilla.org/Releases/Firefox_3.6.13/BuildNotes#XULRunner_Signing
to create a XULRunner release to match each Firefox release. We should automate it too.
With this, bug 565882, and a push to mirrors builder, we would pretty much have automated XULRunner builds.
The signing portion of this is fixed by bug 509158, I think?
Assignee: nobody → rail
Priority: P5 → P3
Depends on: 732962
Priority: P3 → P2
Attached patch tools (obsolete) — Splinter Review
* generate checksums: allow override product, ssh user name/key
* push to mirrors:  allow override product, ssh user name/key; shouldn't exclude zip files for xulrunner

to be tested
Component: Release Engineering → Release Engineering: Releases
QA Contact: release → bhearsum
Attached patch buildbotcustom (obsolete) — Splinter Review
* Uploads files to win32 instead of unsigned/win32 (post_upload.py --signed ...)
* Generates MD5/SHA1/SHA512SUMS files automatically
* Pushes to the internal mirrors just after checksums generation
Duplicate of this bug: 565882
Component: Release Engineering: Releases → Release Engineering: Automation
QA Contact: bhearsum → catlee
Attached patch tools (obsolete) — Splinter Review
Comments incoming
Attachment #602901 - Attachment is obsolete: true
Attachment #605931 - Flags: review?(bhearsum)
Comment on attachment 605931 [details] [diff] [review]
tools

Review of attachment 605931 [details] [diff] [review]:
-----------------------------------------------------------------

::: scripts/release/generate-sums.py
@@ +96,5 @@
> +    if productName == 'firefox':
> +        cmd = 'mkdir -v -m 2775 %s/contrib %s/contrib-localized' % \
> +            (candidatesDir, candidatesDir)
> +        run_remote_cmd(cmd, server=stageServer, username=stageUsername,
> +                       sshKey=stageSshKey)

candidates are applicable to firefox only.

::: scripts/release/push-to-mirrors.py
@@ -29,5 @@
>  DEFAULT_RSYNC_EXCLUDES = ['--exclude=*tests*',
>                            '--exclude=*crashreporter*',
>                            '--exclude=*.log',
>                            '--exclude=*.txt',
> -                          '--exclude=*.zip',

Xulrunner needs *.zip files (SDK)
Attached patch buildbotcustom (obsolete) — Splinter Review
This enables signing on demand for Xulrunner. The patch shouldn't be landed unless we are sure that there is no 11.0 chemspill or we need to land the build system patches to m-r.
Attachment #602910 - Attachment is obsolete: true
Attachment #605942 - Flags: review?(bhearsum)
Comment on attachment 605931 [details] [diff] [review]
tools

Review of attachment 605931 [details] [diff] [review]:
-----------------------------------------------------------------

::: scripts/release/generate-sums.py
@@ +96,5 @@
> +    if productName == 'firefox':
> +        cmd = 'mkdir -v -m 2775 %s/contrib %s/contrib-localized' % \
> +            (candidatesDir, candidatesDir)
> +        run_remote_cmd(cmd, server=stageServer, username=stageUsername,
> +                       sshKey=stageSshKey)

Can we find a better way of doing this than special casing the product, please? I think it's better to pass an explicit --create-contrib option (or maybe --no-contrib) from release.py since we're already forced into special casing things there.

::: scripts/release/push-to-mirrors.py
@@ +165,5 @@
> +    stageSshKey = options.ssh_key or branchConfig["stage_ssh_key"]
> +    stageSshKey = path.join(os.path.expanduser("~"), ".ssh", stageSshKey)
> +
> +    if productName != 'xulrunner':
> +        DEFAULT_RSYNC_EXCLUDES.append('--exclude=*.zip')

Similar thing here. I'm not sure if it's better to have an --exclude-zip option or --additional-exclude=*.zip though.
Attachment #605931 - Flags: review?(bhearsum) → review-
Attachment #605942 - Flags: review?(bhearsum)
Attached patch toolsSplinter Review
To be tested
Attachment #605931 - Attachment is obsolete: true
Attachment #607148 - Flags: feedback?(bhearsum)
Attached patch buildbotcustom (obsolete) — Splinter Review
Attachment #605942 - Attachment is obsolete: true
Attachment #607148 - Flags: feedback?(bhearsum) → feedback+
Attached patch builbotcustomSplinter Review
Attachment #607149 - Attachment is obsolete: true
Attachment #608726 - Flags: review?(bhearsum)
Comment on attachment 607148 [details] [diff] [review]
tools

worked fine in staging
Attachment #607148 - Flags: review?(bhearsum)
Attachment #608726 - Flags: review?(bhearsum) → review+
Attachment #607148 - Flags: review?(bhearsum) → review+
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
bustage found, thanks to jhford:
AssertionError: <buildbotcustom.scheduler.AggregatingScheduler instance at 0x13eb2320> uses unknown builder release-mozilla-1.9.2-xulrunner_checksums

patch incoming
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Attached patch extra checksSplinter Review
Added extra checks. git diff -b (ignore blanks) looks like https://gist.github.com/2211305
Attachment #609564 - Flags: review?(jhford)
Comment on attachment 609564 [details] [diff] [review]
extra checks

looks good to me
Attachment #609564 - Flags: review?(jhford) → review+
Bulk move of bugs to Release Automation component.
Component: Release Engineering: Automation (General) → Release Engineering: Automation (Release Automation)
Flags: checked-in+
No longer blocks: hg-automation
Mass move of bugs to Release Automation component.
No longer blocks: hg-automation
this was deployed during a reconfig today
Somehow, the attached hunk hasn't been applied to the final push. :/
win32 build went to unsigned in 12.0b3
Attachment #610132 - Flags: review?(bhearsum)
Comment on attachment 610132 [details] [diff] [review]
missing post_upload param

Review of attachment 610132 [details] [diff] [review]:
-----------------------------------------------------------------

d'oh!
Attachment #610132 - Flags: review?(bhearsum) → review+
Attachment #610132 - Flags: checked-in+
Attachment #607148 - Flags: checked-in+
Attachment #609564 - Flags: checked-in+
Attachment #608726 - Flags: checked-in+
(In reply to Rail Aliiev [:rail] from comment #23)
> Created attachment 610132 [details] [diff] [review]
> missing post_upload param
> 
> Somehow, the attached hunk hasn't been applied to the final push. :/
> win32 build went to unsigned in 12.0b3

Actually it was removed by another patch http://hg.mozilla.org/build/buildbotcustom/rev/8aff61059e92#l3.563
These changes went live around 11:55AM PDT.
Status: REOPENED → RESOLVED
Closed: 8 years ago8 years ago
Resolution: --- → FIXED
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.