1269731 - partner-repacks push to mirrors should work on reruns

Status: RESOLVED FIXED

(Release Engineering :: Release Automation: Other, defect)

Description

[Rail Aliiev [:rail]]

ATM we use 2 separate push to mirrors: the first starts when all deliverables except partner repacks are ready, the second starts when partner-repacks are ready.

In the following scenario the second push to mirrors fails:

1) a partner repack build starts, uploads some files
2) the first push to mirrors happens
3) a partner repack builder fails for some reason
4) another run of partner repacks runs and uploads different binaries to the candidates directory
5) partner-repacks push to mirrors fails because the files in 1) and 4) do not match

We should either optimize the partner repacks script to not overwrite the files or separate the push to mirrors scripts to not copy all files. I'd prefer the latter, probably.

Comment 1

[Rail Aliiev [:rail]]

Attachment: PR

Comment 2

[Rail Aliiev [:rail]]

Comment on attachment 8751368 [details] [review]
PR

merged and deployed

Comment 3

[Rail Aliiev [:rail]]

It turns out that adding --exclude overrides the default excludes. We don't pass --exclude anywhere except in this case, so I thought I can be brave and move things around a bit. :)

Comment 4

[Rail Aliiev [:rail]]

Attachment: MozReview Request: Bug 1269731 - partner-repacks push to mirrors should work on reruns r=nthomas a=release DONTBUILD

Review commit: https://reviewboard.mozilla.org/r/53052/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/53052/

Comment 5

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 8753169 [details]
MozReview Request: Bug 1269731 - partner-repacks push to mirrors should work on reruns r=nthomas a=release DONTBUILD

https://reviewboard.mozilla.org/r/53052/#review49900

Given how we are using this right now it seems fine to switch the mode. The virus scanner code did something similar, but that's now replaced with TC tasks right ? Do we av scan partner builds ?

::: testing/mozharness/scripts/release/push-candidate-to-releases.py:56
(Diff revision 1)
> +                r"^.*robocop.apk$",
> +                r"^.*contrib.*",
> +                r"^.*/beetmover-checksums/.*$",
> +            ],
>              "action": "append",
> -            "help": "List of patterns to exclude from copy. See script source for default.",
> +            "help": "List of patterns to exclude from copy.",

Please modify the help to make it clear you can only extend the list of excludes.

Comment 6

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/3270563e2403

Comment 7

[Rail Aliiev [:rail]]

Comment on attachment 8753169 [details]
MozReview Request: Bug 1269731 - partner-repacks push to mirrors should work on reruns r=nthomas a=release DONTBUILD

https://hg.mozilla.org/integration/mozilla-inbound/rev/3270563e2403

Comment 8

[Rail Aliiev [:rail]]

Comment on attachment 8753169 [details]
MozReview Request: Bug 1269731 - partner-repacks push to mirrors should work on reruns r=nthomas a=release DONTBUILD

https://hg.mozilla.org/releases/mozilla-beta/rev/b7bf16280fb9
https://hg.mozilla.org/releases/mozilla-release/rev/8533d1610f0f
https://hg.mozilla.org/releases/mozilla-esr45/rev/e0a272d5e162


TODO: aurora

Comment 9

[Rail Aliiev [:rail]]

Comment on attachment 8753169 [details]
MozReview Request: Bug 1269731 - partner-repacks push to mirrors should work on reruns r=nthomas a=release DONTBUILD

aurora: https://hg.mozilla.org/releases/mozilla-aurora/rev/73cd8f727097

Comment 10

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/3270563e2403

Comment 11

[Rail Aliiev [:rail]]

Worked find in 47.0b7 \o/

Comment 12

[Firefox Bug Husbandry Bot]

Removing leave-open keyword from resolved bugs, per :sylvestre.