1250658 - balrog submitter: use blob suffixes

Status: RESOLVED WONTFIX

(Release Engineering :: Release Automation: Other, defect)

Description

[Rail Aliiev [:rail]]

balrog submitter should be able submit to blobs with different suffix so we can run current release process in parallel with release promotion.

Comment 1

[Rail Aliiev [:rail]]

Attachment: MozReview Request: Bug 1250658 - balrog submitter: use blob suffixes r?jlund

Review commit: https://reviewboard.mozilla.org/r/36167/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/36167/

Comment 2

[Rail Aliiev [:rail]]

Attachment: MozReview Request: Bug 1250658 - balrog submitter: use blob suffixes r?jlund

Review commit: https://reviewboard.mozilla.org/r/36171/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/36171/

Comment 3

[Rail Aliiev [:rail]]

https://github.com/mozilla/funsize/blob/master/docker/funsize-balrog-submitter/scripts/funsize-balrog-submitter.py#L114 is how we use it. Balrog client knows how to handle it (it adds "-dummy" to the blob name)

Comment 4

[Jordan Lund (:jlund)]

Comment on attachment 8722657 [details]
MozReview Request: Bug 1250658 - balrog submitter: use blob suffixes r?jlund

https://reviewboard.mozilla.org/r/36167/#review32755

::: buildfarm/release/release-runner.py:326
(Diff revision 1)
> +            if extra_balrog_submitter_params:

won't jinja require this to exist when it tries to evaluate it?

could we pass it even when it is the None case?

Comment 5

[Jordan Lund (:jlund)]

Comment on attachment 8722658 [details]
MozReview Request: Bug 1250658 - balrog submitter: use blob suffixes r?jlund

https://reviewboard.mozilla.org/r/36171/#review32757

::: modules/releaserunner/templates/release-runner.ini.erb:40
(Diff revision 1)
> +extra_balrog_submitter_params = <%= @env_config["extra_balrog_submitter_params"] %>

I didn't know ini allows for `item =` without a value on the other side..

Comment 6

[Rail Aliiev [:rail]]

(In reply to Jordan Lund (:jlund) from comment #5)
> Comment on attachment 8722658 [details]
> MozReview Request: Bug 1250658 - balrog submitter: use blob suffixes r?jlund
> 
> https://reviewboard.mozilla.org/r/36171/#review32757
> 
> ::: modules/releaserunner/templates/release-runner.ini.erb:40
> (Diff revision 1)
> > +extra_balrog_submitter_params = <%= @env_config["extra_balrog_submitter_params"] %>
> 
> I didn't know ini allows for `item =` without a value on the other side..

it works actually :)

https://gist.github.com/rail/49b0eab5a684cced8fee

Comment 7

[Rail Aliiev [:rail]]

(In reply to Jordan Lund (:jlund) from comment #4)
> Comment on attachment 8722657 [details]
> MozReview Request: Bug 1250658 - balrog submitter: use blob suffixes r?jlund
> 
> https://reviewboard.mozilla.org/r/36167/#review32755
> 
> ::: buildfarm/release/release-runner.py:326
> (Diff revision 1)
> > +            if extra_balrog_submitter_params:
> 
> won't jinja require this to exist when it tries to evaluate it?

We use "if ... defined" for this variable: https://github.com/mozilla/releasetasks/blob/master/releasetasks/templates/enUS.yml.tmpl#L268
 
> could we pass it even when it is the None case?

None is still "defined", what may mess up the cmd in https://github.com/mozilla/funsize/blob/master/docker/funsize-balrog-submitter/runme.sh#L22

Comment 8

[Rail Aliiev [:rail]]

Comment on attachment 8722657 [details]
MozReview Request: Bug 1250658 - balrog submitter: use blob suffixes r?jlund

See my commetns

Comment 9

[Rail Aliiev [:rail]]

Comment on attachment 8722657 [details]
MozReview Request: Bug 1250658 - balrog submitter: use blob suffixes r?jlund

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/36167/diff/1-2/

Comment 10

[Rail Aliiev [:rail]]

Attachment: MozReview Request: Bug 1250658 - balrog submitter: use blob suffixes r?jlund

Review commit: https://reviewboard.mozilla.org/r/36199/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/36199/

Comment 11

[Rail Aliiev [:rail]]

https://reviewboard.mozilla.org/r/36167/#review32795

Comment 12

[Rail Aliiev [:rail]]

Attachment: tools.diff

Comment 13

[Jordan Lund (:jlund)]

Comment on attachment 8722731 [details]
MozReview Request: Bug 1250658 - balrog submitter: use blob suffixes r?jlund

https://reviewboard.mozilla.org/r/36199/#review32807

Comment 14

[Rail Aliiev [:rail]]

Comment on attachment 8722658 [details]
MozReview Request: Bug 1250658 - balrog submitter: use blob suffixes r?jlund

remote:   https://hg.mozilla.org/build/puppet/rev/ab221f90daf6
remote:   https://hg.mozilla.org/build/puppet/rev/d816d7fc3bec

Comment 15

[Rail Aliiev [:rail]]

Comment on attachment 8722779 [details] [diff] [review]
tools.diff

https://hg.mozilla.org/build/tools/rev/d3764f6a244c

Comment 16

[Rail Aliiev [:rail]]

Comment on attachment 8722731 [details]
MozReview Request: Bug 1250658 - balrog submitter: use blob suffixes r?jlund

landed to date: https://hg.mozilla.org/projects/date/rev/a2c9b7ae135a

Comment 17

[Rail Aliiev [:rail]]

funsize worked fine, still need to verify updates.py

Comment 18

[Rail Aliiev [:rail]]

Talked to bhearsum and we decided that using a separate balrog instance would be safer for now. In case we want to switch to release promotion we can just copy the blobs manually.

remote:   https://hg.mozilla.org/build/puppet/rev/448a1c65be94
remote:   https://hg.mozilla.org/build/puppet/rev/d06d48ed8a3c