1366070 - Devedition uses the same Balrog blob names as Beta

Status: RESOLVED FIXED

(Release Engineering :: Release Automation: Other, enhancement, P1)

Description

[Rail Aliiev [:rail]]

https://dxr.mozilla.org/mozilla-central/source/taskcluster/docker/funsize-balrog-submitter/scripts/funsize-balrog-submitter.py?q=funsize-balrog-submitter.py&redirect_type=direct#160 uses appName to generate the release blob name, so we will end up submitting the data to the same blob.

appName is generated by https://dxr.mozilla.org/mozilla-central/source/taskcluster/docker/funsize-update-generator/scripts/funsize.py#230 by reading application.ini.

Comment 1

[Rail Aliiev [:rail]]

Example run: https://tools.taskcluster.net/task-inspector/#XbAPShrbTZukmCkx8Enicg/

Comment 2

[Rail Aliiev [:rail]]

I'm testing the following approach now: https://hg.mozilla.org/projects/jamun/rev/d2e96d9498f93ada6b51020734ebd18269beb5cf and a release runner hack:

diff --git a/buildfarm/release/release-runner.py b/buildfarm/release/release-runner.py
--- a/buildfarm/release/release-runner.py
+++ b/buildfarm/release/release-runner.py
@@ -499,7 +499,7 @@ def main(options):
                 "eme_free_repacks_platforms": branchConfig.get("eme_free_repacks_platforms", {}).get(release["product"], []),
                 "sha1_repacks_platforms": branchConfig.get("sha1_repacks_platforms", []),
                 "l10n_changesets": release['l10n_changesets'],
-                "extra_balrog_submitter_params": extra_balrog_submitter_params,
+                "extra_balrog_submitter_params": extra_balrog_submitter_params + " --product " + release["product"],
                 "publish_to_balrog_channels": publish_to_balrog_channels,
                 "snap_enabled": branchConfig.get("snap_enabled", {}).get(release["product"], False),
                 "update_verify_channel": branchConfig.get("update_verify_channel", {}).get(release["product"]),

Comment 3

[Rail Aliiev [:rail]]

With some little tweak (below) the tasks look better:

2017-05-19 16:46:55,593 - DEBUG - Balrog request to https://balrog-admin.stage.mozaws.net/api/releases/devedition-54.0b7-build10

They fail because we have some issues with routing: https://public-artifacts.taskcluster.net/7zJU3mCCQU2y8Zsi59aeGg/0/public/logs/live_backing.log

diff --git a/buildfarm/release/release-runner.py b/buildfarm/release/release-runner.py
--- a/buildfarm/release/release-runner.py
+++ b/buildfarm/release/release-runner.py
@@ -334,7 +334,7 @@ def main(options):
     retrying_tc_config.update({"maxRetries": 12})
     balrog_username = config['balrog'].get("username")
     balrog_password = config["balrog"].get("password")
-    extra_balrog_submitter_params = config["balrog"].get("extra_balrog_submitter_params")
+    extra_balrog_submitter_params = config["balrog"].get("extra_balrog_submitter_params", "")
     beetmover_aws_access_key_id = config["beetmover"].get("aws_access_key_id")
     beetmover_aws_secret_access_key = config["beetmover"].get("aws_secret_access_key")
     gpg_key_path = config["signing"].get("gpg_key_path")

Comment 4

[Johan Lorenzo [:jlorenzo]]

I'm looking at the routing issues.

Comment 5

[Johan Lorenzo [:jlorenzo]]

We're able to ping the machine but not able to establish an HTTPS connection to it[1]. OSI layers from 1 to 3 are fine. Either the TCP or the HTTP layer are causing problems

This can either mean:
a. Port 443 is not open for us. A firewall might block us.
b. Or he http server is simply not up.

[1] https://public-artifacts.taskcluster.net/Gvkq1BLURKSuHMqBkoaiUw/0/public/logs/live_backing.log

Comment 6

[Johan Lorenzo [:jlorenzo]]

It looks like we're firewall'd[1]:
> $ nc -vz balrog-admin.stage.mozaws.net 443
> ec2-52-71-237-144.compute-1.amazonaws.com [52.71.237.144] 443 (?) : Connection timed out

When I run the same command on my machine, I see the port 443 being open.

[1] https://public-artifacts.taskcluster.net/Gvkq1BLURKSuHMqBkoaiUw/0/public/logs/live_backing.log

Comment 7

[Johan Lorenzo [:jlorenzo]]

(In reply to Johan Lorenzo [:jlorenzo] from comment #6)
> [1]
> https://public-artifacts.taskcluster.net/Gvkq1BLURKSuHMqBkoaiUw/0/public/
> logs/live_backing.log

Correct link: https://public-artifacts.taskcluster.net/HoIQOX7gQYSL6r6s8PspHw/0/public/logs/live_backing.log

Comment 8

[Johan Lorenzo [:jlorenzo]]

Bug 1366764 was filed to track the firewall problem.

Comment 9

[Johan Lorenzo [:jlorenzo]]

Rail told me balrogVpnProxy doesn't support the staging instance yet. This means the problem isn't on the firewall side. He added he wanted to try to spin a disposable balrog instance, outside of the staging VPN.

Comment 10

[Rail Aliiev [:rail]]

Attachment: Bug 1366070 - Temp Balrog instance for jamun

Review commit: https://reviewboard.mozilla.org/r/141892/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/141892/

Comment 11

[[e-mail] (:bhearsum)]

Comment on attachment 8870456 [details]
Bug 1366070 - Temp Balrog instance for jamun

https://reviewboard.mozilla.org/r/141892/#review145548

Comment 12

[Release Engineering SlaveAPI Service]

In production: https://hg.mozilla.org/build/buildbot-configs/rev/b5aeb44130b0

Comment 13

[Release Engineering SlaveAPI Service]

In production: https://hg.mozilla.org/build/buildbot-configs/rev/b5aeb44130b0

Comment 14

[Release Engineering SlaveAPI Service]

In production: https://hg.mozilla.org/build/buildbot-configs/rev/ba5316b8399d

Comment 15

[Rail Aliiev [:rail]]

Attachment: Bug 1366070 - Override product name in Balrog submission. a=release DONTBUILD

Review commit: https://reviewboard.mozilla.org/r/141918/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/141918/

Comment 16

[Rail Aliiev [:rail]]

Attachment: Bug 1366070 - Pass --product to Balrog submitters

Review commit: https://reviewboard.mozilla.org/r/141922/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/141922/

Comment 17

[[e-mail] (:bhearsum)]

Comment on attachment 8870490 [details]
Bug 1366070 - Pass --product to Balrog submitters

https://reviewboard.mozilla.org/r/141922/#review145624

::: buildfarm/release/release-runner.py:503
(Diff revision 1)
>                  "beetmover_candidates_bucket": branchConfig["beetmover_buckets"][release["product"]],
>                  "partner_repacks_platforms": branchConfig.get("partner_repacks_platforms", {}).get(release["product"], []),
>                  "eme_free_repacks_platforms": branchConfig.get("eme_free_repacks_platforms", {}).get(release["product"], []),
>                  "sha1_repacks_platforms": branchConfig.get("sha1_repacks_platforms", []),
>                  "l10n_changesets": release['l10n_changesets'],
> -                "extra_balrog_submitter_params": extra_balrog_submitter_params,
> +                "extra_balrog_submitter_params": extra_balrog_submitter_params + " --product " + release["product"].capitalize(),

DevEdition would be better than Devedition, but meh, either is fine for Balrog release names.

Comment 18

[[e-mail] (:bhearsum)]

Comment on attachment 8870489 [details]
Bug 1366070 - Override product name in Balrog submission.  a=release DONTBUILD

https://reviewboard.mozilla.org/r/141918/#review145626

Comment 19

[Rail Aliiev [:rail]]

(In reply to Ben Hearsum (:bhearsum) from comment #17) 
> DevEdition would be better than Devedition, but meh, either is fine for
> Balrog release names.

I thought about that, but that would require to have a product->product_pretty_name mapping and use it in multiple places. I'm not sure I want to implement that. ;)

Comment 20

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/bb3f1daba7d9
Override product name in Balrog submission. r=bhearsum a=release DONTBUILD

Comment 21

[Rail Aliiev [:rail]]

https://hg.mozilla.org/releases/mozilla-beta/rev/e161af4fc095a433ca4b37c39f25406438e96aa9
https://hg.mozilla.org/releases/mozilla-release/rev/b41841055587708d6605fd3e6c400c435618eb5d
https://hg.mozilla.org/releases/mozilla-esr45/rev/6850b68cf666663d976874fef9bba0493ae8b832
https://hg.mozilla.org/releases/mozilla-esr52/rev/c6ea09d03bad3f35d0fbdcf43b8da3ff9bf7191e

FIREFOX_ESR_52_1_X_RELBRANCH: https://hg.mozilla.org/releases/mozilla-esr52/rev/378acd3b36e1f7727902ed8137a15e342e9080df

Comment 22

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/bb3f1daba7d9

Comment 23

[Mike Conley (:mconley) (:⚙️)]

https://hg.mozilla.org/projects/cedar/rev/bb3f1daba7d9fdb049fafd3ca4e7cb5318503d38
Bug 1366070 - Override product name in Balrog submission. r=bhearsum a=release DONTBUILD

Comment 24

[Release Engineering SlaveAPI Service]

In production: https://hg.mozilla.org/build/buildbot-configs/rev/b5aeb44130b0

Comment 25

[Release Engineering SlaveAPI Service]

In production: https://hg.mozilla.org/build/buildbot-configs/rev/ba5316b8399d

Comment 26

[Firefox Bug Husbandry Bot]

Removing leave-open keyword from resolved bugs, per :sylvestre.