1332407 - TC based nightlies should have en-US and "multi locale" routes as well

Status: RESOLVED FIXED

(Firefox Build System :: Task Configuration, task)

Description

[Justin Wood (:Callek)]

Attachment: Diff of routes

Johan's work with automatically pushing aurora to play store depends on `multi` locale v2 routes. This patch adds some missing routes.

f? to mshal on the route diff

Initial attachment was generated with running full taskgraph --json on a before and after of the to-be-attached reviewboard patch.

and then running ::

 cat ../jobs_test2.json | jq '[. | to_entries[] | select(.key) | .value = .value.task.routes ] | from_entries' > ../routes2.json
 cat ../jobs_test1.json | jq '[. | to_entries[] | select(.key) | .value = .value.task.routes ] | from_entries' > ../routes1.json
 diff -U8 ../routes{1,2}.json

Comment 1

[Justin Wood (:Callek)]

Attachment: Bug 1332407 - TC based nightlies should have en-US and "multi locale" routes as well.

Review commit: https://reviewboard.mozilla.org/r/105858/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/105858/

Comment 2

[Michael Shal [:mshal]]

Comment on attachment 8828457 [details] [diff] [review]
Diff of routes

>callek@Centaurus:~/mozilla/hg/mozilla-central$ diff -U8 ../routes{1,2}.json 
>--- ../routes1.json	2017-01-19 14:26:56.068267633 -0500
>+++ ../routes2.json	2017-01-19 14:26:49.392254884 -0500
>@@ -1524,16 +1524,22 @@
>     "tc-treeherder.v2.date.10b21d912f60b8de5c655799a27e9229c8761bb1.0",
>     "tc-treeherder-stage.v2.date.10b21d912f60b8de5c655799a27e9229c8761bb1.0"
>   ],
>   "build-android-api-15-nightly/opt": [
>     "index.gecko.v2.date.nightly.latest.mobile.android-api-15-opt",
>     "index.gecko.v2.date.nightly.2016.11.06.revision.10b21d912f60b8de5c655799a27e9229c8761bb1.mobile.android-api-15-opt",
>     "index.gecko.v2.date.nightly.2016.11.06.latest.mobile.android-api-15-opt",
>     "index.gecko.v2.date.nightly.revision.10b21d912f60b8de5c655799a27e9229c8761bb1.mobile.android-api-15-opt",
>+    "index.gecko.v2.date.revision.10b21d912f60b8de5c655799a27e9229c8761bb1.mobile-l10n.android-api-15-opt.en-US",
>+    "index.gecko.v2.date.pushdate.2016.11.06.20161106160309.mobile-l10n.android-api-15-opt.en-US",
>+    "index.gecko.v2.date.latest.mobile-l10n.android-api-15-opt.en-US",
>+    "index.gecko.v2.date.revision.10b21d912f60b8de5c655799a27e9229c8761bb1.mobile-l10n.android-api-15-opt.multi",
>+    "index.gecko.v2.date.pushdate.2016.11.06.20161106160309.mobile-l10n.android-api-15-opt.multi",
>+    "index.gecko.v2.date.latest.mobile-l10n.android-api-15-opt.multi",

Just wanted to double-check, but since we generate en-US and multi-l10n as part of the same build, both of these artifacts will be present under both routes, correct? Kinda similar to how the l10n chunking ends up with several routes pointing to the same group of artifacts?

>+    "index.gecko.v2.date.revision.10b21d912f60b8de5c655799a27e9229c8761bb1.mobile-l10n.android-x86-opt.en-US",
>+    "index.gecko.v2.date.pushdate.2016.11.06.20161106160309.mobile-l10n.android-x86-opt.en-US",
>+    "index.gecko.v2.date.latest.mobile-l10n.android-x86-opt.en-US",

We don't currently publish an en-US route for android-x86: https://dxr.mozilla.org/mozilla-central/rev/aa3e49299a3aa5cb0db570532e3df9e75d30c2d1/testing/mozharness/configs/builds/releng_sub_android_configs/64_x86.py#4

I believe this is because we don't do mobile-l10n.py for android-x86, and thus don't have the other locales around that would make distinguishing en-US useful.

However, if you need this for consistency with the other android platforms to support some other bug, I don't think it is wrong to add it. It just hasn't been (as of yet) necessary.

Comment 3

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8828459 [details]
Bug 1332407 - TC based nightlies should have en-US and "multi locale" routes as well.

https://reviewboard.mozilla.org/r/105858/#review107290

::: taskcluster/taskgraph/transforms/task.py:678
(Diff revision 1)
> +    task = add_l10n_index_routes(config, task, override_locale="multi")
>      return task
>  
>  
>  @index_builder('l10n')
> -def add_l10n_index_routes(config, task):
> +def add_l10n_index_routes(config, task, override_locale=None):

maybe call this "force_locale"?

Comment 4

[Justin Wood (:Callek)]

(In reply to Michael Shal [:mshal] from comment #2)
> Comment on attachment 8828457 [details] [diff] [review]
> Diff of routes
> 
> Just wanted to double-check, but since we generate en-US and multi-l10n as
> part of the same build, both of these artifacts will be present under both
> routes, correct? Kinda similar to how the l10n chunking ends up with several
> routes pointing to the same group of artifacts?
> 

That is correct.

> >+    "index.gecko.v2.date.revision.10b21d912f60b8de5c655799a27e9229c8761bb1.mobile-l10n.android-x86-opt.en-US",
> >+    "index.gecko.v2.date.pushdate.2016.11.06.20161106160309.mobile-l10n.android-x86-opt.en-US",
> >+    "index.gecko.v2.date.latest.mobile-l10n.android-x86-opt.en-US",
> 
> We don't currently publish an en-US route for android-x86:
> https://dxr.mozilla.org/mozilla-central/rev/
> aa3e49299a3aa5cb0db570532e3df9e75d30c2d1/testing/mozharness/configs/builds/
> releng_sub_android_configs/64_x86.py#4
> 
> I believe this is because we don't do mobile-l10n.py for android-x86, and
> thus don't have the other locales around that would make distinguishing
> en-US useful.
> 
> However, if you need this for consistency with the other android platforms
> to support some other bug, I don't think it is wrong to add it. It just
> hasn't been (as of yet) necessary.

I don't *need* this for consistency, but it certainly helps keep the logic here simplified and free of extra hardcodes/forks. I don't think en-US as a unique thing is that useful, but I also don't see how the consistency in terms of the routes is a bad thing either.

Hope That Helps.

Comment 5

[Justin Wood (:Callek)]

Comment on attachment 8828459 [details]
Bug 1332407 - TC based nightlies should have en-US and "multi locale" routes as well.

https://reviewboard.mozilla.org/r/105858/#review107290

> maybe call this "force_locale"?

I agree, this reads better.

Comment 6

[Justin Wood (:Callek)]

Comment on attachment 8828459 [details]
Bug 1332407 - TC based nightlies should have en-US and "multi locale" routes as well.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/105858/diff/1-2/

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4c3bda4c62e3
TC based nightlies should have en-US and "multi locale" routes as well. r=dustin

Comment 8

[Phil Ringnalda (:philor)]

https://hg.mozilla.org/mozilla-central/rev/4c3bda4c62e3

Comment 9

[Justin Wood (:Callek)]

Comment on attachment 8828459 [details]
Bug 1332407 - TC based nightlies should have en-US and "multi locale" routes as well.

This should land on aurora, its a regression of exposed routes between buildbot based nightlies and taskcluster based nightlies.

It has baked on central for around a week.

Comment 10

[Liz Henry (:lizzard) (relman/hg->git project)]

Comment on attachment 8828459 [details]
Bug 1332407 - TC based nightlies should have en-US and "multi locale" routes as well.

Fix for android locale upload issues. OK for aurora!

Comment 11

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/a943b51dbee9