1210518 - Create chunked tasks for l10n partial updates

Status: RESOLVED FIXED

(Release Engineering :: Release Automation: Other, defect, P3)

Description

[Rail Aliiev [:rail]]

We should generate partial updates in chunks instead of one funsize leaf (3 tasks atm). Possible scenarios:

1) Each l10n repack generates N chunks, where N == number of partials for a particular release. For example if we have 15 locales in a chunk and 3 partials, this will give us 3 leaves (9 tasks). One of the downsides it may take a lot of time (~2h on mac) to generate a single leaf. Also rerunning a failed task would be harder.

2) Chunkify the chunks! Each l10n repack generates NxM chunks, where N == number of partials for a particular release and M == amount of subchunks (partitions) we want to generate. For example if we have 15 locales in a chunk, 3 partials and 4 subchunks this will give us 12 leaves (36 tasks). This is how we chunkify nightly l10n partials.

Comment 1

[Justin Wood (:Callek)]

:rail, still a thing with relpromo?

Comment 2

[Rail Aliiev [:rail]]

Yeah, we still may want chunkify the chunks to parallelize MAR generation.

Comment 3

[Aki Sasaki (not active)]

Aiui funsize is chunked in TC relpromo.