partner repack rebuilds not working

RESOLVED FIXED

Status

defect
RESOLVED FIXED
11 months ago
3 months ago

People

(Reporter: nthomas, Assigned: nthomas)

Tracking

({leave-open})

unspecified

Firefox Tracking Flags

(firefox-esr60 fixed, firefox63 fixed)

Details

Attachments

(1 attachment)

Assignee

Description

11 months ago
Bug 1457034 implemented an action to respin specific partner configs and was tested working at the time. Something appears to have regressed in the meantime, based on testing maple at https://tools.taskcluster.net/groups/E2NYBQ3GRl65wdhIxquaiA

Generated using https://github.com/mozilla-releng/releasewarrior-2.0/blob/master/docs/misc-operations/off-cycle-partner-repacks-and-funnelcake.md on bm83 with
 export PROMOTE_TASK_ID=ZtKvjRK3Q7OuxTS4ipfqFw
 export PARTNER_BUILD_NUM=2
 export PARTNER_SUBSET=funnelcake,seznam

Issues:
* there are funnelcake tasks but not seznam (seems to be querying prod though)
* the repack tasks fail with
 desktop_partner_repacks.py: error: ambiguous option: --p (--partner, --platform?)
due to args '--p=funnelcake --p=seznam'

Updated

11 months ago
Summary: partner reoack rebuilds not working → partner repack rebuilds not working
Assignee

Comment 1

11 months ago
The missing seznam tasks is just due to operator error - seznam only uses the cs locale, which is not in the reduced set of locales on enable.
Assignee

Comment 3

11 months ago
qwant worked OK on that graph but funnelcake artifacts are missing on the repack jobs. It turns out we only support single values for --partner-subset deeper in the stack:

* trigger_action.py turns "--partner-subset foo,bar" into 'release_partners = ['foo', 'bar']' on the action task
* taskcluster/taskgraph/util/partners.py:get_partner_config_by_kind() uses release_partners to filter the config to only include top-level partners, ie funnelcake but not funnelcake133
** in turn this informs the creation of a lot of tasks downstream via the transforms chunk_partners.py, partner_signing.py, and beetmover_repackage_partner.py

* For the repack itself, taskcluster/taskgraph/transforms/partner_repack.py:add_command_arguments() sets up "--partner-foo --partner=bar" when calling mozharness/scripts/desktop_partner_repacks.py
* only the last --partner arg is used because desktop_partner_repacks.py is expecting it to be single valued
* similarly tc-partner-repacks.py expects a single value, but it supports either top or sub-level names
* some partners have strings reused in top and sub-level, eg aol/aol

In an ideal world we would be able specify "--partner-subset funnelcake/funnelcake133,qwant" to specify multiple repacks, and optionally limit to sub-level configs.
Assignee

Updated

11 months ago
Assignee: nobody → nthomas
Assignee

Comment 4

11 months ago
https://tools.taskcluster.net/groups/BlXSJgnwSJKsGFzfc58ymw is a graph with https://hg.mozilla.org/projects/maple/rev/92f12ea70a460885d6454aaf50af5d932f98991e to support sub-config (dist_id) all the way through the stack. Got some more bugs to fix there.

------

We'll need to respin a partner on 61.0.1 in the meantime, and since it's only a single, top-level, partner config it's possible the fix in comment #2 will be sufficient. Testing with https://tools.taskcluster.net/groups/Jo3ZdkssSGe3VlrTmlJHkA via

export PROMOTE_TASK_ID=BBNtJvQPTdmZv2aVu5sY0A
export PARTNER_BUILD_NUM=3
export PARTNER_SUBSET=playanext

ie rerun comment #2 with incremented build num and just one partner. The idea would be to get that rev on release, then combine [1] with [2] to refer to the new slightly changed code when using the promote_firefox_partners action flavor.

[1] https://github.com/mozilla-releng/releasewarrior-2.0/blob/master/docs/development/Maple-(tc-relpro-development---staging).md#triggering-graphs-on-separate-revisions 
[2] https://github.com/mozilla-releng/releasewarrior-2.0/blob/master/docs/misc-operations/off-cycle-partner-repacks-and-funnelcake.md

Comment 6

11 months ago
mozreview-review
Comment on attachment 8990162 [details]
Bug 1469757 - partner repack rebuilds pass the partner names incorrectly, ?

https://reviewboard.mozilla.org/r/255178/#review262072
Attachment #8990162 - Flags: review?(aki) → review+

Comment 7

11 months ago
Pushed by nthomas@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/d32d4a18f0b7
partner repack rebuilds pass the partner names incorrectly, r=aki?
Assignee

Comment 9

11 months ago
Nice to have fix - respins will repack on all platforms, even when it's a no-op because they're not needed. We get green jobs that run for 1+ minutes, depending on the time it takes to clone gecko.

Comment 10

11 months ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/d32d4a18f0b7
Status: NEW → RESOLVED
Last Resolved: 11 months ago
Resolution: --- → FIXED
Assignee

Updated

11 months ago
Status: RESOLVED → REOPENED
Keywords: leave-open
Resolution: FIXED → ---
Assignee

Comment 11

11 months ago
Status - I was able to respin playanext (bug 1473775) with the fix in comment #8, so we can handle 'all of a single partner' case. This PR updates our docs with that restriction:
https://github.com/mozilla-releng/releasewarrior-2.0/pull/161 to make sure our docs match the current reality.

Still planning to support repacking a list of sub-configs, but other work is taking priority right now.
Assignee

Comment 12

10 months ago
(In reply to Nick Thomas [:nthomas] (UTC+12) from comment #4)
> https://tools.taskcluster.net/groups/BlXSJgnwSJKsGFzfc58ymw is a graph with
> https://hg.mozilla.org/projects/maple/rev/
> 92f12ea70a460885d6454aaf50af5d932f98991e to support sub-config (dist_id) all
> the way through the stack. Got some more bugs to fix there.

Backed this out in https://hg.mozilla.org/projects/maple/rev/7f6f3a6407ab0c141296d3759a062214446ca777, for now.
Assignee

Comment 14

3 months ago

Resolving this as the original platform vs partner ambiguity is long fixed. Filed bug 1530231 to track the lack of fine control over respins.

See Also: → 1530231
Assignee

Updated

3 months ago
Status: REOPENED → RESOLVED
Last Resolved: 11 months ago3 months ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.