1799769 - Add tasks to import nightly Debian Packages from storage into an Artifact Registry APT repository

Status: RESOLVED FIXED

(Firefox Build System :: Task Configuration, task)

Description

[Gabriel Bustamante [:gabriel]]

These tasks depend on:

• a scriptworker supporting Artifact Registry APT repositories
• Artifact Registry repository infrastructure (staging and production repositories for Firefox releases)

Comment 1

[Gabriel Bustamante [:gabriel]]

Attachment: Bug 1799769 - Import the Firefox .deb packages to the product delivery APT repo r=jlorenzo,bhearsum

Depends on D169971

Comment 2

[Gabriel Bustamante [:gabriel]]

Attachment: Bug 1799769 - Grant gecko access to the dep APT repo r=#releng-reviewers

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/ci/ci-configuration/rev/b0fd48992dcc
Grant gecko access to the dep APT repo r=releng-reviewers,bhearsum

Comment 4

[Gabriel Bustamante [:gabriel]]

Attachment: Bug 1799769 - Grant gecko access to the production APT repo r=jlorenzo

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/ci/ci-configuration/rev/5d1ff9ec607b
Grant gecko access to the production APT repo r=jlorenzo

Comment 6

[Gabriel Bustamante [:gabriel]]

Attachment: Bug 1799769 - Import the Firefox .deb packages to the product delivery APT repository r?hneiva

Depends on D172599

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d506356d2afb
Import the Firefox .deb packages to the product delivery APT repository r=taskgraph-reviewers,hneiva

Comment 9

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/d506356d2afb

Comment 10

[Sylvestre Ledru [:Sylvestre]]

Looks like it cause this regression:
https://treeherder.mozilla.org/jobs?repo=try&revision=a59985b2eeaef1a55252ec21e738a5d702014aeb&selectedTaskRun=IDaDjWuiQbiFAWBMhWW6XQ.0

[task 2023-03-28T07:47:48.588Z] tools\lint\test\test_perfdocs.py:187: in test_perfdocs_start_and_pass
[task 2023-03-28T07:47:48.588Z]     run_perfdocs(
[task 2023-03-28T07:47:48.588Z] tools\lint\perfdocs\perfdocs.py:78: in run_perfdocs
[task 2023-03-28T07:47:48.589Z]     task_graph = generate_tasks(
[task 2023-03-28T07:47:48.589Z] tools\tryselect\tasks.py:120: in generate_tasks
[task 2023-03-28T07:47:48.589Z]     tg_full = generate("full_task_set")
[task 2023-03-28T07:47:48.589Z] tools\tryselect\tasks.py:106: in generate
[task 2023-03-28T07:47:48.589Z]     tg = getattr(generator, attr)
[task 2023-03-28T07:47:48.589Z] third_party\python\taskcluster_taskgraph\taskgraph\generator.py:159: in full_task_set
[task 2023-03-28T07:47:48.589Z]     return self._run_until("full_task_set")
[task 2023-03-28T07:47:48.589Z] third_party\python\taskcluster_taskgraph\taskgraph\generator.py:423: in _run_until
[task 2023-03-28T07:47:48.589Z]     k, v = next(self._run)
[task 2023-03-28T07:47:48.589Z] third_party\python\taskcluster_taskgraph\taskgraph\generator.py:309: in _run
[task 2023-03-28T07:47:48.589Z]     new_tasks = kind.load_tasks(
[task 2023-03-28T07:47:48.589Z] third_party\python\taskcluster_taskgraph\taskgraph\generator.py:75: in load_tasks
[task 2023-03-28T07:47:48.589Z]     tasks = [
[task 2023-03-28T07:47:48.589Z] third_party\python\taskcluster_taskgraph\taskgraph\generator.py:75: in <listcomp>
[task 2023-03-28T07:47:48.589Z]     tasks = [
[task 2023-03-28T07:47:48.589Z] taskcluster\gecko_taskgraph\transforms\task.py:2208: in check_run_task_caches
[task 2023-03-28T07:47:48.589Z]     for task in tasks:
[task 2023-03-28T07:47:48.589Z] taskcluster\gecko_taskgraph\transforms\task.py:2118: in check_task_dependencies
[task 2023-03-28T07:47:48.589Z]     for task in tasks:
[task 2023-03-28T07:47:48.589Z] taskcluster\gecko_taskgraph\transforms\task.py:2104: in check_task_identifiers
[task 2023-03-28T07:47:48.589Z]     for task in tasks:
[task 2023-03-28T07:47:48.589Z] taskcluster\gecko_taskgraph\transforms\task.py:2085: in chain_of_trust
[task 2023-03-28T07:47:48.589Z]     for task in tasks:
[task 2023-03-28T07:47:48.589Z] taskcluster\gecko_taskgraph\transforms\task.py:1888: in build_task
[task 2023-03-28T07:47:48.589Z]     for task in tasks:
[task 2023-03-28T07:47:48.589Z] taskcluster\gecko_taskgraph\transforms\task.py:1866: in set_task_and_artifact_expiry
[task 2023-03-28T07:47:48.589Z]     for job in jobs:
[task 2023-03-28T07:47:48.589Z] taskcluster\gecko_taskgraph\transforms\task.py:1852: in try_task_config_routes
[task 2023-03-28T07:47:48.589Z]     for task in tasks:
[task 2023-03-28T07:47:48.589Z] taskcluster\gecko_taskgraph\transforms\task.py:1839: in try_task_config_chemspill_prio
[task 2023-03-28T07:47:48.589Z]     yield from tasks
[task 2023-03-28T07:47:48.590Z] taskcluster\gecko_taskgraph\transforms\task.py:1819: in try_task_config_env
[task 2023-03-28T07:47:48.590Z]     yield from tasks
[task 2023-03-28T07:47:48.590Z] taskcluster\gecko_taskgraph\transforms\task.py:1786: in add_index_routes
[task 2023-03-28T07:47:48.590Z]     for task in tasks:
[task 2023-03-28T07:47:48.590Z] taskcluster\gecko_taskgraph\transforms\task.py:1581: in validate
[task 2023-03-28T07:47:48.590Z]     for task in tasks:
[task 2023-03-28T07:47:48.590Z] taskcluster\gecko_taskgraph\transforms\task.py:1559: in task_name_from_label
[task 2023-03-28T07:47:48.590Z]     for task in tasks:
[task 2023-03-28T07:47:48.590Z] taskcluster\gecko_taskgraph\transforms\task.py:1549: in setup_raptor
[task 2023-03-28T07:47:48.590Z]     for task in tasks:
[task 2023-03-28T07:47:48.590Z] taskcluster\gecko_taskgraph\transforms\task.py:1493: in set_defaults
[task 2023-03-28T07:47:48.590Z]     for task in tasks:
[task 2023-03-28T07:47:48.590Z] taskcluster\gecko_taskgraph\transforms\task.py:1469: in set_implementation
[task 2023-03-28T07:47:48.590Z]     for task in tasks:
[task 2023-03-28T07:47:48.590Z] taskcluster\gecko_taskgraph\transforms\beetmover_apt.py:39: in beetmover_apt
[task 2023-03-28T07:47:48.590Z]     gcs_sources.extend(generate_artifact_registry_gcs_sources(dep))
[task 2023-03-28T07:47:48.590Z] taskcluster\gecko_taskgraph\util\scriptworker.py:528: in generate_artifact_registry_gcs_sources
[task 2023-03-28T07:47:48.590Z]     config["paths"][repackage_deb_artifact]["destinations"][0]
[task 2023-03-28T07:47:48.590Z] E   KeyError: 'public/build/target.deb'

Comment 11

[Gabriel Bustamante [:gabriel]]

Attachment: Bug 1799769 - Fix KeyError: 'public/build/target.deb' regression that occurs when generating the full taskgraph outside of mach r?hneiva

Comment 12

[Gabriel Bustamante [:gabriel]]

Thanks [:Sylvestre]

I submitted a patch for this regression.

Comment 13

[Gabriel Bustamante [:gabriel]]

Holding off on landing the fix I had for this regression. The error should go away when the task generation in the run_perfdocs tests are mocked on Bug 1825065. Adding a band-aid for this specific error doesn't seem like the right thing to do if there's a root cause fix in the works.

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7536f4ff829b
Fix KeyError: 'public/build/target.deb' regression that occurs when generating the full taskgraph outside of mach r=hneiva

Comment 15

[Gabriel Bustamante [:gabriel]]

Ah this broke ./mach try fuzzy too, I am landing the patch I had earlier to mitigate the issue. I've been pushing the patch to try using ./mach try fuzzy, I am surprised it's breaking.

Comment 16

[Serban Stanca [:SerbanS]]

https://hg.mozilla.org/mozilla-central/rev/7536f4ff829b

Comment 17

[BugBot [:suhaib / :marco/ :calixte]]

A patch has been attached on this bug, which was already closed. Filing a separate bug will ensure better tracking. If this was not by mistake and further action is needed, please alert the appropriate party. (Or: if the patch doesn't change behavior -- e.g. landing a test case, or fixing a typo -- then feel free to disregard this message)