1893627 - central-as-beta simulation: scriptworker.exceptions.ScriptWorkerTaskException: Bouncer entries are corrupt

Status: NEW

(Release Engineering :: Release Automation: Updates, defect)

Description

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

For today's central-as-early-beta simulation, the release graphs both for Firefox and Developer Edition have been requested and each's release bouncer sub task fails with

2024-04-26 11:40:54,260 - bouncerscript.utils - DEBUG - Locations info: [{'os': 'win', 'id': '144728', 'path': '/firefox/releases/127.0b1/win32/multi/Firefox%20Setup%20127.0b1.msix'}, {'os': 'win64', 'id': '144729', 'path': '/firefox/releases/127.0b1/win64/multi/Firefox%20Setup%20127.0b1.msix'}, {'os': 'win64-aarch64', 'id': '144730', 'path': '/firefox/releases/127.0b1/win64-aarch64/multi/Firefox%20Setup%20127.0b1.msix'}]
2024-04-26 11:40:54,260 - bouncerscript.script - WARNING - Path "/firefox/releases/127.0b1/win64/multi/Firefox%20Setup%20127.0b1.msix" for product "Firefox-127.0b1-msix-SSL" already exists. Skipping...
2024-04-26 11:40:54,260 - bouncerscript.script - INFO - Sanity check to ensure locations have been successfully added...
2024-04-26 11:40:54,261 - bouncerscript.utils - INFO - Calling location_show?product=Firefox-127.0b1-msix-SSL with data: {}
2024-04-26 11:40:54,261 - bouncerscript.utils - INFO - Performing a GET request to https://dev.bounceradmin.nonprod.webservices.mozgcp.net/api/location_show?product=Firefox-127.0b1-msix-SSL with kwargs {'timeout': 60}
2024-04-26 11:40:54,358 - bouncerscript.utils - INFO - Server response: <?xml version="1.0" encoding="utf-8"?><locations><product id="34684" name="Firefox-127.0b1-msix-SSL"><location id="144728" os="win">/firefox/releases/127.0b1/win32/multi/Firefox%20Setup%20127.0b1.msix</location><location id="144729" os="win64">/firefox/releases/127.0b1/win64/multi/Firefox%20Setup%20127.0b1.msix</location><location id="144730" os="win64-aarch64">/firefox/releases/127.0b1/win64-aarch64/multi/Firefox%20Setup%20127.0b1.msix</location></product></locations>
2024-04-26 11:40:54,358 - bouncerscript.utils - DEBUG - Locations info: [{'os': 'win', 'id': '144728', 'path': '/firefox/releases/127.0b1/win32/multi/Firefox%20Setup%20127.0b1.msix'}, {'os': 'win64', 'id': '144729', 'path': '/firefox/releases/127.0b1/win64/multi/Firefox%20Setup%20127.0b1.msix'}, {'os': 'win64-aarch64', 'id': '144730', 'path': '/firefox/releases/127.0b1/win64-aarch64/multi/Firefox%20Setup%20127.0b1.msix'}]
2024-04-26 11:40:54,358 - scriptworker.client - ERROR - Failed to run async_main
Traceback (most recent call last):
  File "/app/lib/python3.9/site-packages/scriptworker/client.py", line 205, in _handle_asyncio_loop
    await async_main(context)
  File "/app/lib/python3.9/site-packages/bouncerscript/script.py", line 166, in async_main
    await action_map[context.action](context)
  File "/app/lib/python3.9/site-packages/bouncerscript/script.py", line 70, in bouncer_submission
    check_locations_match(locations_paths, pr_config["paths_per_bouncer_platform"])
  File "/app/lib/python3.9/site-packages/bouncerscript/task.py", line 126, in check_locations_match
    raise ScriptWorkerTaskException("Bouncer entries are corrupt")
scriptworker.exceptions.ScriptWorkerTaskException: Bouncer entries are corrupt
exit code: 1

Comment 1

[[e-mail] (:bhearsum)]

I think this is fallout from gbrown's testing on https://bugzilla.mozilla.org/show_bug.cgi?id=1890607. The error we're seeing comes from https://github.com/mozilla-releng/scriptworker-scripts/blob/e9909506ebf00cf5542a280a3981e2f0884da013/bouncerscript/src/bouncerscript/task.py#L122, called by https://github.com/mozilla-releng/scriptworker-scripts/blob/e9909506ebf00cf5542a280a3981e2f0884da013/bouncerscript/src/bouncerscript/script.py#L70.

The inputs to that are the logged locations:

358 - bouncerscript.utils - DEBUG - Locations info: [{'os': 'win'
 'id': '144728'
 'path': '/firefox/releases/127.0b1/win32/multi/Firefox%20Setup%20127.0b1.msix'}
 {'os': 'win64'
 'id': '144729'
 'path': '/firefox/releases/127.0b1/win64/multi/Firefox%20Setup%20127.0b1.msix'}
 {'os': 'win64-aarch64'
 'id': '144730'
 'path': '/firefox/releases/127.0b1/win64-aarch64/multi/Firefox%20Setup%20127.0b1.msix'}]

...and the paths_per_bouncer_platform in this part of the task definition:

    Firefox-127.0b1-msix-SSL:
      options:
        ssl_only: true
        add_locales: false
      paths_per_bouncer_platform:
        win: /firefox/releases/127.0b1/win32/multi/Firefox%20Setup%20127.0b1.msix
        win64: /firefox/releases/127.0b1/win64/multi/Firefox%20Setup%20127.0b1.msix

It sounds to me like this will keep failing until simulations pick up that work (which isn't landed yet).

This is somewhat reminiscent of https://github.com/mozilla-releng/scriptworker-scripts/commit/c8f756562b2917beb4ad3b43df9b969036a5cab6, where we disabled certain checks in bouncerscript to give us a better chance of success on Try, where the data is not stable, and can have collisions.

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

The severity field is not set for this bug.
:gbrown, could you have a look please?

For more information, please visit BugBot documentation.