1808416 - Intermittent [DO NOT USE] <bound method TestingMixin.query_symbols_url of <__main__.DesktopUnittest object at 0x7fd8ab92ef98>> failed after 5 tries!

Status: RESOLVED FIXED

(Testing :: Mozbase, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: ncsoregi [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=401254164&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Vo_7ILYIRBqDq8mBBkHJqA/runs/0/artifacts/public/logs/live_backing.log

---

[task 2023-01-03T21:35:57.886Z] 21:35:57  WARNING - Cannot open symbols url https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/K9wOrfhoSj62rWzsp-qkbA/artifacts/public/build/target.crashreporter-symbols.zip (installer url: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/K9wOrfhoSj62rWzsp-qkbA/artifacts/public/build/target.tar.bz2): HTTP Error 404: Not Found
[task 2023-01-03T21:35:57.886Z] 21:35:57     INFO - retry: attempt #1 caught HTTPError exception: HTTP Error 404: Not Found
[task 2023-01-03T21:35:57.886Z] 21:35:57     INFO - retry: Failed, sleeping 10 seconds before retrying
ModemManager[393]: <info>  ModemManager (version 1.10.0) starting in system bus...
[task 2023-01-03T21:36:07.896Z] 21:36:07     INFO - retry: Calling query_symbols_url with args: (), kwargs: {'raise_on_failure': True}, attempt #2
[task 2023-01-03T21:36:07.963Z] 21:36:07  WARNING - Cannot open symbols url https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/K9wOrfhoSj62rWzsp-qkbA/artifacts/public/build/target.crashreporter-symbols.zip (installer url: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/K9wOrfhoSj62rWzsp-qkbA/artifacts/public/build/target.tar.bz2): HTTP Error 404: Not Found
[task 2023-01-03T21:36:07.963Z] 21:36:07     INFO - retry: attempt #2 caught HTTPError exception: HTTP Error 404: Not Found
[task 2023-01-03T21:36:07.965Z] 21:36:07     INFO - retry: Failed, sleeping 20 seconds before retrying
[task 2023-01-03T21:36:27.982Z] 21:36:27     INFO - retry: Calling query_symbols_url with args: (), kwargs: {'raise_on_failure': True}, attempt #3
[task 2023-01-03T21:36:28.041Z] 21:36:28  WARNING - Cannot open symbols url https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/K9wOrfhoSj62rWzsp-qkbA/artifacts/public/build/target.crashreporter-symbols.zip (installer url: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/K9wOrfhoSj62rWzsp-qkbA/artifacts/public/build/target.tar.bz2): HTTP Error 404: Not Found
[task 2023-01-03T21:36:28.042Z] 21:36:28     INFO - retry: attempt #3 caught HTTPError exception: HTTP Error 404: Not Found
[task 2023-01-03T21:36:28.042Z] 21:36:28     INFO - retry: Failed, sleeping 40 seconds before retrying
[task 2023-01-03T21:37:08.060Z] 21:37:08     INFO - retry: Calling query_symbols_url with args: (), kwargs: {'raise_on_failure': True}, attempt #4
[task 2023-01-03T21:37:08.172Z] 21:37:08  WARNING - Cannot open symbols url https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/K9wOrfhoSj62rWzsp-qkbA/artifacts/public/build/target.crashreporter-symbols.zip (installer url: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/K9wOrfhoSj62rWzsp-qkbA/artifacts/public/build/target.tar.bz2): HTTP Error 404: Not Found
[task 2023-01-03T21:37:08.172Z] 21:37:08     INFO - retry: attempt #4 caught HTTPError exception: HTTP Error 404: Not Found
[task 2023-01-03T21:37:08.173Z] 21:37:08     INFO - retry: Failed, sleeping 80 seconds before retrying
[task 2023-01-03T21:38:28.246Z] 21:38:28     INFO - retry: Calling query_symbols_url with args: (), kwargs: {'raise_on_failure': True}, attempt #5
[task 2023-01-03T21:38:28.306Z] 21:38:28  WARNING - Cannot open symbols url https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/K9wOrfhoSj62rWzsp-qkbA/artifacts/public/build/target.crashreporter-symbols.zip (installer url: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/K9wOrfhoSj62rWzsp-qkbA/artifacts/public/build/target.tar.bz2): HTTP Error 404: Not Found
[task 2023-01-03T21:38:28.306Z] 21:38:28     INFO - retry: attempt #5 caught HTTPError exception: HTTP Error 404: Not Found
[task 2023-01-03T21:38:28.307Z] 21:38:28    ERROR - <bound method TestingMixin.query_symbols_url of <__main__.DesktopUnittest object at 0x7fd8ab92ef98>> failed after 5 tries!
[task 2023-01-03T21:38:28.307Z] 21:38:28    ERROR - Caught exception: HTTP Error 404: Not Found

Comment 1

[Natalia Csoregi [:nataliaCs]]

Geoff, can you take a look at this please?
Thank you.

Comment 2

[Geoff Brown [:gbrown]]

This is not intermittent, and this is not causing any tests to fail. All tsan tests (and maybe some others) are configured to try to download symbols from an incorrect url, that always fails, and the task continues.

Because some of the logging is now at the ERROR level, the treeherder log viewer flags it, but that is never the cause of the failed task (in comment 0, dom/media/test/test_background_video_tainted_by_capturestream.html failed, and that caused the task to fail).

Still, it's confusing and inconvenient for star-ing: I'll try to tidy this up.

Comment 3

[Geoff Brown [:gbrown]]

Attachment: Bug 1808416 - Do not check crashreporter symbols on tsan

In mozharness, setting --download-symbols=True or --download-symbols=ondemand
triggers an attempt to access the symbols url (either to download them or just
verify that they exist). asan and tsan builds only have the special "full"
symbols, which are never used by mozharness. Removing the --download-symbols
option from the command line avoids the url check entirely, which avoids
retries, warnings, and errors in the log.

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/bca3ec6d36a0
Do not check crashreporter symbols on tsan r=jmaher

Comment 5

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1791285

Comment 6

[Serban Stanca [:SerbanS]]

https://hg.mozilla.org/mozilla-central/rev/bca3ec6d36a0

Comment 7

[Intermittent Failures Robot]

1 failures in 2607 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• linux1804-64-tsan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1808416&startday=2023-01-02&endday=2023-01-08&tree=all

Comment 8

[Iulian Moraru]

Hi Geoff! Can you please take another look at this? This happened again: https://treeherder.mozilla.org/logviewer?job_id=403269084&repo=mozilla-central&lineNumber=231.
Should we file a new bug? or should we re-open this one?
Thank you!

Comment 9

[Geoff Brown [:gbrown]]

Thanks! Will follow-up in bug 1812143.

Comment 10

[Intermittent Failures Robot]

1 failures in 3914 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform and build breakdown:

• windows10-64-2004-asan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1808416&startday=2023-01-23&endday=2023-01-29&tree=all