1791138 - Intermittent sys-symbols-ubuntu [tier 2] [taskcluster:error] Task timeout after 14400 seconds. Force killing container.

Status: RESOLVED FIXED

(Toolkit :: Crash Reporting, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: imoraru [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=390637069&repo=mozilla-central
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/O0uG5g9tQ7-TmCjoCJGRTA/runs/0/artifacts/public/logs/live_backing.log

---

[task 2022-09-16T04:02:30.952Z] Writing symbol file for packages/usr/lib/firefox-trunk/libmozsandbox.so packages/usr/lib/debug/.build-id/5d/738868b66707b7af96eb12b06ba0086e5d83b2.debug ... done
[task 2022-09-16T04:02:30.997Z] ***** error log for package downloads/firefox-trunk_106.0~a1~hg20220915r635588-0ubuntu0.18.04.1~umd1_amd64.deb
[task 2022-09-16T04:02:30.998Z] 04:02:07 [ERROR] CFI: CfiError { kind: BadDebugInfo, source: Some(UnexpectedEof(ReaderOffsetId(139745351350820))) }
[task 2022-09-16T04:02:30.998Z] 04:02:07 [ERROR] CFI: CfiError { kind: BadDebugInfo, source: Some(UnexpectedEof(ReaderOffsetId(140665259107076))) }
[task 2022-09-16T04:02:30.998Z] ***** error log for package downloads/firefox-trunk_106.0~a1~hg20220915r635588-0ubuntu0.18.04.1~umd1_amd64.deb ends here
[task 2022-09-16T04:02:31.148Z] package_name = firefox-trunk version = 106.0~a1~hg20220915r635588-0ubuntu0.20.04.1~umd1_amd64 dbg_package_name = firefox-trunk
[task 2022-09-16T04:02:50.875Z] Writing symbol file for packages/usr/lib/firefox-trunk/libmozsqlite3.so packages/usr/lib/debug/.build-id/35/86677d69e7e7e5d44454c35df1c904952866f2.debug ... done
[task 2022-09-16T04:02:50.905Z] Writing symbol file for packages/usr/lib/firefox-trunk/libsmime3.so packages/usr/lib/debug/.build-id/32/c4321d3380660e741d25dc9a3659117b9ba935.debug ... done
[task 2022-09-16T04:02:50.926Z] Writing symbol file for packages/usr/lib/firefox-trunk/liblgpllibs.so packages/usr/lib/debug/.build-id/4c/def38b3f670f8631f435b5f39b99859c11b1bf.debug ... done
[task 2022-09-16T04:02:51.034Z] Writing symbol file for packages/usr/lib/firefox-trunk/libmozavcodec.so packages/usr/lib/debug/.build-id/5a/6473a1b59f836fd2762ffcf5f1691beac219c8.debug ... done
[task 2022-09-16T04:02:51.055Z] Writing symbol file for packages/usr/lib/firefox-trunk/libmozgtk.so packages/usr/lib/debug/.build-id/e0/827b9b768b7a60f07b567a8d71deff4b34f449.debug ... done
[task 2022-09-16T04:02:51.080Z] Writing symbol file for packages/usr/lib/firefox-trunk/libnssutil3.so packages/usr/lib/debug/.build-id/33/1c9aa028bfdbc64ea4c9d55f524b47eee01200.debug ... done
[task 2022-09-16T04:02:51.131Z] Writing symbol file for packages/usr/lib/firefox-trunk/libnss3.so packages/usr/lib/debug/.build-id/5c/402bd1e65335965504d8b58dc27dfbcf684b06.debug ... done
[task 2022-09-16T04:02:51.173Z] Writing symbol file for packages/usr/lib/firefox-trunk/gmp-clearkey/0.1/libclearkey.so packages/usr/lib/debug/.build-id/8b/9b18cfa0dcac9af2d737a8b6d9335e0ac62ce4.debug ... done
[task 2022-09-16T04:02:51.201Z] Writing symbol file for packages/usr/lib/firefox-trunk/libnssckbi.so packages/usr/lib/debug/.build-id/5b/f3fab6750e7a44723c49c2aa99179f60b8e538.debug ... done
[task 2022-09-16T04:02:51.221Z] Writing symbol file for packages/usr/lib/firefox-trunk/libplc4.so packages/usr/lib/debug/.build-id/1f/b49c63c336d41eae6163e8f8b2b5a4b5f2e267.debug ... done
[task 2022-09-16T04:02:51.261Z] Writing symbol file for packages/usr/lib/firefox-trunk/libssl3.so packages/usr/lib/debug/.build-id/cc/8797c0a0b121d727cdd3f5fe835bd7727c30d0.debug ... done
[task 2022-09-16T04:02:51.280Z] Writing symbol file for packages/usr/lib/firefox-trunk/libmozwayland.so packages/usr/lib/debug/.build-id/e6/5d69fc387f420c571b6f94b2a833de280a4404.debug ... done
[task 2022-09-16T04:02:51.324Z] Writing symbol file for packages/usr/lib/firefox-trunk/libfreeblpriv3.so packages/usr/lib/debug/.build-id/ea/22fbf1ec9282d3b83e6846fb0bfb5084b019f0.debug ... done
[task 2022-09-16T04:02:51.435Z] Writing symbol file for packages/usr/lib/firefox-trunk/plugin-container packages/usr/lib/debug/.build-id/16/e71b4e6855f741e81f6f245532ba480cb46f3d.debug ... done
[task 2022-09-16T04:02:51.464Z] Writing symbol file for packages/usr/lib/firefox-trunk/libmozavutil.so packages/usr/lib/debug/.build-id/05/8bc448cface4ff89015c5c4357c1a12ce42636.debug ... done
[task 2022-09-16T04:02:51.496Z] Writing symbol file for packages/usr/lib/firefox-trunk/libsoftokn3.so packages/usr/lib/debug/.build-id/43/c1cb3a69eb4a3be068216e464952416e2a8e2b.debug ... done
[task 2022-09-16T04:02:51.609Z] Writing symbol file for packages/usr/lib/firefox-trunk/firefox-trunk packages/usr/lib/debug/.build-id/e9/17dcd68bb39b8a8c946db626a16534a9ab4ef4.debug ... done
[task 2022-09-16T04:02:51.675Z] Writing symbol file for packages/usr/lib/firefox-trunk/libipcclientcerts.so packages/usr/lib/debug/.build-id/47/a5977d2e20e7f84098b2d77122a05bec0f93f7.debug ... done
[task 2022-09-16T04:02:51.709Z] Writing symbol file for packages/usr/lib/firefox-trunk/libnspr4.so packages/usr/lib/debug/.build-id/5e/1d44dff703c8cc254940a14e13cbd7397fa7b5.debug ... done
[task 2022-09-16T04:02:51.729Z] Writing symbol file for packages/usr/lib/firefox-trunk/libplds4.so packages/usr/lib/debug/.build-id/c0/e43ce4ce14b5274ff53ad5ee4e5745281c1d0f.debug ... done
[taskcluster:error] Task timeout after 14400 seconds. Force killing container.
[taskcluster 2022-09-16 04:03:10.029Z] === Task Finished ===
[taskcluster 2022-09-16 04:03:10.029Z] Unsuccessful task run with exit code: -1 completed in 14408.051 seconds

Comment 1

[Intermittent Failures Robot]

1 failures in 3023 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform and build breakdown:

• symbols: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1791138&startday=2022-09-12&endday=2022-09-18&tree=all

Comment 2

[Intermittent Failures Robot]

10 failures in 2909 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 10

Platform and build breakdown:

• symbols: 10
  • opt: 10

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1791138&startday=2022-09-19&endday=2022-09-25&tree=all

Comment 3

[BugBot [:suhaib / :marco/ :calixte]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit auto_nag documentation.

Comment 4

[Treeherder Bug Filer]

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=397528250&repo=mozilla-central

Comment 5

[Intermittent Failures Robot]

2 failures in 3888 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 2

Platform and build breakdown:

• symbols: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1791138&startday=2022-11-21&endday=2022-11-27&tree=all

Comment 6

[BugBot [:suhaib / :marco/ :calixte]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit auto_nag documentation.

Comment 7

[Treeherder Bug Filer]

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=400666826&repo=mozilla-central

Comment 8

[Intermittent Failures Robot]

2 failures in 2999 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 2

Platform and build breakdown:

• symbols: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1791138&startday=2022-12-19&endday=2022-12-25&tree=all

Comment 9

[Intermittent Failures Robot]

5 failures in 3530 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 4
• autoland: 1

Platform and build breakdown:

• symbols: 4
  • opt: 4
• linux1804-64-tsan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1791138&startday=2023-01-09&endday=2023-01-15&tree=all

Comment 10

[BugBot [:suhaib / :marco/ :calixte]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit auto_nag documentation.

Comment 11

[Treeherder Bug Filer]

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=429405624&repo=mozilla-central

Comment 12

[Intermittent Failures Robot]

2 failures in 3337 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 2

Platform and build breakdown:

• symbols: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1791138&startday=2023-09-11&endday=2023-09-17&tree=all

Comment 13

[BugBot [:suhaib / :marco/ :calixte]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit BugBot documentation.

Comment 14

[Treeherder Bug Filer]

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=436293941&repo=mozilla-central

Comment 15

[Intermittent Failures Robot]

3 failures in 3495 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 3

Platform and build breakdown:

• symbols: 3
  • opt: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1791138&startday=2023-11-13&endday=2023-11-19&tree=all

Comment 16

[BugBot [:suhaib / :marco/ :calixte]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit BugBot documentation.

Comment 17

[Treeherder Bug Filer]

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=469652377&repo=mozilla-central

Comment 18

[Gabriele Svelto [:gsvelto]]

Bizarre, there's a three hour gap between the task starting and when we first begin dumping files (see this). I wonder if it's because downloading the debug information took forever?

Comment 19

[Gabriele Svelto [:gsvelto]]

Downloading from https://ppa.launchpadcontent.net is very slow, we need to bump the maximum duration of this task because it's by far the larger of the system symbol scrapers and it's dealing with a very slow repository too.

Comment 20

[Gabriele Svelto [:gsvelto]]

Attachment: Bug 1791138 - Scraping system symbols for Ubuntu is slow and needs more time r=gerard-majax

Comment 21

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/af37905cb86a
Scraping system symbols for Ubuntu is slow and needs more time r=gerard-majax

Comment 22

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/af37905cb86a

Comment 23

[Intermittent Failures Robot]

1 failures in 3459 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform and build breakdown:

• symbols: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1791138&startday=2024-08-05&endday=2024-08-11&tree=all