1747488 - Intermittent damp | Could not find report in browser output: [('tsformat', ('__start_report', '__end_report')), ('tpformat', ('__start_tp_report', '__end_tp_report'))]

Status: RESOLVED INCOMPLETE

(Testing :: Talos, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: apavel [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=362252154&repo=mozilla-beta
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/CEr_Bm0EQYu93ed__ZXxsw/runs/0/artifacts/public/logs/live_backing.log

---

[task 2021-12-23T21:04:34.226Z] 21:04:34     INFO -   '_': '/usr/local/bin/start-worker'}
[task 2021-12-23T21:04:34.226Z] 21:04:34     INFO -  TEST-INFO | started process 4656 (/home/cltbld/tasks/task_164029104946221/build/application/firefox/firefox -profile /tmp/tmphyeis8n5/profile)
[task 2021-12-23T21:04:35.015Z] 21:04:35     INFO -  PID 4656 | Inconsistency detected by ld.so: ../elf/dl-tls.c: 481: _dl_allocate_tls_init: Assertion `listp->slotinfo[cnt].gen <= GL(dl_tls_generation)' failed!
[task 2021-12-23T21:04:35.023Z] 21:04:35     INFO -  PID 4656 | Exiting due to channel error.
[task 2021-12-23T21:04:35.024Z] 21:04:35     INFO -  PID 4656 | Exiting due to channel error.
[task 2021-12-23T21:04:35.069Z] 21:04:35     INFO -  Terminating psutil.Process(pid=4656, name='GeckoMain', started='21:04:33')
[task 2021-12-23T21:04:35.069Z] 21:04:35     INFO -  TEST-INFO | 4656: exit 127
[task 2021-12-23T21:04:35.108Z] 21:04:35     INFO -  mozcrash checking /tmp/tmphyeis8n5/profile/minidumps for minidumps...
[task 2021-12-23T21:04:35.108Z] 21:04:35     INFO -  TEST-UNEXPECTED-ERROR | damp | Could not find report in browser output: [('tsformat', ('__start_report', '__end_report')), ('tpformat', ('__start_tp_report', '__end_tp_report'))]
[task 2021-12-23T21:04:35.108Z] 21:04:35    ERROR -  Traceback (most recent call last):
[task 2021-12-23T21:04:35.108Z] 21:04:35     INFO -    File "/home/cltbld/tasks/task_164029104946221/build/tests/talos/talos/run_tests.py", line 336, in run_tests
[task 2021-12-23T21:04:35.109Z] 21:04:35     INFO -      talos_results.add(mytest.runTest(browser_config, test))
[task 2021-12-23T21:04:35.109Z] 21:04:35     INFO -    File "/home/cltbld/tasks/task_164029104946221/build/tests/talos/talos/ttest.py", line 65, in runTest
[task 2021-12-23T21:04:35.109Z] 21:04:35     INFO -      return self._runTest(browser_config, test_config, setup)
[task 2021-12-23T21:04:35.109Z] 21:04:35     INFO -    File "/home/cltbld/tasks/task_164029104946221/build/tests/talos/talos/ttest.py", line 274, in _runTest
[task 2021-12-23T21:04:35.109Z] 21:04:35     INFO -      counter_management.results() if counter_management else None
[task 2021-12-23T21:04:35.110Z] 21:04:35     INFO -    File "/home/cltbld/tasks/task_164029104946221/build/tests/talos/talos/results.py", line 95, in add
[task 2021-12-23T21:04:35.110Z] 21:04:35     INFO -      global_counters=self.global_counters,
[task 2021-12-23T21:04:35.110Z] 21:04:35     INFO -    File "/home/cltbld/tasks/task_164029104946221/build/tests/talos/talos/results.py", line 346, in __init__
[task 2021-12-23T21:04:35.110Z] 21:04:35     INFO -      self.parse()
[task 2021-12-23T21:04:35.110Z] 21:04:35     INFO -    File "/home/cltbld/tasks/task_164029104946221/build/tests/talos/talos/results.py", line 373, in parse
[task 2021-12-23T21:04:35.110Z] 21:04:35     INFO -      "Could not find report in browser output: %s" % self.report_tokens
[task 2021-12-23T21:04:35.111Z] 21:04:35     INFO -    File "/home/cltbld/tasks/task_164029104946221/build/tests/talos/talos/results.py", line 357, in error
[task 2021-12-23T21:04:35.111Z] 21:04:35     INFO -      raise utils.TalosError(message)
[task 2021-12-23T21:04:35.111Z] 21:04:35     INFO -  talos.utils.TalosError: Could not find report in browser output: [('tsformat', ('__start_report', '__end_report')), ('tpformat', ('__start_tp_report', '__end_tp_report'))]
[task 2021-12-23T21:04:35.111Z] 21:04:35     INFO -  TEST-INFO took 2383782ms
[task 2021-12-23T21:04:35.111Z] 21:04:35     INFO -  SUITE-END | took 2383s
[task 2021-12-23T21:04:35.495Z] 21:04:35    ERROR - Return code: 2
[task 2021-12-23T21:04:35.495Z] 21:04:35  WARNING - setting return code to 2

Comment 1

[Intermittent Failures Robot]

1 failures in 2286 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1

Platform and build breakdown:

• linux1804-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1747488&startday=2021-12-20&endday=2021-12-26&tree=all

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit auto_nag documentation.

Comment 3

[Treeherder Bug Filer]

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=380589008&repo=autoland

Comment 4

[Intermittent Failures Robot]

1 failures in 3336 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• windows10-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1747488&startday=2022-06-06&endday=2022-06-12&tree=all

Comment 5

[Intermittent Failures Robot]

1 failures in 3312 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• windows10-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1747488&startday=2022-06-27&endday=2022-07-03&tree=all

Comment 6

[Intermittent Failures Robot]

1 failures in 3072 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform and build breakdown:

• windows10-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1747488&startday=2022-07-04&endday=2022-07-10&tree=all

Comment 7

[BugBot [:suhaib / :marco/ :calixte]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit auto_nag documentation.

Comment 8

[Treeherder Bug Filer]

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=386269171&repo=autoland

Comment 9

[Intermittent Failures Robot]

1 failures in 3057 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• windows10-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1747488&startday=2022-08-01&endday=2022-08-07&tree=all

Comment 10

[BugBot [:suhaib / :marco/ :calixte]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit auto_nag documentation.

Comment 11

[Treeherder Bug Filer]

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=391468342&repo=autoland

Comment 12

[Intermittent Failures Robot]

1 failures in 2041 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• windows10-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1747488&startday=2022-09-26&endday=2022-10-02&tree=all

Comment 13

[Intermittent Failures Robot]

2 failures in 2793 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1
• mozilla-central: 1

Platform and build breakdown:

• windows10-64-shippable-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1747488&startday=2022-10-17&endday=2022-10-23&tree=all

Comment 14

[Intermittent Failures Robot]

1 failures in 3626 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1

Platform and build breakdown:

• windows10-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1747488&startday=2022-10-31&endday=2022-11-06&tree=all

Comment 15

[BugBot [:suhaib / :marco/ :calixte]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit auto_nag documentation.

Comment 16

[Treeherder Bug Filer]

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=400734624&repo=mozilla-beta

Comment 17

[Intermittent Failures Robot]

1 failures in 2999 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1

Platform and build breakdown:

• windows10-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1747488&startday=2022-12-19&endday=2022-12-25&tree=all

Comment 18

[Intermittent Failures Robot]

1 failures in 2607 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• windows10-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1747488&startday=2023-01-02&endday=2023-01-08&tree=all

Comment 19

[Intermittent Failures Robot]

1 failures in 3717 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 1

Platform and build breakdown:

• windows10-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1747488&startday=2023-01-16&endday=2023-01-22&tree=all

Comment 20

[Intermittent Failures Robot]

1 failures in 3914 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• windows10-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1747488&startday=2023-01-23&endday=2023-01-29&tree=all

Comment 21

[Intermittent Failures Robot]

1 failures in 3766 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• windows10-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1747488&startday=2023-02-06&endday=2023-02-12&tree=all

Comment 22

[Intermittent Failures Robot]

1 failures in 3856 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1

Platform and build breakdown:

• windows10-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1747488&startday=2023-02-13&endday=2023-02-19&tree=all

Comment 23

[Daniel Holbert [:dholbert]]

So the error message here is just a sign that we crashed or aborted at a particular time, which means the log parser doesn't find some expected output in the browser output, which then produces this error message.

The crash/abort itself is the real issue, and that may be happening for different reasons in different cases.

In comment 0, the crash/abort was likely associated with this fatal assertion, near the start of the quoted log snippet:

PID 4656 | Inconsistency detected by ld.so: ../elf/dl-tls.c: 481: _dl_allocate_tls_init: Assertion `listp->slotinfo[cnt].gen <= GL(dl_tls_generation)' failed
That's a known Linux-specific glibc bug, and I've filed bug 1819763 on that specific issue.

The failures that we still have available in our Orange Factor chart are all for Windows, though, so they're a different sort of crash. They all just say TEST-INFO | 5980: exit 1 (where 5980 is the PID) just before the issue, without much of a hint about why we exited, unfortunately.

Comment 24

[BugBot [:suhaib / :marco/ :calixte]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit auto_nag documentation.

Comment 25

[Treeherder Bug Filer]

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=415277345&repo=autoland

Comment 26

[Intermittent Failures Robot]

1 failures in 4077 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• windows10-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1747488&startday=2023-05-08&endday=2023-05-14&tree=all

Comment 27

[Intermittent Failures Robot]

1 failures in 3578 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• windows10-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1747488&startday=2023-05-15&endday=2023-05-21&tree=all

Comment 28

[BugBot [:suhaib / :marco/ :calixte]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit BugBot documentation.

Comment 29

[Treeherder Bug Filer]

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=422022928&repo=autoland

Comment 30

[Intermittent Failures Robot]

1 failures in 2956 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• windows10-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1747488&startday=2023-07-03&endday=2023-07-09&tree=all

Comment 31

[Intermittent Failures Robot]

1 failures in 3309 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• windows10-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1747488&startday=2023-07-10&endday=2023-07-16&tree=all

Comment 32

[Intermittent Failures Robot]

1 failures in 3403 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• windows10-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1747488&startday=2023-07-24&endday=2023-07-30&tree=all

Comment 33

[BugBot [:suhaib / :marco/ :calixte]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit BugBot documentation.

Comment 34

[Treeherder Bug Filer]

New failure instance: https://treeherder.mozilla.org/logviewer?job_id=431056008&repo=mozilla-central

Comment 35

[Intermittent Failures Robot]

1 failures in 4018 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform and build breakdown:

• windows10-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1747488&startday=2023-10-02&endday=2023-10-08&tree=all

Comment 36

[Intermittent Failures Robot]

1 failures in 3484 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform and build breakdown:

• windows10-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1747488&startday=2023-10-16&endday=2023-10-22&tree=all

Comment 37

[Intermittent Failures Robot]

1 failures in 3288 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1

Platform and build breakdown:

• windows10-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1747488&startday=2023-11-06&endday=2023-11-12&tree=all

Comment 38

[Intermittent Failures Robot]

1 failures in 4108 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• windows10-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1747488&startday=2023-11-27&endday=2023-12-03&tree=all

Comment 39

[Intermittent Failures Robot]

1 failures in 3349 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• windows10-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1747488&startday=2023-12-18&endday=2023-12-24&tree=all

Comment 40

[Intermittent Failures Robot]

1 failures in 3217 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1

Platform and build breakdown:

• windows10-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1747488&startday=2024-01-08&endday=2024-01-14&tree=all

Comment 41

[Intermittent Failures Robot]

1 failures in 4034 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1

Platform and build breakdown:

• windows10-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1747488&startday=2024-01-22&endday=2024-01-28&tree=all

Comment 42

[Intermittent Failures Robot]

2 failures in 3952 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1
• try: 1

Platform and build breakdown:

• windows10-64-shippable-qr: 2
  • opt: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1747488&startday=2024-01-29&endday=2024-02-04&tree=all

Comment 43

[Intermittent Failures Robot]

1 failures in 3382 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1

Platform and build breakdown:

• windows10-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1747488&startday=2024-02-19&endday=2024-02-25&tree=all

Comment 44

[Intermittent Failures Robot]

1 failures in 4198 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• linux1804-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1747488&startday=2024-03-04&endday=2024-03-10&tree=all

Comment 45

[Intermittent Failures Robot]

1 failures in 3791 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• windows10-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1747488&startday=2024-03-18&endday=2024-03-24&tree=all

Comment 46

[Intermittent Failures Robot]

1 failures in 3590 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• linux1804-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1747488&startday=2024-03-25&endday=2024-03-31&tree=all

Comment 47

[BugBot [:suhaib / :marco/ :calixte]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit BugBot documentation.