1621195 - Intermittent GECKO(1115) | SUMMARY: ThreadSanitizer: SEGV /builds/worker/checkouts/gecko/toolkit/components/terminator/nsTerminator.cpp:203:7 in mozilla::(anonymous namespace)::RunWatchdog(void*)

Status: RESOLVED DUPLICATE of bug 1358898

(Core :: DOM: Navigation, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: nbeleuzu [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=292387812&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/QiZuFjs7SnOSjZTYPDmwQA/runs/0/artifacts/public/logs/live_backing.log

---

[task 2020-03-10T03:13:12.017Z] 03:13:12 INFO - TEST-OK | docshell/test/mochitest/test_windowedhistoryframes.html | took 3552ms
[task 2020-03-10T03:13:12.225Z] 03:13:12 INFO - TEST-START | Shutdown
[task 2020-03-10T03:13:12.226Z] 03:13:12 INFO - Passed: 294
[task 2020-03-10T03:13:12.227Z] 03:13:12 INFO - Failed: 0
[task 2020-03-10T03:13:12.228Z] 03:13:12 INFO - Todo: 8
[task 2020-03-10T03:13:12.229Z] 03:13:12 INFO - Mode: e10s
[task 2020-03-10T03:13:12.233Z] 03:13:12 INFO - Slowest: 30664ms - /tests/docshell/test/mochitest/test_bug570341.html
[task 2020-03-10T03:13:12.236Z] 03:13:12 INFO - SimpleTest FINISHED
[task 2020-03-10T03:13:12.238Z] 03:13:12 INFO - TEST-INFO | Ran 1 Loops
[task 2020-03-10T03:13:12.240Z] 03:13:12 INFO - SimpleTest FINISHED
[task 2020-03-10T03:13:14.933Z] 03:13:14 INFO - GECKO(1115) | ###!!! [Child][MessageChannel] Error: (msgtype=0x37010E,name=PContent::Msg_AccumulateChildKeyedHistograms) Channel closing: too late to send/recv, messages will be lost
[task 2020-03-10T03:13:14.935Z] 03:13:14 INFO - GECKO(1115) | ###!!! [Child][MessageChannel] Error: (msgtype=0x370112,name=PContent::Msg_RecordDiscardedData) Channel closing: too late to send/recv, messages will be lost
[task 2020-03-10T03:13:15.794Z] 03:13:15 INFO - GECKO(1115) | ###!!! [Child][MessageChannel] Error: (msgtype=0x37010E,name=PContent::Msg_AccumulateChildKeyedHistograms) Channel closing: too late to send/recv, messages will be lost
[task 2020-03-10T03:13:15.794Z] 03:13:15 INFO - GECKO(1115) | ###!!! [Child][MessageChannel] Error: (msgtype=0x370112,name=PContent::Msg_RecordDiscardedData) Channel closing: too late to send/recv, messages will be lost
[task 2020-03-10T03:13:28.140Z] 03:13:28 INFO - GECKO(1115) | console.error: (new Error("SessionFile is closed", "resource:///modules/sessionstore/SessionFile.jsm", 433))
[task 2020-03-10T03:19:17.173Z] 03:19:17 INFO - GECKO(1115) | ThreadSanitizer:DEADLYSIGNAL
[task 2020-03-10T03:19:17.173Z] 03:19:17 INFO - GECKO(1115) | ==1115==ERROR: ThreadSanitizer: SEGV on unknown address 0x000000000000 (pc 0x7fd0b5cda2d2 bp 0x00000000016b sp 0x7fcff04fb270 T1350)
[task 2020-03-10T03:19:17.173Z] 03:19:17 INFO - GECKO(1115) | ==1115==The signal is caused by a WRITE memory access.
[task 2020-03-10T03:19:17.174Z] 03:19:17 INFO - GECKO(1115) | ==1115==Hint: address points to the zero page.
[task 2020-03-10T03:19:17.210Z] 03:19:17 INFO - GECKO(1115) | #0 mozilla::(anonymous namespace)::RunWatchdog(void*) /builds/worker/checkouts/gecko/toolkit/components/terminator/nsTerminator.cpp:203:7 (libxul.so+0x63df2d1)
[task 2020-03-10T03:19:17.211Z] 03:19:17 INFO - GECKO(1115) | #1 _pt_root /builds/worker/checkouts/gecko/nsprpub/pr/src/pthreads/ptthread.c:201:5 (libnspr4.so+0x46a4f)
[task 2020-03-10T03:19:17.212Z] 03:19:17 INFO - GECKO(1115) | #2 __tsan_thread_start_func /builds/worker/fetches/llvm-project/llvm/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:957:15 (firefox+0x5710c)
[task 2020-03-10T03:19:17.213Z] 03:19:17 INFO - GECKO(1115) | #3 start_thread <null> (libpthread.so.0+0x76b9)
[task 2020-03-10T03:19:17.214Z] 03:19:17 INFO - GECKO(1115) | #4 clone /build/glibc-LK5gWL/glibc-2.23/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:109 (libc.so.6+0x10741c)
[task 2020-03-10T03:19:17.214Z] 03:19:17 INFO - GECKO(1115) | ThreadSanitizer can not provide additional info.
[task 2020-03-10T03:19:17.215Z] 03:19:17 INFO - GECKO(1115) | SUMMARY: ThreadSanitizer: SEGV /builds/worker/checkouts/gecko/toolkit/components/terminator/nsTerminator.cpp:203:7 in mozilla::(anonymous namespace)::RunWatchdog(void*)
[task 2020-03-10T03:19:17.215Z] 03:19:17 INFO - GECKO(1115) | ==1115==ABORTING
[task 2020-03-10T03:25:27.212Z] 03:25:27 INFO - Buffered messages finished
[task 2020-03-10T03:25:27.213Z] 03:25:27 ERROR - TEST-UNEXPECTED-TIMEOUT | Last test finished | application timed out after 370 seconds with no output
[task 2020-03-10T03:25:27.214Z] 03:25:27 ERROR - Force-terminating active process(es).

Comment 1

[Daniel Varga [:dvarga]]

Started retriggers: https://treeherder.mozilla.org/#/jobs?repo=autoland&searchStr=linux64-tsan%2Copt%2Cmochitests%2Ctest-linux64-tsan%2Fopt-mochitest-plain-e10s-2%2Cm%282%29&tochange=2a28c94f3db2355dd50815d70f2ad6966839326c&fromchange=9448002ebec9e44bbb16c7d71760125093f0c3d0&selectedJob=292608454

Somewhere between: https://treeherder.mozilla.org/#/jobs?repo=autoland&searchStr=linux64-tsan%2Copt%2Cmochitests%2Ctest-linux64-tsan%2Fopt-mochitest-plain-e10s-2%2Cm%282%29&tochange=99d6961f2d92932a5f48d5f12489a0fda60d0e73&fromchange=693b8a034a9b60684008e81152411d023aa89afb&selectedJob=292619108

Comment 2

[Intermittent Failures Robot]

17 failures in 1083 pushes (0.016 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 17

Platform breakdown:

• linux64-tsan: 17

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1621195&startday=2020-03-10&endday=2020-03-10&tree=all

Comment 3

[Intermittent Failures Robot]

46 failures in 5336 pushes (0.009 failures/push) were associated with this bug in the last 7 days.

This is the #36 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 43
• try: 1
• mozilla-central: 2

Platform breakdown:

• linux64-tsan: 46

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1621195&startday=2020-03-09&endday=2020-03-15&tree=all

Comment 4

[Andreea Pavel [:apavel]]

Christian can you please take a look? This seems to have showed up when you landed the fix in bug 1614294.

Comment 5

[Christian Holler (:decoder)]

(In reply to Andreea Pavel [:apavel] from comment #4)

Christian can you please take a look? This seems to have showed up when you landed the fix in bug 1614294.

As far as I can tell, this is the same bug. This is not caused by 1614294 at all.

If the new shutdown timeout doesn't help, then I would assume that the browser does not shutdown for some reason and really hangs. This seems to happen on all platforms, not just TSan (just on TSan, it is way more frequent, which could be due to the build slowness/timing).

Comment 6

[Andreea Pavel [:apavel]]

Christian are you still working on this, or do you know who can take a look at this?

Comment 7

[Christian Holler (:decoder)]

(In reply to Andreea Pavel [:apavel] from comment #6)

Christian are you still working on this, or do you know who can take a look at this?

I don't know who could take a look, but it would help to consolidate this bug with the other shutdown hangs. The fact that TSan has a different message here does not mean that this issue is TSan specific at all. As mentioned in the other bug, this is likely a shutdown hang. The fact that it shows up more often on TSan might be due to timing, but we've seen this on ASan too, and even on regular builds. Treeherder is unlikely to link all of these bugs together though because they show up slightly different on each platform (including the timeout, which for TSan is now 360 seconds, for ASan it should be 180 seconds and for all other platforms it is 60 seconds (all +10 secs I believe for the watchdog message).

Comment 8

[Andreea Pavel [:apavel]]

Just to be clear, by consolidating with other shutdown bugs, you mean with bug 1358898 ?

Comment 9

[Christian Holler (:decoder)]

(In reply to Andreea Pavel [:apavel] from comment #8)

Just to be clear, by consolidating with other shutdown bugs, you mean with bug 1358898 ?

Yes, that sounds like the right meta bug.

Comment 11

[Intermittent Failures Robot]

31 failures in 4955 pushes (0.006 failures/push) were associated with this bug in the last 7 days.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 22
• mozilla-central: 5
• try: 3
• mozilla-beta: 1

Platform breakdown:

• linux64-tsan: 30
• linux1804-64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1621195&startday=2020-03-16&endday=2020-03-22&tree=all