1797720 - Intermittent SUMMARY: ThreadSanitizer: data race /builds/worker/workspace/obj-build/dist/include/mozilla/RefPtr.h:286:27 in get

Status: RESOLVED FIXED

(Core :: WebRTC: Audio/Video, defect)

Description

[Treeherder Bug Filer]

Filed by: archaeopteryx [at] coole-files.de
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=394351144&repo=mozilla-central
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/JjLhGZdcRCeVSPXe5-ziNg/runs/0/artifacts/public/logs/live_backing.log

---

Linux ThreadSanitizer (tsan) media (mda) tasks hit this frequently since bug 1780310 landed.

[task 2022-10-26T04:53:55.331Z] 04:53:55     INFO - TEST-PASS | dom/media/webrtc/tests/mochitests/test_peerConnection_addAudioTrackToExistingVideoStream.html | PeerConnectionWrapper (pcLocal) successfully added an ICE candidate 
[task 2022-10-26T04:53:55.332Z] 04:53:55     INFO - GECKO(11680) | [Child 11783: Socket Thread]: I/mtransport Flow[transport_0(none)]; Layer[dtls]: ****** SSL handshake completed ******
[task 2022-10-26T04:53:55.334Z] 04:53:55     INFO - GECKO(11680) | [Child 11783: Socket Thread]: I/mtransport Flow[transport_0(none)]; Layer[dtls]: Selected ALPN string: webrtc
[task 2022-10-26T04:53:55.335Z] 04:53:55     INFO - TEST-FAIL | dom/media/webrtc/tests/mochitests/test_peerConnection_addAudioTrackToExistingVideoStream.html | The author of the test has indicated that flaky timeouts are expected.  Reason: WebRTC inherently depends on timeouts 
[task 2022-10-26T04:53:55.489Z] 04:53:55     INFO - GECKO(11680) | [Child 11783: Socket Thread]: D/mtransport Created SRTP flow!
[task 2022-10-26T04:53:55.494Z] 04:53:55     INFO - GECKO(11680) | [Child 11783: Socket Thread]: I/mtransport NrIceCtx(PC:{186ec0cf-cfd7-4d00-987b-b48268923bda} 1666760032774068 (id=6442450949 url=https://example.com/tests/dom/media/webrtc/tests/moc)/STREAM(PC:{186ec0cf-cfd7-4d00-987b-b48268923bda} 1666760032774068 (id=6442450949 url=https://example.com/tests/dom/media/webrtc/tests/moc transport-id=transport_0) : parsing trickle candidate candidate:2 1 TCP 2105458943 172.17.0.4 9 typ host tcptype active
[task 2022-10-26T04:53:55.496Z] 04:53:55     INFO - GECKO(11680) | (ice/ERR) ICE(PC:{186ec0cf-cfd7-4d00-987b-b48268923bda} 1666760032774068 (id=6442450949 url=https://example.com/tests/dom/media/webrtc/tests/moc): peer (PC:{186ec0cf-cfd7-4d00-987b-b48268923bda} 1666760032774068 (id=6442450949 url=https://example.com/tests/dom/media/webrtc/tests/moc:default), stream(PC:{186ec0cf-cfd7-4d00-987b-b48268923bda} 1666760032774068 (id=6442450949 url=https://example.com/tests/dom/media/webrtc/tests/moc transport-id=transport_0 - 053a3375:21550a3e7441d323034e623ddc706bfb) tried to trickle ICE in inappropriate state 4
[task 2022-10-26T04:53:55.499Z] 04:53:55     INFO - GECKO(11680) | [Child 11783: Socket Thread]: D/mtransport Trickle candidate is redundant for stream 'PC:{186ec0cf-cfd7-4d00-987b-b48268923bda} 1666760032774068 (id=6442450949 url=https://example.com/tests/dom/media/webrtc/tests/moc transport-id=transport_0' because it is completed: candidate:2 1 TCP 2105458943 172.17.0.4 9 typ host tcptype active
[task 2022-10-26T04:53:55.500Z] 04:53:55     INFO - GECKO(11680) | [Child 11783: Socket Thread]: I/mtransport NrIceCtx(PC:{186ec0cf-cfd7-4d00-987b-b48268923bda} 1666760032774068 (id=6442450949 url=https://example.com/tests/dom/media/webrtc/tests/moc)/STREAM(PC:{186ec0cf-cfd7-4d00-987b-b48268923bda} 1666760032774068 (id=6442450949 url=https://example.com/tests/dom/media/webrtc/tests/moc transport-id=transport_0) : parsing trickle candidate candidate:3 1 TCP 2105524479 fd15:4ba5:5a2b:100a:0:242:ac11:4 9 typ host tcptype active
[task 2022-10-26T04:53:55.502Z] 04:53:55     INFO - GECKO(11680) | (ice/ERR) ICE(PC:{186ec0cf-cfd7-4d00-987b-b48268923bda} 1666760032774068 (id=6442450949 url=https://example.com/tests/dom/media/webrtc/tests/moc): peer (PC:{186ec0cf-cfd7-4d00-987b-b48268923bda} 1666760032774068 (id=6442450949 url=https://example.com/tests/dom/media/webrtc/tests/moc:default), stream(PC:{186ec0cf-cfd7-4d00-987b-b48268923bda} 1666760032774068 (id=6442450949 url=https://example.com/tests/dom/media/webrtc/tests/moc transport-id=transport_0 - 053a3375:21550a3e7441d323034e623ddc706bfb) tried to trickle ICE in inappropriate state 4
[task 2022-10-26T04:53:55.503Z] 04:53:55     INFO - GECKO(11680) | [Child 11783: Socket Thread]: D/mtransport Trickle candidate is redundant for stream 'PC:{186ec0cf-cfd7-4d00-987b-b48268923bda} 1666760032774068 (id=6442450949 url=https://example.com/tests/dom/media/webrtc/tests/moc transport-id=transport_0' because it is completed: candidate:3 1 TCP 2105524479 fd15:4ba5:5a2b:100a:0:242:ac11:4 9 typ host tcptype active
[task 2022-10-26T04:53:55.505Z] 04:53:55     INFO - GECKO(11680) | [Child 11783: Socket Thread]: I/mtransport Flow[transport_0(none)]; Layer[dtls]: Handshake would have blocked
[task 2022-10-26T04:53:55.507Z] 04:53:55     INFO - GECKO(11680) | [Child 11783: Socket Thread]: I/mtransport NrIceCtx(PC:{186ec0cf-cfd7-4d00-987b-b48268923bda} 1666760032774068 (id=6442450949 url=https://example.com/tests/dom/media/webrtc/tests/moc)/STREAM(PC:{186ec0cf-cfd7-4d00-987b-b48268923bda} 1666760032774068 (id=6442450949 url=https://example.com/tests/dom/media/webrtc/tests/moc transport-id=transport_0) : parsing trickle candidate
[task 2022-10-26T04:53:55.509Z] 04:53:55     INFO - GECKO(11680) | (ice/ERR) ICE(PC:{186ec0cf-cfd7-4d00-987b-b48268923bda} 1666760032774068 (id=6442450949 url=https://example.com/tests/dom/media/webrtc/tests/moc): peer (PC:{186ec0cf-cfd7-4d00-987b-b48268923bda} 1666760032774068 (id=6442450949 url=https://example.com/tests/dom/media/webrtc/tests/moc:default), stream(PC:{186ec0cf-cfd7-4d00-987b-b48268923bda} 1666760032774068 (id=6442450949 url=https://example.com/tests/dom/media/webrtc/tests/moc transport-id=transport_0 - 053a3375:21550a3e7441d323034e623ddc706bfb) tried to trickle ICE in inappropriate state 4
[task 2022-10-26T04:53:55.510Z] 04:53:55     INFO - GECKO(11680) | [Child 11783: Socket Thread]: D/mtransport Trickle candidate is redundant for stream 'PC:{186ec0cf-cfd7-4d00-987b-b48268923bda} 1666760032774068 (id=6442450949 url=https://example.com/tests/dom/media/webrtc/tests/moc transport-id=transport_0' because it is completed:
[task 2022-10-26T04:53:55.534Z] 04:53:55     INFO - GECKO(11680) | [Child 11783: Socket Thread]: I/mtransport Flow[transport_0(none)]; Layer[dtls]: ****** SSL handshake completed ******
[task 2022-10-26T04:53:55.534Z] 04:53:55     INFO - GECKO(11680) | [Child 11783: Socket Thread]: I/mtransport Flow[transport_0(none)]; Layer[dtls]: Selected ALPN string: webrtc
[task 2022-10-26T04:53:55.537Z] 04:53:55     INFO - GECKO(11680) | [Child 11783: Socket Thread]: D/mtransport Created SRTP flow!
[task 2022-10-26T04:53:55.539Z] 04:53:55     INFO - GECKO(11680) | [Child 11783: WebrtcWorker #2]: I/signaling [WebrtcWorker #2|WebrtcVideoSessionConduit] VideoStreamFactory.cpp:187: CreateEncoderStreams Input frame 320x240, RID  scaling to 320x240
[task 2022-10-26T04:53:55.539Z] 04:53:55     INFO - GECKO(11680) | [Child 11783: WebrtcWorker #2]: I/signaling [WebrtcWorker #2|WebrtcVideoSessionConduit] VideoStreamFactory.cpp:208: CreateEncoderStreams Stream with RID  maxFps=60 (global max fps = 60)
[task 2022-10-26T04:53:55.749Z] 04:53:55     INFO - GECKO(11680) | ==================
[task 2022-10-26T04:53:55.750Z] 04:53:55     INFO - GECKO(11680) | WARNING: ThreadSanitizer: data race (pid=11783)
[task 2022-10-26T04:53:55.750Z] 04:53:55     INFO - GECKO(11680) |   Read of size 8 at 0x7b7800026b40 by thread T36 (mutexes: write M0):
[task 2022-10-26T04:53:55.752Z] 04:53:55     INFO - GECKO(11680) |     #0 get /builds/worker/workspace/obj-build/dist/include/mozilla/RefPtr.h:286:27 (libxul.so+0x51e9a7c) (BuildId: b671e36cea7876193a893dd328b8e3c8099a4fc0)
[task 2022-10-26T04:53:55.752Z] 04:53:55     INFO - GECKO(11680) |     #1 operator-> /builds/worker/workspace/obj-build/dist/include/mozilla/RefPtr.h:316:12 (libxul.so+0x51e9a7c)
[task 2022-10-26T04:53:55.752Z] 04:53:55     INFO - GECKO(11680) |     #2 mozilla::WebrtcVideoConduit::SendVideoFrame(webrtc::VideoFrame) /builds/worker/checkouts/gecko/dom/media/webrtc/libwebrtcglue/VideoConduit.cpp:1328:9 (libxul.so+0x51e9a7c)
[task 2022-10-26T04:53:55.753Z] 04:53:55     INFO - GECKO(11680) |     #3 OnVideoFrameConverted /builds/worker/checkouts/gecko/dom/media/webrtc/transportbridge/MediaPipeline.cpp:740:11 (libxul.so+0x5291a7c) (BuildId: b671e36cea7876193a893dd328b8e3c8099a4fc0)
[task 2022-10-26T04:53:55.753Z] 04:53:55     INFO - GECKO(11680) |     #4 operator() /builds/worker/checkouts/gecko/dom/media/webrtc/transportbridge/MediaPipeline.cpp:793:21 (libxul.so+0x5291a7c)
[task 2022-10-26T04:53:55.754Z] 04:53:55     INFO - GECKO(11680) |     #5 ApplyWithArgsImpl<(lambda at /builds/worker/checkouts/gecko/dom/media/webrtc/transportbridge/MediaPipeline.cpp:792:9)> /builds/worker/checkouts/gecko/dom/media/MediaEventSource.h:214:5 (libxul.so+0x5291a7c)
[task 2022-10-26T04:53:55.755Z] 04:53:55     INFO - GECKO(11680) |     #6 mozilla::detail::ListenerImpl<mozilla::AbstractThread, mozilla::MediaPipelineTransmit::MediaPipelineTransmit(std::string const&, RefPtr<mozilla::MediaTransportHandler>, RefPtr<mozilla::AbstractThread>, RefPtr<nsISerialEventTarget>, bool, RefPtr<mozilla::MediaSessionConduit>)::$_4, webrtc::VideoFrame>::ApplyWithArgs(webrtc::VideoFrame&&) /builds/worker/checkouts/gecko/dom/media/MediaEventSource.h:236:5 (libxul.so+0x5291a7c)
[task 2022-10-26T04:53:55.761Z] 04:53:55     INFO - GECKO(11680) |     #7 applyImpl<mozilla::detail::Listener<webrtc::VideoFrame>, void (mozilla::detail::Listener<webrtc::VideoFrame>::*)(webrtc::VideoFrame &&), StoreCopyPassByRRef<webrtc::VideoFrame>, 0UL> /builds/worker/workspace/obj-build/dist/include/nsThreadUtils.h:1147:12 (libxul.so+0x52815e6) (BuildId: b671e36cea7876193a893dd328b8e3c8099a4fc0)
[task 2022-10-26T04:53:55.761Z] 04:53:55     INFO - GECKO(11680) |     #8 apply<mozilla::detail::Listener<webrtc::VideoFrame>, void (mozilla::detail::Listener<webrtc::VideoFrame>::*)(webrtc::VideoFrame &&)> /builds/worker/workspace/obj-build/dist/include/nsThreadUtils.h:1153:12 (libxul.so+0x52815e6)
[task 2022-10-26T04:53:55.762Z] 04:53:55     INFO - GECKO(11680) |     #9 mozilla::detail::RunnableMethodImpl<mozilla::detail::Listener<webrtc::VideoFrame>*, void (mozilla::detail::Listener<webrtc::VideoFrame>::*)(webrtc::VideoFrame&&), true, (mozilla::RunnableKind)0, webrtc::VideoFrame&&>::Run() /builds/worker/workspace/obj-build/dist/include/nsThreadUtils.h:1200:13 (libxul.so+0x52815e6)
[task 2022-10-26T04:53:55.763Z] 04:53:55     INFO - GECKO(11680) |     #10 mozilla::TaskQueue::Runner::Run() /builds/worker/checkouts/gecko/xpcom/threads/TaskQueue.cpp:259:20 (libxul.so+0x11ff0d2) (BuildId: b671e36cea7876193a893dd328b8e3c8099a4fc0)
[task 2022-10-26T04:53:55.763Z] 04:53:55     INFO - GECKO(11680) |     #11 nsThreadPool::Run() /builds/worker/checkouts/gecko/xpcom/threads/nsThreadPool.cpp:309:14 (libxul.so+0x121be14) (BuildId: b671e36cea7876193a893dd328b8e3c8099a4fc0)
[task 2022-10-26T04:53:55.764Z] 04:53:55     INFO - GECKO(11680) |     #12 nsThread::ProcessNextEvent(bool, bool*) /builds/worker/checkouts/gecko/xpcom/threads/nsThread.cpp:1198:16 (libxul.so+0x1212bbb) (BuildId: b671e36cea7876193a893dd328b8e3c8099a4fc0)
[task 2022-10-26T04:53:55.765Z] 04:53:55     INFO - GECKO(11680) |     #13 NS_ProcessNextEvent(nsIThread*, bool) /builds/worker/checkouts/gecko/xpcom/threads/nsThreadUtils.cpp:465:10 (libxul.so+0x1219656) (BuildId: b671e36cea7876193a893dd328b8e3c8099a4fc0)
[task 2022-10-26T04:53:55.766Z] 04:53:55     INFO - GECKO(11680) |     #14 mozilla::ipc::MessagePumpForNonMainThreads::Run(base::MessagePump::Delegate*) /builds/worker/checkouts/gecko/ipc/glue/MessagePump.cpp:300:20 (libxul.so+0x1ed908e) (BuildId: b671e36cea7876193a893dd328b8e3c8099a4fc0)
[task 2022-10-26T04:53:55.767Z] 04:53:55     INFO - GECKO(11680) |     #15 RunInternal /builds/worker/checkouts/gecko/ipc/chromium/src/base/message_loop.cc:381:10 (libxul.so+0x1df3fd7) (BuildId: b671e36cea7876193a893dd328b8e3c8099a4fc0)
[task 2022-10-26T04:53:55.767Z] 04:53:55     INFO - GECKO(11680) |     #16 RunHandler /builds/worker/checkouts/gecko/ipc/chromium/src/base/message_loop.cc:374:3 (libxul.so+0x1df3fd7)
[task 2022-10-26T04:53:55.768Z] 04:53:55     INFO - GECKO(11680) |     #17 MessageLoop::Run() /builds/worker/checkouts/gecko/ipc/chromium/src/base/message_loop.cc:356:3 (libxul.so+0x1df3fd7)
[task 2022-10-26T04:53:55.768Z] 04:53:55     INFO - GECKO(11680) |     #18 nsThread::ThreadFunc(void*) /builds/worker/checkouts/gecko/xpcom/threads/nsThread.cpp:383:10 (libxul.so+0x120dda2) (BuildId: b671e36cea7876193a893dd328b8e3c8099a4fc0)
[task 2022-10-26T04:53:55.769Z] 04:53:55     INFO - GECKO(11680) |     #19 _pt_root /builds/worker/checkouts/gecko/nsprpub/pr/src/pthreads/ptthread.c:201:5 (libnspr4.so+0x45843) (BuildId: d4812cb7d923f88a946a41a895c130e02d80bcd0)
[task 2022-10-26T04:53:55.770Z] 04:53:55     INFO - GECKO(11680) |   Previous write of size 8 at 0x7b7800026b40 by thread T30 (mutexes: write M1):
...

Comment 1

[Andrew McCreight [:mccr8]]

Stacks are similar to those in bug 1797392, though not identical.

Comment 2

[Dan Baker[:dbaker]]

Attachment: Bug 1797720 - Add DataMutex around VideoStreamFactory;r?pehrsons

Comment 3

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Landed: https://hg.mozilla.org/integration/autoland/rev/a88d85edd043ff8127171c48971684c1ecefb9cf

Backed out for causing bustages in VideoConduit.cpp: https://hg.mozilla.org/integration/autoland/rev/cb6636d245eed517ab56a96bb17025e91a57d336
Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&selectedTaskRun=eE6BdBOUSNe9iHCisF2Snw.0&resultStatus=testfailed%2Cbusted%2Cexception&revision=a88d85edd043ff8127171c48971684c1ecefb9cf
Failure log: https://treeherder.mozilla.org/logviewer?job_id=394816250&repo=autoland

dom/media/webrtc/libwebrtcglue/VideoConduit.cpp:356:7: error: field 'mRecvStreamConfig' will be initialized after field 'mVideoStreamFactory' [-Werror,-Wreorder-ctor

Comment 4

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Add DataMutex around VideoStreamFactory;r=pehrsons
https://hg.mozilla.org/integration/autoland/rev/1c151606ed1ca94364da60d2c1424ba6584e057a
https://hg.mozilla.org/mozilla-central/rev/1c151606ed1c

Comment 6

[Intermittent Failures Robot]

1 failures in 3267 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform and build breakdown:

• linux1804-64-tsan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1797720&startday=2023-09-04&endday=2023-09-10&tree=all

Comment 7

[Intermittent Failures Robot]

1 failures in 4252 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 1

Platform and build breakdown:

• linux1804-64-tsan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1797720&startday=2024-04-15&endday=2024-04-21&tree=all

Comment 8

[Intermittent Failures Robot]

3 failures in 3649 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1
• try: 1
• mozilla-central: 1

Platform and build breakdown:

• linux1804-64-tsan-qr: 3
  • opt: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1797720&startday=2024-05-06&endday=2024-05-12&tree=all

Comment 9

[Intermittent Failures Robot]

1 failures in 4547 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• linux1804-64-tsan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1797720&startday=2024-05-13&endday=2024-05-19&tree=all

Comment 10

[Intermittent Failures Robot]

1 failures in 3995 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform and build breakdown:

• linux1804-64-tsan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1797720&startday=2024-05-20&endday=2024-05-26&tree=all

Comment 11

[Intermittent Failures Robot]

1 failures in 4625 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• linux1804-64-tsan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1797720&startday=2024-06-10&endday=2024-06-16&tree=all

Comment 12

[Intermittent Failures Robot]

1 failures in 3780 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• linux1804-64-tsan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1797720&startday=2024-07-08&endday=2024-07-14&tree=all

Comment 13

[Intermittent Failures Robot]

1 failures in 3459 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1

Platform and build breakdown:

• linux1804-64-tsan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1797720&startday=2024-08-05&endday=2024-08-11&tree=all

Comment 14

[Intermittent Failures Robot]

1 failures in 3002 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 1

Platform and build breakdown:

• linux1804-64-tsan-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1797720&startday=2024-11-18&endday=2024-11-24&tree=all

Comment 15

[Intermittent Failures Robot]

Summary

1 failures in 4035 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• linux1804-64-tsan-qr: 1
  • opt: 1

Table

	no_variant
linux1804-64-tsan-qr--/opt	1

For more details, see:

https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1797720&startday=2025-01-20&endday=2025-01-26&tree=all