1729367 - CamerasParent causing failures in getUserMedia related tests on Windows 10 x86 WebRender

Status: RESOLVED FIXED

(Core :: WebRTC, defect, P2)

Description

[Nico Grunbaum [:ng, @chew:mozilla.org]]

[task 2021-09-03T22:19:40.306Z] 22:19:40 INFO - PID 3960 | WARNING: At least one completion condition is taking too long to complete. Conditions: [{"name":"CamerasParent 1","state":"(none)","filename":"/builds/worker/checkouts/gecko/dom/media/systemservices/CamerasParent.cpp","lineNumber":1105,"stack":["resource://gre/modules/nsAsyncShutdown.jsm:addBlocker:162"]}] Barrier: profile-before-change

https://treeherder.mozilla.org/jobs?repo=try&author=mfroman%40mozilla.com&selectedTaskRun=RJhd1-SZRuavtxG1Fhxfbw.0

https://treeherder.mozilla.org/logviewer?job_id=350494858&repo=try&lineNumber=7439

These look potentially related as well:
https://treeherder.mozilla.org/jobs?repo=try&author=mfroman%40mozilla.com&selectedTaskRun=d4egtGeNT5iM1w7MhorhSQ.0
https://treeherder.mozilla.org/logviewer?job_id=350495332&repo=try&lineNumber=8733

Could the sharing failures here be caused by CamerasParent locking up in an earlier test?
https://treeherder.mozilla.org/jobs?repo=try&author=mfroman%40mozilla.com&selectedTaskRun=WOtg5xvIQ1SE_w_S5raEdg.0

Comment 1

[Andreas Pehrson [:pehrsons]]

(In reply to Nico Grunbaum [:ng, @chew:mozilla.org] from comment #0)

Could the sharing failures here be caused by CamerasParent locking up in an earlier test?
https://treeherder.mozilla.org/jobs?repo=try&author=mfroman%40mozilla.com&selectedTaskRun=WOtg5xvIQ1SE_w_S5raEdg.0

Yes, and in particular an earlier screen sharing test. I suspect it's related to bug 1729362, even though there are some runs in debug where we hit this rather than the assert in bug 1729362. Could be a fluke.

Comment 2

[Intermittent Failures Robot]

15 failures in 3626 pushes (0.004 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• elm: 15

Platform and build breakdown:

• windows10-64-2004-qr: 8
  • debug: 8
• windows10-32-2004-qr: 3
  • debug: 3
• windows10-64-2004-asan-qr: 4
  • opt: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1729367&startday=2021-10-04&endday=2021-10-10&tree=all

Comment 3

[Intermittent Failures Robot]

24 failures in 3315 pushes (0.007 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• elm: 24

Platform and build breakdown:

• windows10-64-2004-qr: 12
  • debug: 12
• windows10-64-2004-asan-qr: 6
  • opt: 6
• windows10-32-2004-qr: 6
  • debug: 6

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1729367&startday=2021-10-11&endday=2021-10-17&tree=all

Comment 4

[Intermittent Failures Robot]

8 failures in 3186 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• elm: 8

Platform and build breakdown:

• windows10-32-2004-qr: 2
  • debug: 2
• windows10-64-2004-asan-qr: 2
  • opt: 2
• windows10-64-2004-qr: 4
  • debug: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1729367&startday=2021-10-18&endday=2021-10-24&tree=all

Comment 5

[Nico Grunbaum [:ng, @chew:mozilla.org]]

Attachment: Bug 1729367 - P0 - Temp revert of Windows pid capture to allow for clean cherry-pick;r?mjf

Comment 6

[Nico Grunbaum [:ng, @chew:mozilla.org]]

Attachment: Bug 1729367 - P1 - ws fixups to match upstream to cleanly cherrypick;r?mjf

Depends on D129710

Comment 7

[Nico Grunbaum [:ng, @chew:mozilla.org]]

Attachment: Bug 1729367 - P2 - Cherrypick OpenOffice capture fix for underlayment;r?mjf

This will allow the following cherry-pick to apply more easily, and it nets us a fix for the inability to capture OpenOffice on Win

Cherry-pick of 439ffe462a66ad9fa9a251b265e4ab28c2647d25
See review https://webrtc-review.googlesource.com/c/src/+/171723

Depends on D129711

Comment 8

[Nico Grunbaum [:ng, @chew:mozilla.org]]

Attachment: Bug 1729367 - P3 - Cherry-pick win window enumeration deadlock fix PT1;r?mjf

This is the first of two upstream patches which address a window enumeration deadlock when querying window titles of the windows owned by the querying process.
Neither truely fixes the issue.

Cherry-pick of 449a78b1e20ea85b11f967cf3a184ee610ce21c3
See review: https://webrtc-review.googlesource.com/c/src/+/195365

Depends on D129712

Comment 9

[Nico Grunbaum [:ng, @chew:mozilla.org]]

Attachment: Bug 1729367 - P6 - Restore PID recording post cherry-pick;r?mjf

This restores the code from P0, which was removed to make cherry-picking 439ffe462a66ad9fa9a251b265e4ab28c2647d25 and 449a78b1e20ea85b11f967cf3a184ee610ce21c3 easier.

Depends on D129713

Comment 10

[Nico Grunbaum [:ng, @chew:mozilla.org]]

Attachment: Bug 1729367 - P7 - restore mac PID tracking using new API;r?mjf

Depends on D129714

Comment 11

[Michael Froman [:mjf]]

Attachment: Bug 1729367 - P4 - Cherry-pick win window enumeration deadlock fix PT2. r?ng!

This is a altered version (to account for file name changes) of
upstream's 78c73477c75f3262cad837ce7b46adf284166b14.

Comment 12

[Michael Froman [:mjf]]

Attachment: Bug 1729367 - P5 - Change line endings back to LF. r?ng!

Depends on D129748

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/mozilla-central/rev/f294d3c331ad
P0 - Temp revert of Windows pid capture to allow for clean cherry-pick;r=mjf
https://hg.mozilla.org/mozilla-central/rev/f08497b4423e
P1 - ws fixups to match upstream to cleanly cherrypick;r=mjf
https://hg.mozilla.org/mozilla-central/rev/f909dc428e52
P2 - Cherrypick OpenOffice capture fix for underlayment;r=mjf
https://hg.mozilla.org/mozilla-central/rev/30b766eaa3a4
P3 - Cherry-pick win window enumeration deadlock fix PT1;r=mjf
https://hg.mozilla.org/mozilla-central/rev/9af9cf9696cb
P4 - Cherry-pick win window enumeration deadlock fix PT2. r=ng
https://hg.mozilla.org/mozilla-central/rev/d0bb0a3af9cc
P5 - Change line endings back to LF. r=ng
https://hg.mozilla.org/mozilla-central/rev/72a83cb2e571
P6 - Restore PID recording post cherry-pick;r=mjf
https://hg.mozilla.org/mozilla-central/rev/1495ca5ef535
P7 - restore mac PID tracking using new API;r=mjf a=webrtc-update

Comment 14

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/f294d3c331ad
https://hg.mozilla.org/mozilla-central/rev/f08497b4423e
https://hg.mozilla.org/mozilla-central/rev/f909dc428e52
https://hg.mozilla.org/mozilla-central/rev/30b766eaa3a4
https://hg.mozilla.org/mozilla-central/rev/9af9cf9696cb
https://hg.mozilla.org/mozilla-central/rev/d0bb0a3af9cc
https://hg.mozilla.org/mozilla-central/rev/72a83cb2e571
https://hg.mozilla.org/mozilla-central/rev/1495ca5ef535