1323880 - Intermittent browser/base/content/test/webrtc/browser_devices_get_user_media_screen.js | expected notification recording-window-ended - Got undefined, expected 1 and five more

Status: RESOLVED FIXED

(Firefox :: Site Permissions, defect)

Description

[Phil Ringnalda (:philor)]

Pretty frequent on Win8 PGO, which usually means that it fails when the browser is fast and it'll fail some on Win7 and OS X too.

https://treeherder.mozilla.org/logviewer.html#?job_id=40662105&repo=mozilla-inbound

15:05:12     INFO - TEST-UNEXPECTED-FAIL | browser/base/content/test/webrtc/browser_devices_get_user_media_screen.js | expected notification recording-window-ended - Got undefined, expected 1
15:05:12     INFO - Stack trace:
15:05:12     INFO - chrome://mochikit/content/browser-test.js:test_is:913
15:05:12     INFO - chrome://mochitests/content/browser/browser/base/content/test/webrtc/head.js:listener:212
15:05:12     INFO - Not taking screenshot here: see the one that was previously logged
15:05:12     INFO - TEST-UNEXPECTED-FAIL | browser/base/content/test/webrtc/browser_devices_get_user_media_screen.js | recording-device-events notification unexpected - Got -1, expected 0
15:05:12     INFO - Stack trace:
15:05:12     INFO - chrome://mochikit/content/browser-test.js:test_is:913
15:05:12     INFO - chrome://mochitests/content/browser/browser/base/content/test/webrtc/head.js:listener:237
15:05:12     INFO - Not taking screenshot here: see the one that was previously logged
15:05:12     INFO - TEST-UNEXPECTED-FAIL | browser/base/content/test/webrtc/browser_devices_get_user_media_screen.js | expected nothing to be shared - {"screen":"Screen"} deepEqual {} - JS frame :: chrome://mochitests/content/browser/browser/base/content/test/webrtc/head.js :: checkNotSharing :: line 488
15:05:12     INFO - Stack trace:
15:05:12     INFO -     chrome://mochitests/content/browser/browser/base/content/test/webrtc/head.js:checkNotSharing:488
15:05:12     INFO -     process@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:932:23
15:05:12     INFO -     walkerLoop@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:813:7
15:05:12     INFO -     Promise*scheduleWalkerLoop@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:744:11
15:05:12     INFO -     schedulePromise@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:776:7
15:05:12     INFO -     completePromise@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:711:7
15:05:12     INFO -     listener@chrome://mochitests/content/browser/browser/base/content/test/webrtc/head.js:240:7
15:05:12     INFO - Not taking screenshot here: see the one that was previously logged
15:05:12     INFO - TEST-UNEXPECTED-FAIL | browser/base/content/test/webrtc/browser_devices_get_user_media_screen.js | no sharing indicator on the control center icon - 
15:05:12     INFO - Stack trace:
15:05:12     INFO -     chrome://mochitests/content/browser/browser/base/content/test/webrtc/head.js:checkNotSharing:491
15:05:12     INFO -     process@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:932:23
15:05:12     INFO -     walkerLoop@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:813:7
15:05:12     INFO -     Promise*scheduleWalkerLoop@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:744:11
15:05:12     INFO -     schedulePromise@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:776:7
15:05:12     INFO -     completePromise@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:711:7
15:05:12     INFO -     listener@chrome://mochitests/content/browser/browser/base/content/test/webrtc/head.js:240:7
15:05:12     INFO - waiting for the global indicator to be hidden
15:05:12     INFO - TEST DEVICES: No test devices found (in media.{audio,video}_loopback_dev, using fake streams.
15:05:12     INFO - TEST-PASS | browser/base/content/test/webrtc/browser_devices_get_user_media_screen.js | WebRTC indicator hidden - 
15:05:12     INFO - TEST-PASS | browser/base/content/test/webrtc/browser_devices_get_user_media_screen.js | camera global indicator as expected - 
15:05:12     INFO - TEST-PASS | browser/base/content/test/webrtc/browser_devices_get_user_media_screen.js | microphone global indicator as expected - 
15:05:12     INFO - TEST-PASS | browser/base/content/test/webrtc/browser_devices_get_user_media_screen.js | screen global indicator as expected - 
15:05:12     INFO - TEST-PASS | browser/base/content/test/webrtc/browser_devices_get_user_media_screen.js | WebRTC menu should be hidden - 
15:05:12     INFO - TEST-PASS | browser/base/content/test/webrtc/browser_devices_get_user_media_screen.js | popup WebRTC indicator hidden - 
15:05:12     INFO - Not taking screenshot here: see the one that was previously logged
15:05:12     INFO - TEST-UNEXPECTED-FAIL | browser/base/content/test/webrtc/browser_devices_get_user_media_screen.js | recording-window-ended notification unexpected - Got 1, expected 0
15:05:12     INFO - Stack trace:
15:05:12     INFO - chrome://mochikit/content/browser-test.js:test_is:913
15:05:12     INFO - chrome://mochitests/content/browser/browser/base/content/test/webrtc/head.js:listener:237
15:05:12     INFO - Not taking screenshot here: see the one that was previously logged
15:05:12     INFO - TEST-UNEXPECTED-FAIL | browser/base/content/test/webrtc/browser_devices_get_user_media_screen.js | recording-device-events notification unexpected - Got 1, expected 0
15:05:12     INFO - Stack trace:
15:05:12     INFO - chrome://mochikit/content/browser-test.js:test_is:913
15:05:12     INFO - chrome://mochitests/content/browser/browser/base/content/test/webrtc/head.js:listener:237

Comment 1

[Intermittent Failures Robot]

20 failures in 526 pushes (0.038 failures/push) were associated with this bug in the last 7 days.  

Repository breakdown:
* mozilla-inbound: 16
* mozilla-central: 2
* autoland: 2

Platform breakdown:
* windows8-64: 17
* linux32: 2
* windows7-32-vm: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1323880&startday=2016-12-12&endday=2016-12-18&tree=all

Comment 2

[Florian Quèze [:florian]]

(In reply to OrangeFactor Robot from comment #1)

> Platform breakdown:
> * windows8-64: 17
> * linux32: 2
> * windows7-32-vm: 1

The 2 linux32 failures here have a different error message ("recording-device-events notification unexpected - Got 1, expected 0" or "expected notification recording-device-events - Got 2, expected 1") and are actually bug 1320994.

Comment 3

[Florian Quèze [:florian]]

(In reply to Florian Quèze [:florian] [:flo] from comment #2)
> (In reply to OrangeFactor Robot from comment #1)
> 
> > Platform breakdown:
> > * windows8-64: 17
> > * linux32: 2
> > * windows7-32-vm: 1
> 
> The 2 linux32 failures here have a different error message
> ("recording-device-events notification unexpected - Got 1, expected 0" or
> "expected notification recording-device-events - Got 2, expected 1") and are
> actually bug 1320994.

Well, I see the more specific bug 1324303 and bug 1324261 have been filed for these 2 specific failures.

Comment 4

[Intermittent Failures Robot]

38 failures in 609 pushes (0.062 failures/push) were associated with this bug in the last 7 days. 

This is the #16 most frequent failure this week. 

Repository breakdown:
* mozilla-inbound: 19
* autoland: 11
* mozilla-central: 7
* graphics: 1

Platform breakdown:
* windows8-64: 31
* windows7-32-vm: 6
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1323880&startday=2016-12-19&endday=2016-12-25&tree=all

Comment 5

[Intermittent Failures Robot]

23 failures in 305 pushes (0.075 failures/push) were associated with this bug in the last 7 days. 

This is the #29 most frequent failure this week. 

Repository breakdown:
* mozilla-inbound: 10
* autoland: 10
* try: 1
* mozilla-central: 1
* graphics: 1

Platform breakdown:
* windows8-64: 19
* windows7-32-vm: 2
* linux32: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1323880&startday=2016-12-26&endday=2017-01-01&tree=all

Comment 6

[Intermittent Failures Robot]

27 failures in 563 pushes (0.048 failures/push) were associated with this bug in the last 7 days.  

Repository breakdown:
* autoland: 10
* mozilla-inbound: 9
* mozilla-central: 7
* graphics: 1

Platform breakdown:
* windows8-64: 25
* linux32: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1323880&startday=2017-01-02&endday=2017-01-08&tree=all

Comment 7

[Intermittent Failures Robot]

42 failures in 722 pushes (0.058 failures/push) were associated with this bug in the last 7 days. 

This is the #49 most frequent failure this week. 

Repository breakdown:
* autoland: 16
* mozilla-central: 13
* mozilla-inbound: 12
* graphics: 1

Platform breakdown:
* windows8-64: 40
* windows7-32-vm: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1323880&startday=2017-01-09&endday=2017-01-15&tree=all

Comment 8

[Intermittent Failures Robot]

18 failures in 690 pushes (0.026 failures/push) were associated with this bug in the last 7 days.  

Repository breakdown:
* mozilla-inbound: 8
* autoland: 4
* mozilla-central: 3
* graphics: 2
* try: 1

Platform breakdown:
* windows8-64: 15
* windows7-32-vm: 3

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1323880&startday=2017-01-16&endday=2017-01-22&tree=all

Comment 9

[Intermittent Failures Robot]

33 failures in 749 pushes (0.044 failures/push) were associated with this bug in the last 7 days.  

Repository breakdown:
* mozilla-inbound: 17
* autoland: 9
* mozilla-central: 4
* mozilla-aurora: 2
* try: 1

Platform breakdown:
* windows8-64: 29
* windows7-32-vm: 2
* osx-10-10: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1323880&startday=2017-01-23&endday=2017-01-29&tree=all

Comment 10

[Joel Maher ( :jmaher ) (UTC -8)]

:flo, I see a lot of win8 errors here as well as a good amount of linux32 errors in bug 1324303 and bug 1324261.

If you add those up, this test is trending on our top 10 list of intermittent failures- by itself, this bug is the only one to cross the >=30 failures/week threshold where we will investigate and work towards resolving/disabling the test case.

Can you take a look at this test case and try to solve these issues?  Do let me know if you could work better with more information and I can try to get it for you!

Comment 11

[Florian Quèze [:florian]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=0c38a8dab6ff

Comment 12

[Florian Quèze [:florian]]

(In reply to Joel Maher ( :jmaher) from comment #10)
> :flo, I see a lot of win8 errors here as well as a good amount of linux32
> errors in bug 1324303 and bug 1324261.
> 
> If you add those up, this test is trending on our top 10 list of
> intermittent failures- by itself, this bug is the only one to cross the >=30
> failures/week threshold where we will investigate and work towards
> resolving/disabling the test case.
> 
> Can you take a look at this test case and try to solve these issues?

So the failure here is with undefined as the result. I think that's because we are not waiting long enough.

I don't know why the order of the recording-device-events and recording-window-ended notifications isn't deterministic on Windows 8, but I don't think the order is very important, so the patch I just pushed to try waits for both.


bug 1324303 and bug 1324261 are the recording-device-events notifications count being non deterministic, that's actually bug 1320994, and is the reason why the test is currently disabled on Linux opt.


Additionally, on brasstacks I see two OS X failures associated with this bug, but they are a different issue:

TEST-UNEXPECTED-FAIL | browser/base/content/test/webrtc/browser_devices_get_user_media_screen.js | NotFoundError, likely because there's no other application running. - Got error: OverconstrainedError: Constraints could be not satisfied., expected error: NotFoundError: The object can not be found here.

We could make the test hide that if needed, but really it's a platform bug that needs fixing.

Comment 13

[Florian Quèze [:florian]]

Attachment: Wait for the recording-window-ended notification

Comment 14

[Panos Astithas (he/him) [:past] (please ni?)]

Comment on attachment 8832634 [details] [diff] [review]
Wait for the recording-window-ended notification

Review of attachment 8832634 [details] [diff] [review]:
-----------------------------------------------------------------

LGTM

Comment 15

[Intermittent Failures Robot]

32 failures in 733 pushes (0.044 failures/push) were associated with this bug in the last 7 days.  

Repository breakdown:
* autoland: 13
* mozilla-inbound: 12
* mozilla-central: 6
* graphics: 1

Platform breakdown:
* windows8-64: 26
* windows7-32-vm: 4
* osx-10-10: 1
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1323880&startday=2017-01-30&endday=2017-02-05&tree=all

Comment 16

[Florian Quèze [:florian]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/b7f8236613d805aafb25c083b0af3efaa5fdf6aa
Bug 1323880 - Wait for the recording-window-ended notification instead of expecting it to happen before recording-device-events when reloading the page, r=past.

Comment 17

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/b7f8236613d8

Comment 18

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/61fac7c3d346

Comment 19

[Intermittent Failures Robot]

5 failures in 836 pushes (0.006 failures/push) were associated with this bug in the last 7 days.  
Repository breakdown:
* mozilla-inbound: 2
* mozilla-central: 2
* autoland: 1

Platform breakdown:
* windows8-64: 4
* windows7-32-vm: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1323880&startday=2017-02-06&endday=2017-02-12&tree=all