osx 10.10.5 error for test file browser/components/loop/test/shared/test_shared_all.py

RESOLVED FIXED in Firefox 44

Status

Hello (Loop)
General
P2
normal
Rank:
28
RESOLVED FIXED
3 years ago
3 years ago

People

(Reporter: jmaher, Assigned: standard8)

Tracking

unspecified
mozilla44
Points:
5

Firefox Tracking Flags

(firefox44 fixed)

Details

Attachments

(1 attachment)

(Reporter)

Description

3 years ago
we are close to getting osx 10.10.5 up and running, unfortunately the Mn job fails on debug and we need to get it fixed.

here is a clip from the log:
09:55:09     INFO -  TEST-START | test_shared_all.py TestSharedUnits.test_units
09:55:30    ERROR -  TEST-UNEXPECTED-FAIL | test_shared_all.py TestSharedUnits.test_units | AssertionError: build/tests/marionette/tests/browser/components/loop/test/shared/index.html: 1 failure(s) encountered:
09:55:30    ERROR -  TEST-UNEXPECTED-FAIL | index.html | should long only the warnings we expect - AssertionError: expected 2 to deeply equal 0
09:55:30     INFO -  AssertionError: expected 2 to deeply equal 0
09:55:30     INFO -  Traceback (most recent call last):
09:55:30     INFO -    File "/builds/slave/test/build/venv/lib/python2.7/site-packages/marionette/marionette_test.py", line 296, in run
09:55:30     INFO -      testMethod()
09:55:30     INFO -    File "/builds/slave/test/build/tests/marionette/tests/browser/components/loop/test/shared/test_shared_all.py", line 16, in test_units
09:55:30     INFO -      self.check_page("index.html")
09:55:30     INFO -    File "/builds/slave/test/build/tests/marionette/tests/browser/components/loop/test/shared/frontend_tester.py", line 128, in check_page
09:55:30     INFO -      raise AssertionError(self.get_failure_details(page))
09:55:30     INFO -  TEST-INFO took 20530ms

there is no screenshots available (might be harness specific), or other info in the debug log.
(Reporter)

Comment 1

3 years ago
:standard8, can you look at this and help us figure out if we can fix the test, the os configuration, or disable the test in the manifest?
Flags: needinfo?(standard8)
(Assignee)

Comment 2

3 years ago
Ok, I've just done a try push to attempt to add some more debug info. If it works, its likely something we'll want to add to the test setup anyway (or modify slightly) so useful to see:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=cf2f915db9e4
(Assignee)

Comment 3

3 years ago
Ok, the try run failed, but I don't understand why. The patch I want to try is:

https://hg.mozilla.org/try/rev/9e9b0cb8a2d0

Joel, can you help here please?
Flags: needinfo?(standard8) → needinfo?(jmaher)
(Reporter)

Comment 4

3 years ago
this looks like infra structure errors or build errors, not related to the test.  any chance the parent revision you pushed with was bad?
Flags: needinfo?(jmaher)
(Assignee)

Comment 5

3 years ago
Ah, yes it was a bad one - forgot to look. I've pushed afresh:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=4be066010010
(Reporter)

Comment 6

3 years ago
pushed to try with updated revisions:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=f69624997f77

lets see if 3rd time is the charm.
(Assignee)

Comment 8

3 years ago
Ok, I've pushed something that should get us a screenshot url out, not perfect, but it'll do for now:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=9dc7c75ec206
(Assignee)

Comment 9

3 years ago
I forgot that a screenshot isn't going to solve this issue, since primarily these are warnings that go to the console. I've found a possible change to the test that might now give us more information and pushed again:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=c83192b2ae9e
(Assignee)

Comment 10

3 years ago
Ok, we now have something reasonable as an output:

TEST-UNEXPECTED-FAIL | index.html | should long only the warnings we expect - Error: No stores registered for event type ,connectionFailure,No stores registered for event type ,connectionFailure

That's a bit weird though as these failures appear to be part of the test code, and we don't see them normally. Could be timing, but its strange that afaik we haven't seen this normally.

I'll dig into it a bit today.
(Assignee)

Comment 11

3 years ago
Now with a stack output to see who's trying to dispatch the connectionFailure action:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=334ff430d321
(Assignee)

Comment 12

3 years ago
Green Mn: https://treeherder.mozilla.org/#/jobs?repo=try&revision=572aab8ad03b

So either the builder probably doesn't have any mic/camera devices set up whereas our other boxes have at least one *Or* there's an async timing issue that's revealed on this machine.

I'll tidy the patch up tomorrow and get it ready for review.
Assignee: nobody → standard8
Iteration: --- → 44.3 - Nov 2
Points: --- → 5
Rank: 28
Priority: -- → P2
(Assignee)

Comment 13

3 years ago
Created attachment 8678068 [details] [diff] [review]
Fix an issue with Loop's unit tests failing when no devices are installed, due to bad stubbing.

This fixes the stubbing for hasAudioOrVideoDevices when doing the join room tests, so that hasAudioOrVideoDevices never actually gets called.
Attachment #8678068 - Flags: review?(mdeboer)
(Assignee)

Comment 14

3 years ago
Comment on attachment 8678068 [details] [diff] [review]
Fix an issue with Loop's unit tests failing when no devices are installed, due to bad stubbing.

Just remembered that Mike is out for a couple of days, passing to Ed or Dan to get this reviewed.
Attachment #8678068 - Flags: review?(mdeboer)
Attachment #8678068 - Flags: review?(edilee)
Attachment #8678068 - Flags: review?(dmose)

Comment 15

3 years ago
Comment on attachment 8678068 [details] [diff] [review]
Fix an issue with Loop's unit tests failing when no devices are installed, due to bad stubbing.

Took me a while to figure out it's the other tests not in the patch that call joinRoom triggering the unstubbed function.
Attachment #8678068 - Flags: review?(edilee)
Attachment #8678068 - Flags: review?(dmose)
Attachment #8678068 - Flags: review+
https://hg.mozilla.org/mozilla-central/rev/4278e19ea239
Status: NEW → RESOLVED
Last Resolved: 3 years ago
status-firefox44: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla44
You need to log in before you can comment on or make changes to this bug.