Closed Bug 1216551 Opened 4 years ago Closed 4 years ago

osx 10.10.5 error for test file browser/components/loop/test/shared/test_shared_all.py

Categories

(Hello (Loop) :: General, defect, P2)

defect
Points:
5

Tracking

(firefox44 fixed)

RESOLVED FIXED
mozilla44
Iteration:
44.3 - Nov 2
Tracking Status
firefox44 --- fixed

People

(Reporter: jmaher, Assigned: standard8)

References

Details

Attachments

(1 file)

we are close to getting osx 10.10.5 up and running, unfortunately the Mn job fails on debug and we need to get it fixed.

here is a clip from the log:
09:55:09     INFO -  TEST-START | test_shared_all.py TestSharedUnits.test_units
09:55:30    ERROR -  TEST-UNEXPECTED-FAIL | test_shared_all.py TestSharedUnits.test_units | AssertionError: build/tests/marionette/tests/browser/components/loop/test/shared/index.html: 1 failure(s) encountered:
09:55:30    ERROR -  TEST-UNEXPECTED-FAIL | index.html | should long only the warnings we expect - AssertionError: expected 2 to deeply equal 0
09:55:30     INFO -  AssertionError: expected 2 to deeply equal 0
09:55:30     INFO -  Traceback (most recent call last):
09:55:30     INFO -    File "/builds/slave/test/build/venv/lib/python2.7/site-packages/marionette/marionette_test.py", line 296, in run
09:55:30     INFO -      testMethod()
09:55:30     INFO -    File "/builds/slave/test/build/tests/marionette/tests/browser/components/loop/test/shared/test_shared_all.py", line 16, in test_units
09:55:30     INFO -      self.check_page("index.html")
09:55:30     INFO -    File "/builds/slave/test/build/tests/marionette/tests/browser/components/loop/test/shared/frontend_tester.py", line 128, in check_page
09:55:30     INFO -      raise AssertionError(self.get_failure_details(page))
09:55:30     INFO -  TEST-INFO took 20530ms

there is no screenshots available (might be harness specific), or other info in the debug log.
:standard8, can you look at this and help us figure out if we can fix the test, the os configuration, or disable the test in the manifest?
Flags: needinfo?(standard8)
Ok, I've just done a try push to attempt to add some more debug info. If it works, its likely something we'll want to add to the test setup anyway (or modify slightly) so useful to see:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=cf2f915db9e4
Ok, the try run failed, but I don't understand why. The patch I want to try is:

https://hg.mozilla.org/try/rev/9e9b0cb8a2d0

Joel, can you help here please?
Flags: needinfo?(standard8) → needinfo?(jmaher)
this looks like infra structure errors or build errors, not related to the test.  any chance the parent revision you pushed with was bad?
Flags: needinfo?(jmaher)
Ah, yes it was a bad one - forgot to look. I've pushed afresh:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=4be066010010
pushed to try with updated revisions:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=f69624997f77

lets see if 3rd time is the charm.
Ok, I've pushed something that should get us a screenshot url out, not perfect, but it'll do for now:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=9dc7c75ec206
I forgot that a screenshot isn't going to solve this issue, since primarily these are warnings that go to the console. I've found a possible change to the test that might now give us more information and pushed again:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=c83192b2ae9e
Ok, we now have something reasonable as an output:

TEST-UNEXPECTED-FAIL | index.html | should long only the warnings we expect - Error: No stores registered for event type ,connectionFailure,No stores registered for event type ,connectionFailure

That's a bit weird though as these failures appear to be part of the test code, and we don't see them normally. Could be timing, but its strange that afaik we haven't seen this normally.

I'll dig into it a bit today.
Now with a stack output to see who's trying to dispatch the connectionFailure action:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=334ff430d321
Green Mn: https://treeherder.mozilla.org/#/jobs?repo=try&revision=572aab8ad03b

So either the builder probably doesn't have any mic/camera devices set up whereas our other boxes have at least one *Or* there's an async timing issue that's revealed on this machine.

I'll tidy the patch up tomorrow and get it ready for review.
Assignee: nobody → standard8
Iteration: --- → 44.3 - Nov 2
Points: --- → 5
Rank: 28
Priority: -- → P2
This fixes the stubbing for hasAudioOrVideoDevices when doing the join room tests, so that hasAudioOrVideoDevices never actually gets called.
Attachment #8678068 - Flags: review?(mdeboer)
Comment on attachment 8678068 [details] [diff] [review]
Fix an issue with Loop's unit tests failing when no devices are installed, due to bad stubbing.

Just remembered that Mike is out for a couple of days, passing to Ed or Dan to get this reviewed.
Attachment #8678068 - Flags: review?(mdeboer)
Attachment #8678068 - Flags: review?(edilee)
Attachment #8678068 - Flags: review?(dmose)
Comment on attachment 8678068 [details] [diff] [review]
Fix an issue with Loop's unit tests failing when no devices are installed, due to bad stubbing.

Took me a while to figure out it's the other tests not in the patch that call joinRoom triggering the unstubbed function.
Attachment #8678068 - Flags: review?(edilee)
Attachment #8678068 - Flags: review?(dmose)
Attachment #8678068 - Flags: review+
https://hg.mozilla.org/mozilla-central/rev/4278e19ea239
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla44
You need to log in before you can comment on or make changes to this bug.