Closed
Bug 1499871
Opened 6 years ago
Closed 6 years ago
Green up wpt audio-output directory on fennec
Categories
(Testing :: web-platform-tests, defect)
Tracking
(firefox65 fixed)
RESOLVED
FIXED
mozilla65
Tracking | Status | |
---|---|---|
firefox65 | --- | fixed |
People
(Reporter: KWierso, Assigned: KWierso)
References
Details
Attachments
(3 files)
https://docs.google.com/spreadsheets/d/11LE-ITakYnH9XIiKzSgFntOazXPxDrx9jtZ-tk71Kls/edit?usp=sharing
setSinkId seems to not be defined, despite the wpt metadata setting media.setsinkid.enabled to true.
Assignee | ||
Comment 1•6 years ago
|
||
This seems similar to the failures I see in bug 1499866, where things that should be enabled aren't getting enabled, maybe it's something in the harness?
The log output includes >> 1:04.72 INFO Setting pref media.setsinkid.enabled (true) << but maybe that isn't actually making it into the profile on Fennec?
Flags: needinfo?(james)
Flags: needinfo?(dburns)
Assignee | ||
Comment 2•6 years ago
|
||
If it matters, these are being run on builds from fairly current mozilla-central. I assume these builds get the EARLY_BETA_OR_EARLIER treatment, but I haven't looked to confirm that. Don't know if that could make a difference for some of these features...
Comment 3•6 years ago
|
||
(In reply to Wes Kocher (:KWierso) from comment #1)
> This seems similar to the failures I see in bug 1499866, where things that
> should be enabled aren't getting enabled, maybe it's something in the
> harness?
>
> The log output includes >> 1:04.72 INFO Setting pref media.setsinkid.enabled
> (true) << but maybe that isn't actually making it into the profile on Fennec?
Can you run 1 test and then when it is finished and the emulator is left open can you have a look at the about:config for that preferece to make sure it is set. Part of me wonders if we are setting the profile on the host but not actually making sure we do it for mobile...
Flags: needinfo?(dburns)
Comment 4•6 years ago
|
||
The preferences are set using Chrome js executed via marionette [1], so in theory it ought to work on mobile. It's possible that it doesn't of course. You could also try reading back the pref after setting it and logging the value you get. The pref setting is happening at [2], so adding something like
logging.info("%s has value " % (name, self.prefs.get(name)))
After that should allow us to check consistency.
[1] https://searchfox.org/mozilla-central/source/testing/web-platform/tests/tools/wptrunner/wptrunner/executors/executormarionette.py#209
[2] https://searchfox.org/mozilla-central/source/testing/web-platform/tests/tools/wptrunner/wptrunner/executors/executormarionette.py#209
Flags: needinfo?(james)
Assignee | ||
Comment 5•6 years ago
|
||
(In reply to David Burns :automatedtester from comment #3)
> Can you run 1 test and then when it is finished and the emulator is left
> open can you have a look at the about:config for that preferece to make sure
> it is set. Part of me wonders if we are setting the profile on the host but
> not actually making sure we do it for mobile...
When I do that, after a test that does NOT explicitly set that pref, media.setsinkid.enabled is 'false', but when the test explicitly DOES set the pref, it is 'true'. So the preference IS getting set correctly. There goes that theory...
Alex, is this expected to not work on Android, maybe?
Flags: needinfo?(achronop)
Comment 6•6 years ago
|
||
The setSinkId method should be available but the enumerate devices does not work on android so it return 0 devices, so:
These 2 failures are not expected. I am testing on my android device and the method is there. It must be a failure of the test harness.
FAIL setSinkId on default audio output should always work - audio.setSinkId is not a function
@http://web-platform.test:8000/audio-output/setSinkId.html:12:19
Test.prototype.step@http://web-platform.test:8000/resources/testharness.js:1561:20
promise_test/tests.promise_tests</<@http://web-platform.test:8000/resources/testharness.js:591:31
promise_test/tests.promise_tests<@http://web-platform.test:8000/resources/testharness.js:590:20
FAIL setSinkId fails with NotFoundError on made up deviceid - audio.setSinkId is not a function
@http://web-platform.test:8000/audio-output/setSinkId.html:14:55
Test.prototype.step@http://web-platform.test:8000/resources/testharness.js:1561:20
promise_test/tests.promise_tests</<@http://web-platform.test:8000/resources/testharness.js:591:31
promise_test/tests.promise_tests<@http://web-platform.test:8000/resources/testharness.js:590:20
The following error is expected, so over all, we expect that the test will fail in android with that error.
FAIL List device, setSinkId should on the default, the rest of the devices will get a NotAlowedError - assert_not_equals: media device list includes at least one audio output device got disallowed value 0
@http://web-platform.test:8000/audio-output/setSinkId.html:20:3
The same for the other test.
Flags: needinfo?(achronop)
Comment 7•6 years ago
|
||
The feature landed one week ago in case you don't test the latest. Before that the _function does not exist_ error is expected. See also Bug 934425.
Assignee | ||
Comment 8•6 years ago
|
||
Okay, so running on a build from a recent (last few hours) pull from autoland, I get these errors:
Unexpected Results
------------------
/audio-output/setSinkId.html
FAIL List device, setSinkId should on the default, the rest of the devices will get a NotAlowedError - assert_not_equals: media device list includes at least one audio output device got disallowed value 0
@http://web-platform.test:8000/audio-output/setSinkId.html:20:3
/audio-output/setSinkId.https.html
FAIL List device, setSinkId should on the default, the rest of the devices will get a NotAlowedError - assert_not_equals: media device list includes at least one audio output device got disallowed value 0
@https://web-platform.test:8443/audio-output/setSinkId.https.html:22:3
Flags: needinfo?(achronop)
Assignee | ||
Comment 9•6 years ago
|
||
Assignee | ||
Comment 10•6 years ago
|
||
(In reply to Wes Kocher (:KWierso) from comment #9)
> Created attachment 9019564 [details]
> Bug 1499871 - Fix spelling of "NotAllowedError" r?achronop
(This doesn't fix the failures, but the misspelling annoyed me enough to correct it.)
Comment 11•6 years ago
|
||
(In reply to Wes Kocher (:KWierso) from comment #8)
> Okay, so running on a build from a recent (last few hours) pull from
> autoland, I get these errors:
>
> Unexpected Results
> ------------------
> /audio-output/setSinkId.html
> FAIL List device, setSinkId should on the default, the rest of the devices
> will get a NotAlowedError - assert_not_equals: media device list includes at
> least one audio output device got disallowed value 0
> @http://web-platform.test:8000/audio-output/setSinkId.html:20:3
> /audio-output/setSinkId.https.html
> FAIL List device, setSinkId should on the default, the rest of the devices
> will get a NotAlowedError - assert_not_equals: media device list includes at
> least one audio output device got disallowed value 0
> @https://web-platform.test:8443/audio-output/setSinkId.https.html:22:3
That looks good. It's the expected failure on android (for now).
Flags: needinfo?(achronop)
Assignee | ||
Updated•6 years ago
|
Keywords: leave-open
Comment 12•6 years ago
|
||
Pushed by wkocher@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/121ccc3d4437
Fix spelling of "NotAllowedError" r=achronop
Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/13711 for changes under testing/web-platform/tests
Comment 14•6 years ago
|
||
bugherder |
Assignee | ||
Updated•6 years ago
|
Keywords: leave-open
Assignee | ||
Comment 15•6 years ago
|
||
Upstream PR merged
Comment 17•6 years ago
|
||
Pushed by wkocher@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/ca744888101e
Update test metadata for wpt audio-output on fennec r=achronop
Comment 18•6 years ago
|
||
bugherder |
Status: NEW → RESOLVED
Closed: 6 years ago
status-firefox65:
--- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla65
Assignee | ||
Comment 19•6 years ago
|
||
These tests started passing again on try, and started doing the same for me locally after I force-updated the android avd. Patch incoming to update the expectation data.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Assignee | ||
Comment 20•6 years ago
|
||
At some point, these started passing again on CI. Force-updating the Android AVD makes these tests start passing locally as well. Let's just remove the android-specific failing expectations since they don't seem necessary any more.
Comment 21•6 years ago
|
||
It's Bug 1482150 that has just been landed. Those tests are not on try yet, right? I did not see a change there.
See Also: → 1482150
Assignee | ||
Comment 22•6 years ago
|
||
Correct, nothing runs without https://hg.mozilla.org/try/rev/29da107e44d72adf184b5cbcce9a53a157f3cc41 applied, and then only on try with you specifically asking for them to run.
Sample failure: https://treeherder.mozilla.org/logviewer.html#?job_id=210768951&repo=try
Comment 23•6 years ago
|
||
Pushed by wkocher@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/91ad57be0886
Mark the wpt audio-output expectations as passing everywhere r=jgraham
Comment 24•6 years ago
|
||
bugherder |
Status: REOPENED → RESOLVED
Closed: 6 years ago → 6 years ago
Resolution: --- → FIXED
You need to log in
before you can comment on or make changes to this bug.
Description
•