Closed Bug 1652523 Opened 4 years ago Closed 4 years ago

[wpt-sync] Sync PR 24580 - Update test threshold for Mac 10.11

Categories

(Core :: Web Audio, task, P4)

Product:
Core
Component:
Web Audio
Type:
task

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla80
Tracking Flags:
Tracking Status
firefox80 --- fixed

People

(Reporter: mozilla.org, Unassigned)

References

(
URL
)

Details

(Whiteboard: [wptsync downstream])

Sync web-platform-tests PR 24580 into mozilla-central (this bug is closed when the sync is complete).

PR: https://github.com/web-platform-tests/wpt/pull/24580
Details from upstream follow.

Raymond Toy <rtoy@chromium.org> wrote:

Update test threshold for Mac 10.11

The threshold for the -100 Hz sine wave is apparently just a little
too tight for MacOS 10.11. Adjust the threshold slightly.

Bug: 1104535
Change-Id: I7102093e2641a748d07ca63c05e9e8e63c9f8dce
Reviewed-on: https://chromium-review.googlesource.com/2295211
WPT-Export-Revision: 069f44cd25e9e24c8166911d99ee0a58af9b97f3

PR 24580 applied with additional changes from upstream: 980dc0c7625921634185a5369c05506196102b53
Component: web-platform-tests → Web Audio
Product: Testing → Core

CI Results

Ran 0 Firefox configurations based on mozilla-central, and Firefox, Chrome, and Safari on GitHub CI

Total 1 tests and 45 subtests

Status Summary

Firefox

OK : 1
PASS: 21
FAIL: 12

Chrome

OK : 1
PASS: 33

Safari

OK : 1
PASS: 9
FAIL: 6

Links

GitHub PR Head
GitHub PR Base

Details

New Tests That Don't Pass

/webaudio/the-audio-api/the-oscillatornode-interface/osc-basic-waveform.html
X Custom: 100 Hz does not equal [1,1.0141456127166748,1.0280853509902954,1.0418163537979126,1.0553359985351562,1.0686413049697876,1.081729769706726,1.094598650932312,1.1072453260421753,1.1196671724319458,1.1318618059158325,1.1438266038894653,1.1555593013763428,1.1670573949813843,1.1783186197280884,1.189340591430664...] with an element-wise tolerance of {"absoluteThreshold":0.0000018478,"relativeThreshold":0}.
Index Actual Expected AbsError RelError Test threshold
[132] 6.4742153882980347e-1 6.4741957187652588e-1 1.9669532775878906e-6 3.0381430575024735e-6 1.8477999999999999e-6
[134] 6.1133599281311035e-1 6.1133408546447754e-1 1.9073486328125000e-6 3.1199775673612907e-6 1.8477999999999999e-6
[135] 5.9310543537139893e-1 5.9310334920883179e-1 2.0861625671386719e-6 3.5173677065245060e-6 1.8477999999999999e-6
[136] 5.7475429773330688e-1 5.7475227117538452e-1 2.0265579223632813e-6 3.5259676629357436e-6 1.8477999999999999e-6
[137] 5.5628657341003418e-1 5.5628448724746704e-1 2.0861625671386719e-6 3.7501721061127269e-6 1.8477999999999999e-6
...and 97 more errors.
Max AbsError of 4.5299530029296875e-6 at index of 197.
[197] -6.1586797237396240e-1 -6.1587250232696533e-1 4.5299530029296875e-6 7.3553421947141031e-6 1.8477999999999999e-6
Max RelError of 4.6893454976520917e-4 at index of 165.
[165] 7.5594307854771614e-3 7.5558875687420368e-3 3.5432167351245880e-6 4.6893454976520917e-4 1.8477999999999999e-6
: FAIL
< [Test 5] 1 out of 2 assertions were failed.: FAIL
< [Test 0] 2 out of 2 assertions were failed.: FAIL
X Sine: 100 Hz does not equal [0,0.014247103594243526,0.028491314500570297,0.04272974282503128,0.056959498673677444,0.07117769122123718,0.08538143336772919,0.09956784546375275,0.11373404413461685,0.1278771609067917,0.14199431240558624,0.1560826450586319,0.17013929784297943,0.18416140973567963,0.19814614951610565,0.2120906561613083...] with an element-wise tolerance of {"absoluteThreshold":0.0000018045,"relativeThreshold":0}.
Index Actual Expected AbsError RelError Test threshold
[210] 1.4904043078422546e-1 1.4904226362705231e-1 1.8328428268432617e-6 1.2297470410336594e-5 1.8045000000000000e-6
[211] 1.3493725657463074e-1 1.3493916392326355e-1 1.9073486328125000e-6 1.4134878098823558e-5 1.8045000000000000e-6
[212] 1.2080668658018112e-1 1.2080866843461990e-1 1.9818544387817383e-6 1.6404902598974448e-5 1.8045000000000000e-6
[213] 1.0665159672498703e-1 1.0665365308523178e-1 2.0563602447509766e-6 1.9280729588396244e-5 1.8045000000000000e-6
[214] 9.2474862933158875e-2 9.2476986348628998e-2 2.1234154701232910e-6 2.2961555668760949e-5 1.8045000000000000e-6
...and 41 more errors.
Max AbsError of 4.4107437133789063e-6 at index of 254.
[254] -4.5938178896903992e-1 -4.5937737822532654e-1 4.4107437133789063e-6 9.6015692597196596e-6 1.8045000000000000e-6
Max RelError of 3.6860789043135361e-4 at index of 221.
[221] -7.1263583377003670e-3 -7.1237324737012386e-3 2.6258639991283417e-6 3.6860789043135361e-4 1.8045000000000000e-6
: FAIL
X Sine: SNR (db) is not greater than or equal to 118.91. Got 112.67687558535255.: FAIL
X Custom: SNR (db) is not greater than or equal to 122.43. Got 112.79630741561516.: FAIL
X Sine: SNR (db) is not greater than or equal to 130.95. Got 112.67687558535255.: FAIL

AUDIT TASK RUNNER FINISHED: 4 out of 6 tasks were failed.: FAIL

X Custom: SNR (db) is not greater than or equal to 138.76. Got 133.53606166055903.: FAIL
< [Test 4] 2 out of 2 assertions were failed.: FAIL
X Sine: -100 Hz does not equal [0,-0.014247103594243526,-0.028491314500570297,-0.04272974282503128,-0.056959498673677444,-0.07117769122123718,-0.08538143336772919,-0.09956784546375275,-0.11373404413461685,-0.1278771609067917,-0.14199431240558624,-0.1560826450586319,-0.17013929784297943,-0.18416140973567963,-0.19814614951610565,-0.2120906561613083...] with an element-wise tolerance of {"absoluteThreshold":4.7684e-7,"relativeThreshold":0}.
Index Actual Expected AbsError RelError Test threshold
[126] -9.7492843866348267e-1 -9.7492790222167969e-1 5.3644180297851563e-7 5.5023740910077999e-7 4.7683999999999997e-7
[127] -9.7165924310684204e-1 -9.7165870666503906e-1 5.3644180297851563e-7 5.5208871108633394e-7 4.7683999999999997e-7
[128] -9.6819281578063965e-1 -9.6819221973419189e-1 5.9604644775390625e-7 6.1562821473358384e-7 4.7683999999999997e-7
[129] -9.6452987194061279e-1 -9.6452921628952026e-1 6.5565109252929688e-7 6.7976281221583207e-7 4.7683999999999997e-7
[130] -9.6067112684249878e-1 -9.6067041158676147e-1 7.1525573730468750e-7 7.4453811492256024e-7 4.7683999999999997e-7
...and 108 more errors.
Max AbsError of 4.4107437133789063e-6 at index of 254.
[254] 4.5938178896903992e-1 4.5937737822532654e-1 4.4107437133789063e-6 9.6015692597196596e-6 4.7683999999999997e-7
Max RelError of 3.6860789043135361e-4 at index of 221.
[221] 7.1263583377003670e-3 7.1237324737012386e-3 2.6258639991283417e-6 3.6860789043135361e-4 4.7683999999999997e-7
: FAIL
< [Test 1] 2 out of 2 assertions were failed.: FAIL

Pushed by wptsync@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/718a16ea9026
[wpt PR 24580] - Update test threshold for Mac 10.11, a=testonly
https://hg.mozilla.org/integration/autoland/rev/23bad89cb9cb
[wpt PR 24580] - Update wpt metadata, a=testonly

https://hg.mozilla.org/mozilla-central/rev/718a16ea9026
https://hg.mozilla.org/mozilla-central/rev/23bad89cb9cb

Status: NEW → RESOLVED
Closed: 4 years ago
status-firefox80: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla80
You need to log in before you can comment on or make changes to this bug.