Closed Bug 1475158 Opened 6 years ago Closed 4 years ago

[wpt-sync] Sync PR 11935 - [webaudio] Generalize convolver test tolerances for numerical error

Categories

(Core :: Web Audio, defect, P4)

Product:
Core
Component:
Web Audio
Type:
defect

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla78
Tracking Flags:
Tracking Status
firefox78 --- fixed

People

(Reporter: wpt-sync, Unassigned)

References

(Blocks 1 open bug,
URL
)

Details

(Whiteboard: [wptsync downstream])

Sync web-platform-tests PR 11935 into mozilla-central (this bug is closed when the sync is complete). PR: https://github.com/web-platform-tests/wpt/pull/11935 Details from upstream follow. Karl Tomlinson <karlt+@karlt.net> wrote: > [webaudio] Generalize convolver test tolerances for numerical error > > Gecko uses FFTs for convolution, even with short response buffers, and so > precision would be expected to be similar to that with longer buffers in Blink > and Webkit. > > In Gecko, the actual calculations performed depend on what instructions are > available on the processor, and different code is used on some platforms. The > precision in the answers is similar with different processors and code, but > the slightly larger errors may occur in different tests. > > This patch picks a single expected precision for similar tests and uses that > for all such tests.
Component: web-platform-tests → Web Audio
Product: Testing → Core
Blocks: 1476231
Whiteboard: [wptsync downstream] → [wptsync downstream error]
Too many busted try pushes. Check the try results for infrastructure issues.
Blocks: 1620237
Whiteboard: [wptsync downstream error] → [wptsync downstream]
Whiteboard: [wptsync downstream] → [wptsync downstream error]
Whiteboard: [wptsync downstream error] → [wptsync downstream]
Whiteboard: [wptsync downstream] → [wptsync downstream error]
Whiteboard: [wptsync downstream error] → [wptsync downstream]
Whiteboard: [wptsync downstream] → [wptsync downstream error]
Whiteboard: [wptsync downstream error] → [wptsync downstream]
Whiteboard: [wptsync downstream] → [wptsync downstream error]
Whiteboard: [wptsync downstream error] → [wptsync downstream]
Whiteboard: [wptsync downstream] → [wptsync downstream error]
Whiteboard: [wptsync downstream error] → [wptsync downstream]
Whiteboard: [wptsync downstream] → [wptsync downstream error]
Whiteboard: [wptsync downstream error] → [wptsync downstream]
Whiteboard: [wptsync downstream] → [wptsync downstream error]
Whiteboard: [wptsync downstream error] → [wptsync downstream]
Whiteboard: [wptsync downstream] → [wptsync downstream error]
Whiteboard: [wptsync downstream error] → [wptsync downstream]
Whiteboard: [wptsync downstream] → [wptsync downstream error]
Whiteboard: [wptsync downstream error] → [wptsync downstream]

CI Results

Ran 12 Firefox configurations based on mozilla-central, and Firefox, Chrome, and Safari on GitHub CI

Total 74 tests

Status Summary

Firefox

OK : 3
PASS: 145[GitHub] 216[Gecko-android-em-7.0-x86_64-debug-geckoview, Gecko-android-em-7.0-x86_64-opt-geckoview, Gecko-linux1804-64-debug, Gecko-linux1804-64-opt, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows10-64-debug, Gecko-windows10-64-opt, Gecko-windows10-64-qr-debug, Gecko-windows10-64-qr-opt, Gecko-windows7-32-debug, Gecko-windows7-32-opt]
FAIL: 15

Chrome

OK : 3
PASS: 162

Safari

OK : 3
PASS: 40
FAIL: 37

Links

Gecko CI (Treeherder)
GitHub PR Head
GitHub PR Base

Details

New Tests That Don't Pass

/webaudio/the-audio-api/the-convolvernode-interface/convolver-response-2-chan.html
X 5.1-in explicit: Channel 1 does not equal [0,0,0,2.4002127647399902,1.8464014530181885,1.242234230041504,0.578858494758606,0.3615039587020874,0.16441935300827026,-0.7429117560386658,-1.5301964282989502,-1.898935079574585,-0.7277188301086426,0.01055973768234253,0.7105643153190613,1.7486152648925781...] with an element-wise tolerance of {"absoluteThreshold":4.76837158203125e-7,"relativeThreshold":0}.
Index Actual Expected AbsError RelError Test threshold
[226] 1.7922031879425049e+0 1.7922037839889526e+0 5.9604644775390625e-7 3.3257738493737067e-7 4.7683715820312500e-7
[261] -2.1812639236450195e+0 -2.1812646389007568e+0 7.1525573730468750e-7 3.2790873906301391e-7 4.7683715820312500e-7
[1217] -6.7543286085128784e-1 -6.7543232440948486e-1 5.3644180297851563e-7 7.9421991455252677e-7 4.7683715820312500e-7
Max AbsError of 7.1525573730468750e-7 at index of 261.
Max RelError of 7.9421991455252677e-7 at index of 1217.
: FAIL
X 2-in explicit mode: Channel 1 does not equal [0,0,0,0.9458408951759338,0.8448333740234375,0.8210252523422241,0.8620985746383667,0.8430315852165222,0.855602502822876,0.7933436632156372,0.9865825176239014,0.3972480297088623,-0.7786127924919128,-0.9223549962043762,-0.7896472215652466,-0.8727429509162903...] with an element-wise tolerance of {"absoluteThreshold":0,"relativeThreshold":0}.
Index Actual Expected AbsError RelError Test threshold
[0] -2.9802322387695313e-8 0.0000000000000000e+0 2.9802322387695313e-8 Infinity 0.0000000000000000e+0
[1] -8.9406967163085938e-8 0.0000000000000000e+0 8.9406967163085938e-8 Infinity 0.0000000000000000e+0
[2] -5.5879354476928711e-8 0.0000000000000000e+0 5.5879354476928711e-8 Infinity 0.0000000000000000e+0
[3] 9.4584083557128906e-1 9.4584089517593384e-1 5.9604644775390625e-8 6.3017622815202648e-8 0.0000000000000000e+0
[4] 8.4483331441879272e-1 8.4483337402343750e-1 5.9604644775390625e-8 7.0551953329600660e-8 0.0000000000000000e+0
...and 991 more errors.
Max AbsError of 4.1723251342773438e-7 at index of 444.
[444] -8.3554542064666748e-1 -8.3554583787918091e-1 4.1723251342773438e-7 4.9935323056215830e-7 0.0000000000000000e+0
Max RelError of Infinity at index of 0.
: FAIL
X 2-in explicit mode: Channel 0 does not equal [0,0,0.9458408951759338,0.8448333740234375,0.8210252523422241,0.8620985746383667,0.8430315852165222,0.855602502822876,0.7933436632156372,0.9865825176239014,0.3972480297088623,-0.7786127924919128,-0.9223549962043762,-0.7896472215652466,-0.8727429509162903,-0.8325281143188477...] with an element-wise tolerance of {"absoluteThreshold":0,"relativeThreshold":0}.
Index Actual Expected AbsError RelError Test threshold
[0] 2.9802322387695313e-8 0.0000000000000000e+0 2.9802322387695313e-8 Infinity 0.0000000000000000e+0
[1] -7.4505805969238281e-8 0.0000000000000000e+0 7.4505805969238281e-8 Infinity 0.0000000000000000e+0
[2] 9.4584077596664429e-1 9.4584089517593384e-1 1.1920928955078125e-7 1.2603524563040530e-7 0.0000000000000000e+0
[3] 8.4483331441879272e-1 8.4483337402343750e-1 5.9604644775390625e-8 7.0551953329600660e-8 0.0000000000000000e+0
[6] 8.4303164482116699e-1 8.4303158521652222e-1 5.9604644775390625e-8 7.0702742128080419e-8 0.0000000000000000e+0
...and 961 more errors.
Max AbsError of 4.1723251342773438e-7 at index of 267.
[267] 8.6412906646728516e-1 8.6412948369979858e-1 4.1723251342773438e-7 4.8283564129919487e-7 0.0000000000000000e+0
Max RelError of Infinity at index of 0.
: FAIL
< [3-channel input explicit mode] 2 out of 2 assertions were failed.: FAIL

AUDIT TASK RUNNER FINISHED: 3 out of 10 tasks were failed.: FAIL

X 3-in explicit: Channel 0 does not equal [0,0,0.9458408951759338,0.8448333740234375,0.8210252523422241,0.8620985746383667,0.8430315852165222,0.855602502822876,0.7933436632156372,0.9865825176239014,0.3972480297088623,-0.7786127924919128,-0.9223549962043762,-0.7896472215652466,-0.8727429509162903,-0.8325281143188477...] with an element-wise tolerance of {"absoluteThreshold":0,"relativeThreshold":0}.
Index Actual Expected AbsError RelError Test threshold
[0] 2.9802322387695313e-8 0.0000000000000000e+0 2.9802322387695313e-8 Infinity 0.0000000000000000e+0
[1] -7.4505805969238281e-8 0.0000000000000000e+0 7.4505805969238281e-8 Infinity 0.0000000000000000e+0
[2] 9.4584077596664429e-1 9.4584089517593384e-1 1.1920928955078125e-7 1.2603524563040530e-7 0.0000000000000000e+0
[3] 8.4483331441879272e-1 8.4483337402343750e-1 5.9604644775390625e-8 7.0551953329600660e-8 0.0000000000000000e+0
[6] 8.4303164482116699e-1 8.4303158521652222e-1 5.9604644775390625e-8 7.0702742128080419e-8 0.0000000000000000e+0
...and 961 more errors.
Max AbsError of 4.1723251342773438e-7 at index of 267.
[267] 8.6412906646728516e-1 8.6412948369979858e-1 4.1723251342773438e-7 4.8283564129919487e-7 0.0000000000000000e+0
Max RelError of Infinity at index of 0.
: FAIL
< [2-channel input, explicit mode] 2 out of 2 assertions were failed.: FAIL
X 3-in explicit: Channel 1 does not equal [0,0,0,0.9458408951759338,0.8448333740234375,0.8210252523422241,0.8620985746383667,0.8430315852165222,0.855602502822876,0.7933436632156372,0.9865825176239014,0.3972480297088623,-0.7786127924919128,-0.9223549962043762,-0.7896472215652466,-0.8727429509162903...] with an element-wise tolerance of {"absoluteThreshold":0,"relativeThreshold":0}.
Index Actual Expected AbsError RelError Test threshold
[0] -2.9802322387695313e-8 0.0000000000000000e+0 2.9802322387695313e-8 Infinity 0.0000000000000000e+0
[1] -8.9406967163085938e-8 0.0000000000000000e+0 8.9406967163085938e-8 Infinity 0.0000000000000000e+0
[2] -5.5879354476928711e-8 0.0000000000000000e+0 5.5879354476928711e-8 Infinity 0.0000000000000000e+0
[3] 9.4584083557128906e-1 9.4584089517593384e-1 5.9604644775390625e-8 6.3017622815202648e-8 0.0000000000000000e+0
[4] 8.4483331441879272e-1 8.4483337402343750e-1 5.9604644775390625e-8 7.0551953329600660e-8 0.0000000000000000e+0
...and 991 more errors.
Max AbsError of 4.1723251342773438e-7 at index of 444.
[444] -8.3554542064666748e-1 -8.3554583787918091e-1 4.1723251342773438e-7 4.9935323056215830e-7 0.0000000000000000e+0
Max RelError of Infinity at index of 0.
: FAIL
X 5.1-in explicit: Channel 0 does not equal [0,0,2.2955899238586426,2.0720272064208984,1.532062292098999,0.8335829973220825,-0.21171289682388306,-0.005277678370475769,0.8338430523872375,1.0169568061828613,0.3252672553062439,-0.6764181852340698,-1.054646611213684,-0.5476447343826294,0.25095105171203613,-0.10892623662948608...] with an element-wise tolerance of {"absoluteThreshold":4.76837158203125e-7,"relativeThreshold":0}.
Index Actual Expected AbsError RelError Test threshold
[203] -1.6055089235305786e+0 -1.6055095195770264e+0 5.9604644775390625e-7 3.7125064690425225e-7 4.7683715820312500e-7
[574] -6.4303541183471680e-1 -6.4303600788116455e-1 5.9604644775390625e-7 9.2692546054754973e-7 4.7683715820312500e-7
[598] 2.4209821224212646e+0 2.4209828376770020e+0 7.1525573730468750e-7 2.9544023450864054e-7 4.7683715820312500e-7
[609] -3.5243377089500427e-1 -3.5243326425552368e-1 5.0663948059082031e-7 1.4375472805072478e-6 4.7683715820312500e-7
[612] -1.3514939546585083e+0 -1.3514945507049561e+0 5.9604644775390625e-7 4.4102763673223926e-7 4.7683715820312500e-7
...and 6 more errors.
Max AbsError of 7.1525573730468750e-7 at index of 598.
Max RelError of 2.0121786521101719e-6 at index of 1143.
[1143] 2.5178602337837219e-1 2.5178653001785278e-1 5.0663948059082031e-7 2.0121786521101719e-6 4.7683715820312500e-7
: FAIL
< [5.1-channel input explicit mode] 2 out of 2 assertions were failed.: FAIL
/webaudio/the-audio-api/the-convolvernode-interface/convolver-response-4-chan.html
Executing "delayed buffer set": FAIL (Chrome: PASS, Safari: FAIL)
/webaudio/the-audio-api/the-convolvernode-interface/convolver-response-1-chan.html
X 5.1 chan downmix explicit: Channel 0 does not equal [0,0,2.2955899238586426,2.0720272064208984,1.532062292098999,0.8335829973220825,-0.21171289682388306,-0.005277678370475769,0.8338430523872375,1.0169568061828613,0.3252672553062439,-0.6764181852340698,-1.054646611213684,-0.5476447343826294,0.25095105171203613,-0.10892623662948608...] with an element-wise tolerance of {"absoluteThreshold":4.76837158203125e-7,"relativeThreshold":0}.
Index Actual Expected AbsError RelError Test threshold
[203] -1.6055089235305786e+0 -1.6055095195770264e+0 5.9604644775390625e-7 3.7125064690425225e-7 4.7683715820312500e-7
[574] -6.4303541183471680e-1 -6.4303600788116455e-1 5.9604644775390625e-7 9.2692546054754973e-7 4.7683715820312500e-7
[598] 2.4209821224212646e+0 2.4209828376770020e+0 7.1525573730468750e-7 2.9544023450864054e-7 4.7683715820312500e-7
[609] -3.5243377089500427e-1 -3.5243326425552368e-1 5.0663948059082031e-7 1.4375472805072478e-6 4.7683715820312500e-7
[612] -1.3514939546585083e+0 -1.3514945507049561e+0 5.9604644775390625e-7 4.4102763673223926e-7 4.7683715820312500e-7
...and 6 more errors.
Max AbsError of 7.1525573730468750e-7 at index of 598.
Max RelError of 2.0121786521101719e-6 at index of 1143.
[1143] 2.5178602337837219e-1 2.5178653001785278e-1 5.0663948059082031e-7 2.0121786521101719e-6 4.7683715820312500e-7
: FAIL
X 5.1 chan downmix explicit: Channel 1 does not equal [0,0,2.4002127647399902,1.8464014530181885,1.242234230041504,0.578858494758606,0.3615039587020874,0.16441935300827026,-0.7429117560386658,-1.5301964282989502,-1.898935079574585,-0.7277188301086426,0.01055973768234253,0.7105643153190613,1.7486152648925781,0.26711004972457886...] with an element-wise tolerance of {"absoluteThreshold":4.76837158203125e-7,"relativeThreshold":0}.
Index Actual Expected AbsError RelError Test threshold
[185] -1.6890264749526978e+0 -1.6890270709991455e+0 5.9604644775390625e-7 3.5289336564707304e-7 4.7683715820312500e-7
[221] -1.0010230541229248e+0 -1.0010237693786621e+0 7.1525573730468750e-7 7.1452422927844007e-7 4.7683715820312500e-7
[225] 1.7922031879425049e+0 1.7922037839889526e+0 5.9604644775390625e-7 3.3257738493737067e-7 4.7683715820312500e-7
[349] 1.7407279014587402e+0 1.7407284975051880e+0 5.9604644775390625e-7 3.4241206977892302e-7 4.7683715820312500e-7
[819] -2.4523670673370361e+0 -2.4523677825927734e+0 7.1525573730468750e-7 2.9165924555919635e-7 4.7683715820312500e-7
...and 3 more errors.
Max AbsError of 7.1525573730468750e-7 at index of 221.
Max RelError of 1.9420639938334170e-6 at index of 864.
[864] -3.3760464191436768e-1 -3.3760529756546021e-1 6.5565109252929688e-7 1.9420639938334170e-6 4.7683715820312500e-7
: FAIL

AUDIT TASK RUNNER FINISHED: 1 out of 11 tasks were failed.: FAIL (Safari: FAIL)

< [5.1-channel input, explicit] 2 out of 2 assertions were failed.: FAIL

Pushed by wptsync@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/77dc3674000e [wpt PR 11935] - [webaudio] Generalize convolver test tolerances for numerical error, a=testonly https://hg.mozilla.org/integration/autoland/rev/d94d118c0756 [wpt PR 11935] - Update wpt metadata, a=testonly
Pushed by wptsync@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/9d93ae046a71 [wpt PR 11935] - [webaudio] Generalize convolver test tolerances for numerical error, a=testonly https://hg.mozilla.org/integration/autoland/rev/2caad7825718 [wpt PR 11935] - Update wpt metadata, a=testonly

https://hg.mozilla.org/mozilla-central/rev/9d93ae046a71
https://hg.mozilla.org/mozilla-central/rev/2caad7825718

Status: NEW → RESOLVED
Closed: 4 years ago
status-firefox78: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla78
You need to log in before you can comment on or make changes to this bug.