1309381 - New spatial audio on Facebook demo has crazy slow mouse movement relative to Chrome.

Status: NEW

(Core :: Graphics, defect, P3)

Description

[Martin Best (:mbest)]

https://media.fb.com/2016/10/07/introducing-spatial-audio-for-360-videos-on-facebook/

Test the 360 video examples in Chrome and Firefox.

Experienced behavior: Firefox seems to lag behind and stutter even though video playback is smooth and canvas frame rate is high.

Expected behavior: Movement should be smooth as seen in Chrome.

Comment 1

[Maire Reavy [:mreavy]]

Jean-Yves, can you take a look at this and tell me if this is really a playback issue?

Comment 2

[Martin Best (:mbest)]

One thing I'd like to add on that note, the video playback is consistently smooth, this suggests that the 3D is also smooth as matter of necessity.  I would certainly look at input as a potential cause.

Comment 3

[Anthony Jones (:ajones, :kentuckyfriedtakahe, :k17e)]

There is a known issue on YouTube with polling the screen size.

Comment 4

[Andrew Overholt [:overholt]]

I just get that my browser isn't supported (nightly or release on Windows 10).

Comment 5

[Desigan Chinniah [:cyberdees] [:dees] [London - GMT]]

:jya per Comment 1 - any feedback?

Comment 6

[JW Wang [:jwwang] [:jw_wang]]

Got error messages like:

[Child 2986] WARNING: We can't support non-premultiplied alpha for video!: '(aSurfaceFlags & SFE_PREFER_NO_PREMULTIPLY_ALPHA) == 0', file /media/jwwang/DATA/codebase/mozilla-central/firefox/layout/base/nsLayoutUtils.cpp, line 7274
JavaScript warning: https://www.facebook.com/rsrc.php/v3izp-4/yj/l/en_US/4XAKaL43YwK.js, line 1056: Error: WebGL: texImage2D: Failed to hit GPU-copy fast-path. Falling back to CPU upload.
JavaScript warning: https://www.facebook.com/rsrc.php/v3izp-4/yj/l/en_US/4XAKaL43YwK.js, line 1056: Error: WebGL: texImage2D: Conversion requires pixel reformatting.
[Child 2986] WARNING: We can't support non-premultiplied alpha for video!: '(aSurfaceFlags & SFE_PREFER_NO_PREMULTIPLY_ALPHA) == 0', file /media/jwwang/DATA/codebase/mozilla-central/firefox/layout/base/nsLayoutUtils.cpp, line 7274
JavaScript warning: https://www.facebook.com/rsrc.php/v3izp-4/yj/l/en_US/4XAKaL43YwK.js, line 1056: Error: WebGL: texImage2D: Failed to hit GPU-copy fast-path. Falling back to CPU upload.
JavaScript warning: https://www.facebook.com/rsrc.php/v3izp-4/yj/l/en_US/4XAKaL43YwK.js, line 1056: Error: WebGL: texImage2D: Conversion requires pixel reformatting.
[Child 2986] WARNING: We can't support non-premultiplied alpha for video!: '(aSurfaceFlags & SFE_PREFER_NO_PREMULTIPLY_ALPHA) == 0', file /media/jwwang/DATA/codebase/mozilla-central/firefox/layout/base/nsLayoutUtils.cpp, line 7274

Looks like a gfx issue.

Comment 7

[David Bolter [:davidb] (NeedInfo me for attention)]

Milan, in partner triage we thought we'd leave this to you. Should we get activity here?

Comment 8

[Milan Sreckovic [:milan] (needinfo for best results)]

Sure, we get facebook to not use non-premult alpha for video.  This seems to be an evangelism.  The above is a warning that they are doing things which drop us off the optimized path and into the weeds.
This starts in layout/video - Matt, is there something that we can do with this kind of data?

Comment 9

[Matt Woodrow (:mattwoodrow)]

What platform is this happening on?

non-premult shouldn't really matter for video, as it's usually opaque and there isn't any alpha to be multiplied.

It sounds like we're just missing a fast path for the video format here, shouldn't be too hard to fix.

Comment 10

[Kelsey Gilbert [:jgilbert]]

Bug 1350066 is trying to remove "WARNING: We can't support non-premultiplied alpha for video!".

Comment 11

[Kelsey Gilbert [:jgilbert]]

Bug 1350107 will make it clear why we're falling off the fast path.

It's likely they're either:
* picking the wrong premult setting (they can fix this)
* our code is failing to blit from the layers::Image (we'd need to fix)

Comment 12

[Kelsey Gilbert [:jgilbert]]

> Error: WebGL perf warning: texImage2D: Failed to hit GPU-copy fast-path. (src type 11) Falling back to CPU upload. (UNPACK_PREMULTIPLY_ALPHA_WEBGL is not false)  16RV_DNduLc.js:29:14353
> Error: WebGL perf warning: texImage2D: Conversion requires pixel reformatting. (27->20)  16RV_DNduLc.js:29:14353

Comment 13

[Kelsey Gilbert [:jgilbert]]

(In reply to Jeff Gilbert [:jgilbert] from comment #12)
> > Error: WebGL perf warning: texImage2D: Failed to hit GPU-copy fast-path. (src type 11) Falling back to CPU upload. (UNPACK_PREMULTIPLY_ALPHA_WEBGL is not false)  16RV_DNduLc.js:29:14353
> > Error: WebGL perf warning: texImage2D: Conversion requires pixel reformatting. (27->20)  16RV_DNduLc.js:29:14353

This doesn't necessarily mean that fixing the premult setting will fix it, since it's before we try blitting. I'll check.

Comment 14

[Emma Humphries ☕️🎸🧞‍♀️✨ (she/they) [:emceeaich] (Pacific Time) use needinfo]

This is an assigned P1 bug without activity in two weeks. 

If you intend to continue working on this bug for the current release/iteration/sprint, remove the 'stale-bug' keyword.

Otherwise we'll reset the priority of the bug back to '--' on Monday, August 28th.

Comment 15

[Sylvestre Ledru [:Sylvestre]]

Moving to p3 because no activity for at least 24 weeks.