Open Bug 1529812 Opened 7 years ago Updated 3 years ago

Panomoments fail to load when e10s is enabled.

Categories

(Core :: Graphics: CanvasWebGL, defect)

Product:
Core
Component:
Graphics: CanvasWebGL
Platform:
Unspecified
Android
Type:
defect

Tracking

()

Tracking Flags:
Tracking Status
firefox65 --- wontfix
firefox66 --- wontfix
firefox67 --- ?
firefox68 --- ?

People

(Reporter: rbarker, Unassigned)

References

(
URL
)

Details

Attachments

(1 file)

mse-log.txt
69.93 KB, text/plain
Details

Panomoment pages such as: https://my.panomoments.com/u/dustinkerstein/m/split-croatia-harbor?hideInfo=1 fail to load in GeckoView when e10s is enabled.

Blocks: 1528330
status-firefox67: --- → affected
OS: All → Android
Whiteboard: [geckoview:fxr:p1]

Can be reproduced in GeckoView Example as well as Firefox Reality.

[qf] because this site is also very slow to load on Firefox desktop. On my Windows laptop, Chrome loads the HD scene in 1 second and UHD in 2 seconds. Firefox 67 Nightly loads the HD scene in 6 seconds and UDH in 9 seconds.

https://my.panomoments.com/u/dustinkerstein/m/split-croatia-harbor?hideInfo=1

Here is a Gecko profile of Firefox 67 Nightly loading the UHD scene: http://bit.ly/2T3I780

URL: https://my.panomoments.com/u/dustinke...
status-firefox65: --- → affected
status-firefox66: --- → affected
Whiteboard: [geckoview:fxr:p1] → [geckoview:fxr:p1] [qf]

This is now really confusing bug. About some page not loading somewhere, and then about performance issue.

Whiteboard: [geckoview:fxr:p1] [qf] → [geckoview:fxr:p1]

P1 because [geckoview:fxr:p1]

status-firefox65: affected → wontfix
Priority: -- → P1

Desktop linux profile with lots of threads captured, since it wasn't obvious where the problem would be:
https://perfht.ml/2Tjhiwx

Near the end, there's 1.4s of CPU spent on MediaPDecoder in mp4 decoding, all in mozilla::MP4TrackDemuxer::EnsureUpToDateIndex()

However, the primary difference seems to be fetching the mp4, which takes 6 seconds. (Chrome takes about 2-3 seconds to do the entire load vs 7-9 for us). The difference seems to be in how they fetch and process hd-dashinit.mp4; they take about 6ish seconds to fetch it as well -- but does so in ~250K chunks using range requests.

Chris - over to you for media perf triage/thoughts.

No idea how this would interact with e10s on GV; it shows no obvious differences e10s or not on desktop. Randall - any more details on what happens on GVE? Profile? any interesting logs (or logs we can turn on Chris?)

Flags: needinfo?(rbarker)
Flags: needinfo?(cpearce)

So looking at the network side of things a little bit:

Firefox does HTTP GET request and the first two are answered with 206 with 919 Bytes and 217KB. When Firefox then does another GET request the answer takes several seconds (in my case 5.4s) because it gets a 200 OK with the 56MB in the answer.

Chrome also does two initial HTTP GET requests, which are also answered with 206 and 919 and 217KB answers. But then Chrome sends a bunch of OPTIONS requests, before then fetching the content with lots of smaller GET requests all answered with 206.

I'm curious if the different GET requests in Chrome/Firefox are due to different client JS or something in the different networking stacks. While we could debug their obfuscated JS, has anyone contacted the PanoMoments team (info@panomoments.com according to https://www.panomoments.com/about)?

Maybe Honza can see what the difference is in the networking traffic between Chrome and Firefox (comment 6 and comment 7 - loading https://my.panomoments.com/u/dustinkerstein/m/split-croatia-harbor?hideInfo=1)?

Flags: needinfo?(honzab.moz)

If I load the page I see the behaviour Nils described; one big 56MB GET.

Then if I set the useragent string to Chrome and reload, I see a lot of small requests like Nils sees in Chrome. The page then fails to load after downloading the last segment. I didn't look into why.

I would guess that the page is UA sniffing, and using ReadableStreams for Chrome and not for Firefox.

So perhaps the GV problem is in ReadableStreams? Or maybe the site's UA sniffing is failing on GVE or FXR?

Flags: needinfo?(cpearce)

Since ReadableStream support in Firefox only came out with 65 https://developer.mozilla.org/en-US/docs/Web/API/ReadableStream my guess would that pages likes this don't rely on proper feature detection, but instead look at UA strings like Chris points out. So besides confirming the ReadableStream usage this looks like a webcompat issue to me and not something we need to fix in Firefox.

Jason as you were involved implementing readable streams:

  • is there an easy way to check if Readable Streams is being used on this page?
  • are you aware of any issues with Readable Streams as comment #9 appears to indicate that it doesn't work by pretending to be Chrome?
Flags: needinfo?(jorendorff)

  1. Well, as it happens, there is still a pref for ReadableStreams. I've been meaning to remove it but haven't yet. So you can set javascript.options.streams to false in about:config and see if the behavior changes.

    If that doesn't give clear results, I think you'd have to either attach with a debugger, or write an extension to monkeypatch the ReadableStream constructor and some methods to alert before doing their thing.

  2. ReadableStream is awfully solid; the most likely issue is that there are some related features we haven't implemented yet: WritableStream, TransformStream, .pipeTo and .pipeThrough, and some DOM text-decoding stuff. Mabye the site uses some of these so-far-Chrome-only features if you UA-spoof Chrome?

Flags: needinfo?(jorendorff) → needinfo?(drno)
Flags: needinfo?(honzab.moz)

Tom, do you recall any ReadableStream stuff when you were looking into Bug 1531373?

Flags: needinfo?(twisniewski)

No, and I just did a quick search through the sources on that page, and saw no instances of ReadableStream or getReader, so I don't suspect they're using them.

Flags: needinfo?(twisniewski)

My understanding of bug 1531373 comment 10 is that the site has a path for using MSE, and a path for downloading the whole video as a blob and using <video src=blob>.

It appears that on desktop Firefox we're hitting the blob path (r.MSE=false), downloading the whole 56/157mb video in a single chunk (which takes forever), and then playing it.

If we switch to forcing the MSE path (by editing the script, or UA spoofing) then we can see the individual MSE chunks being downloaded, but the video doesn't ever play. That suggests either a bug in our MSE implementation, or a bug in the sites usage of MSE (but one that doesn't break blink's impl).

On GeckoView, it appears that the video doesn't ever load, but we don't have much information on why. It seems plausible that we're taking the r.MSE=true path (the condition shows in bug 1531373 does check for Android), which appears to be broken.

So I think the next steps here are:

  • Debug the site on GeckoView to identify which path we're taking to see if the difference is due to the UA sniffing returning different results.

  • Force the MSE path on Desktop, debug why the MSE attempts are failing and figure out if there's a Gecko bug.

Attached file mse-log.txt —

Attaching a log file from running with MSE forced, and MOZ_LOG=MediaSource:5,MediaDecoder:5

Everything looks fine in there as far as I can tell. The video is paused the whole time, and the JS code is seeking through it constantly.

That seems expected though, as the page draws the video frames into WebGL.

When I run the working version (no MSE), then I see calls to WebGL context's texImage2D and into nsLayoutUtils::SurfaceFromElement to retrieve the current video frame.

With the MSE version, this never happens.

It's going to be hard to debug exactly why the website isn't attempting to draw the video frames into their WebGL canvas with the minified JS.

Can we ask them to investigate?

Lars, Mike -- As Matt says in comment 17, we need help from Panomoments to make progress here, else we're stalled. Do you have contacts there or can we make contacts to get this un-stalled?

Flags: needinfo?(miket)
Flags: needinfo?(larsberg)

I don't have any, but I can help look if Lars also doesn't (I suspect his odds are higher, given the webvr connection).

Flags: needinfo?(miket)

In the meantime, I'll try out info@panomoments.com.

edit: Adam Stevenson also helped reach out over LinkedIn.

I believe cvan has contacts there.

Flags: needinfo?(rbarker) → needinfo?(cvan)

Hey! I'm from the PanoMoments team. Adam reached out to me and I'd be happy to help where I can (though I am traveling and have very limited internet currently). A few quick notes:

  1. MSE - PanoMoments does use MSE, but as you've discovered it's only enabled on Chrome. It does this via user-agent sniffing due to issues we had with our MSE implementation on Firefox. Here's an older JSFiddle that utilizes the core of our WebGL + MSE implementation which is functional in Chrome but broken in Firefox - https://jsfiddle.net/dustinkerstein/eujpb60b/ - Here's another JSFiddle (non-WebGL MSE code) - https://jsfiddle.net/u3enjLzz/ - that allows you to use the [ ] keyboard keys to index frames. The latter works on Firefox (and Chrome) but once WebGL is added to the mix (as per the first JSFiddle) it breaks on Firefox. If you have any ideas on why this is broken / a possible workaround I'd absolutely love to get MSE working on Firefox. MSE allows for both "streaming" PanoMoments and higher performance decoding on Chrome.

  2. Slower downloads on Firefox - With the non-MSE PanoMoments code, the JS will download a portion (the first frame) of the mp4 via Fetch and append that as a blob to the video.src while the rest of the file downloads via a XMLHttpRequest. Then the entirety of the mp4 blob is set to video.src when the XMLHttpRequest is done. When I did some initial testing of Firefox Reality, I noticed that these downloads take much longer than they should given the available bandwidth of the connection. I don't believe I see this on Desktop Firefox.

  3. Slower decode performance on Firefox - On all platforms I see lower decode performance on Firefox (the non-MSE code uses Seek() and FastSeek() - both perform the same). I think this is likely due to some low level GPU decoding / WebGL code in Firefox, but if you'd like to look into that, then I'd be happy to help get more debug / info. Some of this can be seen with this JSFiddle - https://jsfiddle.net/dustinkerstein/qz9ka6xm/

  4. e10s issues - I've not personally seen this issue. I believe I have e10s enabled as per my about:support page showing -
    "Multiprocess Windows 1/1 (Enabled by default) Web Content Processes 5/4" Running on "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:65.0) Gecko/20100101 Firefox/65.0" - But let me know if there's a specific way to test the bug you're seeing.

Let me know if there's anything else I can help provide.

Thank you Dustin! Maire, I'll kick it back over to you to find the right folks to dig into the issues Dustin has raised in his response.

Flags: needinfo?(mreavy)
Flags: needinfo?(larsberg)
Flags: needinfo?(cvan)

Thanks for that!

The video isn't playing because Gecko is reporting video.readyState === video.HAVE_FUTURE_DATA, not video.HAVE_ENOUGH_DATA. Switching that value over (or using readyState >= HAVE_FUTURE_DATA) makes it play correctly.

It looks like we don't know the total duration of the video (since it's MSE), and we consider anything less than 10 seconds of buffered video to be 'future' and not 'enough'.

The relevant spec texts looks to be this: https://w3c.github.io/media-source/#buffer-monitoring

It sounds like HAVE_ENOUGH_DATA is an implementation specific decision on when we think we have enough data to play uninterrupted to the end (though the example given there is 500ms, and we use 10000ms).

I think HAVE_FUTURE_DATA is the right condition to be checking to determine if the the current frame could be displayed, or possibly even HAVE_CURRENT_DATA.

From the log attached what that code does is seek to 0, load 0-1, then seek to 1, load 1-2, then seek to 2, load 2-3 and repeat.

play() has never been called.

There's no spec related to how to determine HAVE_ENOUGH_DATA. We use 10s with MSE for continuous data ahead of currentTime.

And if all the code need to know is if there's a frame to display, then here waiting for the "seeked" event or "loadeddata" is what you need, be spec-compliant and work across browser.

It doesn't make any sense to wait for canplaythrough

(In reply to dustin.kerstein from comment #22)

  1. Slower downloads on Firefox - With the non-MSE PanoMoments code, the JS will download a portion (the first frame) of the mp4 via Fetch and append that as a blob to the video.src while the rest of the file downloads via a XMLHttpRequest. Then the entirety of the mp4 blob is set to video.src when the XMLHttpRequest is done. When I did some initial testing of Firefox Reality, I noticed that these downloads take much longer than they should given the available bandwidth of the connection. I don't believe I see this on Desktop Firefox.

It would be more efficient to use ReadableStream / Fetch in combination with MSE.

This would allow for lower latency start.

  1. Slower decode performance on Firefox - On all platforms I see lower decode performance on Firefox (the non-MSE code uses Seek() and FastSeek() - both perform the same). I think this is likely due to some low level GPU decoding / WebGL code in Firefox, but if you'd like to look into that, then I'd be happy to help get more debug / info. Some of this can be seen with this JSFiddle - https://jsfiddle.net/dustinkerstein/qz9ka6xm/

On windows, that's due to the low latency decoding being enabled in Chrome.
You can enable it in Firefox in about:config and setting the pref:
media.wmf.low-latency.enabled

With that pref on I get higher frame/s value in Firefox with your fiddle for 1080p (56fps vs 58fps), and about the same for 4k (57.6fps vs 57.8fps)

But more importantly, you need to listen to the proper event to determine if a frame is ready.

Depends on: 1305340
Flags: needinfo?(mreavy)

Thanks. The reason for testing the video state was due to how Firefox + WebGL/Three.js would render a black frame if you called texture.needsUpdate before the video frame was actually ready (this breaks the PanoMoments viewer UX). Chrome behavior is different - it would just leave the previous frame data visible (which is fine for the UX). Relying on the seeked/loadeddata events doesn't work for Firefox as they're not guaranteed to have fired before the frame is rendered on screen. I will try testing out video.HAVE_FUTURE_DATA in the next few days once I have some good internet. Even the video state method isn't full-proof in the non-MSE implementation on Firefox. Under load I can still get a black flicker sometimes. I can try to get you a JSFiddle in a week or so that illustrates this black frame issue, but let me know if anything else comes to mind as far as a more foolproof way to have a "video frame displayed/rendered callback" as that's what I'm really after - though there is no spec defined that provides that to my knowledge.

As for the original e10s bug, is that still reproducible? If so, how can I get it setup?

(In reply to dustin.kerstein from comment #28)

Thanks. The reason for testing the video state was due to how Firefox + WebGL/Three.js would render a black frame if you called texture.needsUpdate before the video frame was actually ready (this breaks the PanoMoments viewer UX). Chrome behavior is different - it would just leave the previous frame data visible (which is fine for the UX). Relying on the seeked/loadeddata events doesn't work for Firefox as they're not guaranteed to have fired before the frame is rendered on screen. I will try testing out video.HAVE_FUTURE_DATA in the next few days once I have some good internet. Even the video state method isn't full-proof in the non-MSE implementation on Firefox. Under load I can still get a black flicker sometimes. I can try to get you a JSFiddle in a week or so that illustrates this black frame issue, but let me know if anything else comes to mind as far as a more foolproof way to have a "video frame displayed/rendered callback" as that's what I'm really after - though there is no spec defined that provides that to my knowledge.

testing HAVE_FUTURE_DATA or whatever else will not make a difference as this indicates how much data is buffered, it has nothing to do with something having painted or not.

We specifically delay the loadeddata event until the frame is painted. I don't believe we do the same for the seeked event ; we could/should

HAVE_FUTURE_DATA does seem to allow me to enable the MSE code-path on Firefox, which is great. But we'll have to create a new path just for Firefox as Chrome doesn't behave the same way with HAVE_FUTURE_DATA. We'll try to get this tested (across several FF versions) over the next few weeks. I'll post updates here when ready.

Yah, having the seeked event wait until the frame is actually painted would be ideal for my use-case.

Getting FF to support Low Latency (and frame accurate) decoding on Windows will be great.

Let me know if that e10s bug is still there though - I haven't seen it and would love to replicate if possible.

(In reply to dustin.kerstein from comment #30)

HAVE_FUTURE_DATA does seem to allow me to enable the MSE code-path on Firefox, which is great. But we'll have to create a new path just for Firefox as Chrome doesn't behave the same way with HAVE_FUTURE_DATA. We'll try to get this tested (across several FF versions) over the next few weeks. I'll post updates here when ready.

AFAIK, Chrome doesn't handle HAVE_FUTURE_DATA and HAVE_ENOUGH_DATA differently, it's one and the same.

HAVE_FUTURE_DATA for us, is as per-spec which is based on the amount of data buffered, not amount of data decoded.

Anyhow, glad it works. And it should work on all release version of Firefox.

Yah, having the seeked event wait until the frame is actually painted would be ideal for my use-case.

Getting FF to support Low Latency (and frame accurate) decoding on Windows will be great.

This is now enabled in Firefox Nightly on Windows 10.

HAVE_FUTURE_DATA for us, is as per-spec which is based on the amount of data buffered, not amount of data decoded.

Yes, but as we only append 1 frame's worth of bytes to the MSE buffer at a time, the HAVE_ENOUGH_DATA state in Chrome becomes a reasonably "proxy" for a decoded frame callback - it's be best we've been able to find...

I just did a bit more testing with the MSE path on Firefox 66 on OSX, and while it does work, it's not providing any decoding performance advantages; in fact it's actually much slower decoding than the simple Seek() method. Chrome behaves differently where the MSE path is actually faster decoding than Seek(). So I'm not sure it'd be worth enabling the MSE path on Firefox at the current time until we understand why the decoding is degraded. The initial "time to view" will be better as it's "streaming" with MSE, but the decode performance is more important IMO.

I'll try to get a Windows machine to test on in a few days to see if the above holds there as well.

(In reply to dustin.kerstein from comment #30)

Let me know if that e10s bug is still there though - I haven't seen it and would love to replicate if possible.

I believe the e10s bit of this bug title is specific to GeckoView/Android, and is related to the UA detection giving that version the (currently broken) MSE path.

HAVE_FUTURE_DATA does seem to allow me to enable the MSE code-path on Firefox, which is great. But we'll have to create a new path just for Firefox as Chrome doesn't behave the same way with HAVE_FUTURE_DATA. We'll try to get this tested (across several FF versions) over the next few weeks. I'll post updates here when ready.

Can you look for >= HAVE_FUTURE_DATA (since it's an int) to cover all the possible ready states that have data available?

I believe the e10s bit of this bug title is specific to GeckoView/Android, and is related to the UA detection giving that version the (currently broken) MSE path.

Ah ok. So it's not really about "multi-process Firefox" - it's just the MSE code path not triggering. Got it :)

Can you look for >= HAVE_FUTURE_DATA (since it's an int) to cover all the possible ready states that have data available?

I just tried that on https://jsfiddle.net/dustinkerstein/eujpb60b/ and it's the same behavior. Try comparing this fiddle on Firefox vs. Chrome. For me on OSX I see Chrome rendering at 60fps. On Firefox (with it set to >= HAVE_FUTURE_DATA) I see 5-10fps.

Also try removing the video.readyState >= video.HAVE_FUTURE_DATA condition in the Render loop and you'll see the "black flashing texture" issue on Firefox (though the true decoding speed isn't any better). On Chrome the black flashing doesn't happen - though when testing higher resolutions / lower device specs Chrome will choke / freeze as it is trying to decode too fast unless you have that readyState condition.

Are you able to replicate those numbers/behaviors?

(In reply to dustin.kerstein from comment #32)

HAVE_FUTURE_DATA for us, is as per-spec which is based on the amount of data buffered, not amount of data decoded.

Yes, but as we only append 1 frame's worth of bytes to the MSE buffer at a time, the HAVE_ENOUGH_DATA state in Chrome becomes a reasonably "proxy" for a decoded frame callback - it's be best we've been able to find...

Oh I see, I only looked at the 1s bit, so it's a single frame but one second long.
So yes, we would fire canplay while chrome would wait to decode.

I just did a bit more testing with the MSE path on Firefox 66 on OSX, and while it does work, it's not providing any decoding performance advantages; in fact it's actually much slower decoding than the simple Seek() method. Chrome behaves differently where the MSE path is actually faster decoding than Seek(). So I'm not sure it'd be worth enabling the MSE path on Firefox at the current time until we understand why the decoding is degraded. The initial "time to view" will be better as it's "streaming" with MSE, but the decode performance is more important IMO.

We'll investigate on mac.
On Windows, at least on my machine, speed is similar with a slight gain in Firefox (2-3%).

It looks like the reason seeking is so slow is because the video has the (default) preload="metadata" attribute, and the video is paused, so we shut down decoders at every opportunity to save resources. Every time you seek we have to create a new decoder, which is really slow on OSX.

That's maybe not ideal behaviour for your use case, we can consider trying to find a better balance.

In the short term adding "video.preload = "auto";" to the fiddle (straight after creating the video element, the same time as assigning src) makes a massive difference to performance.

There's also a bit of slowness when copying video textures into WebGL, where we delete the texture handle immediately, and that appears to be synchronously flushing.

That probably doesn't affect actual upload time, but it does change how long we spend inside the texImage2D call, so it prevents other JS code from running. Not an issue for this fiddle, but could be reducing performance on real-world workloads.

(In reply to Matt Woodrow (:mattwoodrow) from comment #36)

It looks like the reason seeking is so slow is because the video has the (default) preload="metadata" attribute, and the video is paused, so we shut down decoders at every opportunity to save resources. Every time you seek we have to create a new decoder, which is really slow on OSX.

That's maybe not ideal behaviour for your use case, we can consider trying to find a better balance.

In the short term adding "video.preload = "auto";" to the fiddle (straight after creating the video element, the same time as assigning src) makes a massive difference to performance.

Wow, adding that does make a massive difference on OSX. I still need to use the readyState condition to protect against that black flickering (it would be great if it just left the previous texture visible rather than clearing it to black...) but the performance now does seem to be on par with Chrome MSE on that test fiddle. I don't have access to Firefox Reality, but given this workaround, I feel pretty good about deploying to production now. Would you guys be able to test that today / tomorrow on Reality?

There's also a bit of slowness when copying video textures into WebGL, where we delete the texture handle immediately, and that appears to be synchronously flushing.

Yah, that seems like it would be a good optimization at a later time. And on shared gpu memory mobile platforms, is it possible to do zero-copy for video decode?

(In reply to Jean-Yves Avenard [:jya] from comment #35)

Oh I see, I only looked at the 1s bit, so it's a single frame but one second long.
So yes, we would fire canplay while chrome would wait to decode.

Yah, the video file's framerate is meaningless for PanoMoments decoding. We chose 1 fps only because it makes for simple math and reduces any possible rounding errors. But this does rely on the decoders not enforcing any timing delays when seeking via Seek() or with MSE - though we haven't seen any issues with this yet.

FYI, MSE is now enabled in production for PanoMoments on all versions of Firefox. Thanks!

It definitely should be tested a bit on other platforms (Windows, Reality, etc.) and versions, but it's looking pretty good on OSX Firefox 66.

Lars, can you please test this on FirefoxReality to see if it's working as expected now?

Thanks so much Dustin!

Flags: needinfo?(larsberg)

(In reply to Matt Woodrow (:mattwoodrow) from comment #41)

Lars, can you please test this on FirefoxReality to see if it's working as expected now?

I think I jumped the gun a bit. Testing on Android Firefox (not Reality) appears to be quite broken with the MSE path. Is the Android build pretty much the same as the Reality?

Yah, give this fiddle (with HAVE_FUTURE_DATA) a try on Android Firefox (and probably also Reality) - https://jsfiddle.net/dustinkerstein/4rtLvnmd/ - Something funky is going on there (decode < .25fps and black screens). Any idea what could be going on there?

(In reply to dustin.kerstein from comment #43)

Yah, give this fiddle (with HAVE_FUTURE_DATA) a try on Android Firefox (and probably also Reality) - https://jsfiddle.net/dustinkerstein/4rtLvnmd/ - Something funky is going on there (decode < .25fps and black screens). Any idea what could be going on there?

On Android, preload=auto == metadata, and you're seeing a side effect of bug 1499224, where it takes over 2s to create a decoder.

Let me get back to you while I think of a workaround.

(In reply to Jean-Yves Avenard [:jya] from comment #44)

On Android, preload=auto == metadata, and you're seeing a side effect of bug 1499224, where it takes over 2s to create a decoder.

Ok, I will disable MSE on Android Firefox (and hopefully also catch Reality due to it being Android...) on production until figure out a workaround.

(In reply to Jean-Yves Avenard [:jya] from comment #44)

(In reply to dustin.kerstein from comment #43)

Yah, give this fiddle (with HAVE_FUTURE_DATA) a try on Android Firefox (and probably also Reality) - https://jsfiddle.net/dustinkerstein/4rtLvnmd/ - Something funky is going on there (decode < .25fps and black screens). Any idea what could be going on there?

On Android, preload=auto == metadata, and you're seeing a side effect of bug 1499224, where it takes over 2s to create a decoder.

what about preload=none ?

It's a bug of ours IMHO, but interestingly, the resource saving mode is only used if preload is set to metadata. So that should work for your case.

The other option is set the muted attribute to your video element and call play/pause on it.

so adding:
video.preload = "none";
video.muted = 1;
video.addEventListener('loadedmetadata', () => { video.play(); video.pause(); });
after setting src.

Preload none seems to be the same as auto. I'll try the other method shortly.

And the same for the play/pause method on Android (OSX/Windows work fine with both methods and preload=auto). Here is a fiddle - https://jsfiddle.net/51z4kgwx for the play/pause method. Can you replicate on Android? Are we still seeing bug 1499224, or is this something else?

Just make sure to request the desktop site in Android Firefox so you can see the full window.

I can reproduce on desktop without the preload line.

HTMLMediaElement::play() returns a Promise, you need to wait for that to resolve before pausing again.

This seems to work for me:

video.addEventListener('loadedmetadata', async () => {
try {
await video.play();
console.log("playing");
video.pause();
} catch(err) {
console.log("failed");
}
});

That code appears to do the same thing on Android Firefox - it hiccups through the first few frames and then freezes. It never plays smoothly (same as all other above methods on Android Firefox + MSE). Tested via https://jsfiddle.net/3okh0c94/1/ - Does this fiddle work for you on Android Firefox?

cvan, can you please investigate this fix? Thanks!

Flags: needinfo?(larsberg) → needinfo?(cvan)

Hey! Just wanted to check in. Is there anything else I can help get?

(ni? Matt for Comment #51 -- assuming that was for you)

Flags: needinfo?(matt.woodrow)

Does Panomoments work now? Randall thinks the issues have been fixed.

status-firefox67: affected → ?
status-firefox68: affected → ?

(In reply to Chris Peterson [:cpeterson] from comment #55)

Does Panomoments work now? Randall thinks the issues have been fixed.

The MSE viewer still doesn't work on Android Firefox / Reality. It does work on Desktop Firefox (and is enabled on production PanoMoments.com for non-mobile devices) with the preload=auto workaround, but Android Firefox / Reality doesn't respond the same way as the Desktop version.

Try this JSFiddle on Android Firefox - https://jsfiddle.net/dustinkerstein/4rtLvnmd/ and compare to Desktop Firefox. On my devices I see <1fps decoding (and other black frame glitches - though that's a different issue). Are you able to replicate that behavior?

jya, did you manage to figure out how we could get the preload=auto behaviour to work on mobile?

Flags: needinfo?(matt.woodrow) → needinfo?(jyavenard)

(In reply to Matt Woodrow (:mattwoodrow) from comment #57)

jya, did you manage to figure out how we could get the preload=auto behaviour to work on mobile?

:cpearce just pushed bug 1540573 which will make android behave like desktop in regards to buffering and preload default when connected to a wifi network.

Not a 100% solution, but should address most problems.

Depends on: 1540573
Flags: needinfo?(jyavenard)

jya, thanks for the update. It looks like nsINetworkLinkService is used to test wifi/cellular internally, but from here https://developer.mozilla.org/en-US/docs/Web/API/NetworkInformation/type it appears navigator.connection.type isn't supported by Firefox Android. Is that correct? Do you have any suggestions on how to best test this condition? Thanks!

(In reply to dustin.kerstein from comment #59)

jya, thanks for the update. It looks like nsINetworkLinkService is used to test wifi/cellular internally, but from here https://developer.mozilla.org/en-US/docs/Web/API/NetworkInformation/type it appears navigator.connection.type isn't supported by Firefox Android. Is that correct? Do you have any suggestions on how to best test this condition? Thanks!

You can use navigator.connection.type from JS in Firefox for Android to determine what type of connection the browser is running under.

More specifically, the documentation on MDN appears to be out of date.

K, thanks for the clarification on the MDN documentation.

I just tried out today's Firefox Nightly on Android with a good WiFi connection and still see the same broken playback (jittery ~1fps on average and eventual complete stall) when testing https://jsfiddle.net/dustinkerstein/4rtLvnmd - Do you see the same? Do you think there's something else causing this?

I'm editing a bunch of GeckoView bugs. If you'd like to filter all this bugmail, search and destroy emails containing this UUID:

e88a5094-0fc0-4b7c-b7c5-aef00a11dbc9

Priority: P1 → P2
Rank: 15
Whiteboard: [geckoview:fxr:p1]

We are planning to enable e10s by default in the v7 release of FxR. I tested panomoments with e10s enabled and while they load slowly they do load and work. Should probably close this unless we want to morph it into figuring out why the content loads slowly.

Randall, yeah they are loading slowly due to Firefox Android (and Reality) still using the non-MSE path. I'd love to enable MSE for those devices, but the workarounds for desktop (posted above) don't seem to work on Android. You can try this test on Android Firefox - https://jsfiddle.net/dustinkerstein/4rtLvnmd - Please do let me know if you can think of any other workarounds or whether this does seem like an Android MSE bug in Firefox.

Hey all, I just wanted to sync up. Is there any further debug I can get you? We'd love to get MSE enabled on Android Firefox and are happy to help in any way we can. Thanks!

I confirm that it's loading on Firefox Reality, although both https://my.panomoments.com/u/dustinkerstein/m/split-croatia-harbor?hideInfo=1 and the latest example https://jsfiddle.net/dustinkerstein/4rtLvnmd are far from being usable here.
It runs as something like 2fps? and in the panomoments example I just get black screen flickering and a big delay when moving my head around in VR.

There may be a new issue with Firefox Reality + PanoMoments on first load (switching quality after load I think might help - but something is still broken there which I'll investigate). But if we can figure out why that JSFiddle works correctly on Firefox Desktop but not the mobile Firefox app, then that'll be a huge win and we could enable MSE on PanoMoments for Reality and the mobile Firefox app.

Side note - That specific fiddle won't work correctly on Chrome/Safari due to the video.readyState === video.HAVE_FUTURE_DATA condition. Those browsers require video.readyState === video.HAVE_ENOUGH_DATA.

Hi, sorry to keep bugging you guys. I know you have more pressing issues. We've gotten PanoMoments to work perfectly on desktop Firefox, but feel pretty terrible about the way things are on the mobile / VR versions. Is there anything else I can get you to help debug what's going on? Do you have any other possible workarounds we could try?

Thanks,
Dustin

Sorry to hijack this thread like this - but there seems to be a major WebGL texture upload performance regression over the last 6 months or so. I created a bug over at MozillaReality - https://github.com/MozillaReality/FirefoxReality/issues/3973 and here - https://bugzilla.mozilla.org/show_bug.cgi?id=1736923

Would someone be able to please take a quick look to see how widespread that could be? Thanks.

Flags: needinfo?(etoop)

Moving this to the WebGL team & resetting priority as I don't think it was ever a bug in GeckoView itself. Can someone please take a look?

Rank: 15
Component: General → Canvas: WebGL
Flags: needinfo?(etoop)
Priority: P2 → --
Product: GeckoView → Core

Clear a needinfo that is pending on an inactive user.

For more information, please visit auto_nag documentation.

Flags: needinfo?(hearcomestreble+bz)

Sorry, there was a problem with the detection of inactive users. I'm reverting the change.

Flags: needinfo?(drno)

(In reply to Suhaib Mujahid [:suhaib] from comment #74)

Sorry, there was a problem with the detection of inactive users. I'm reverting the change.

TBH I'm not sure what is asked of me here :-)

Flags: needinfo?(drno)

(In reply to Nils Ohlmeier [:drno] from comment #75)

TBH I'm not sure what is asked of me here :-)

This is the original question: https://bugzilla.mozilla.org/show_bug.cgi?id=1529812#c12

(In reply to Suhaib Mujahid [:suhaib] from comment #76)

(In reply to Nils Ohlmeier [:drno] from comment #75)

TBH I'm not sure what is asked of me here :-)

This is the original question: https://bugzilla.mozilla.org/show_bug.cgi?id=1529812#c12

Ha. Got it, thanks.
Since I no longer work on any of this I'm not sure who is the right person to judge if this should be closed or continued to be pursued.

Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: