1041233 - Allow silent WebM files where animGIFs are allowed

Status: REOPENED

(Core :: Audio/Video: Playback, enhancement, P3)

Description

[Kornel]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:33.0) Gecko/20100101 Firefox/33.0 (Beta/Release)
Build ID: 20140704030208

Steps to reproduce:

<img src="video.webm">

div {background: url(video.webm)}



Actual results:

Browsers have an artificial distinction between GIF and other video formats. GIF videos are allowed to play when embedded as images in HTML and CSS, but more efficient codecs aren't.

The distinction seems to be a historical accident, as it doesn't make sense to give the most privileged native support to the worst video codec.

<img src="video.webm"> should play the video just as well as <img src="video.gif"> does (with no sound, no controls, looped).



Expected results:

I'm advocating for giving other video codecs the same level of support as animGIFs enjoy.

AnimGIF as a medium for sharing short, silent video clips is very successful, but as a video codec it's rubbish.

My hope is that if browsers supported other video formats in all places where they support GIFs (with the same unobtrusive UI) then page authors would be able to implement "GIFs" with more efficient codecs.

Because neither <video> nor <img>/<picture> support both GIF and WebM at the same time it's unnecessarily difficult to support both.

I've written more about it here: https://pornel.net/efficient-gifs

Comment 1

[Eric Portis]

This issue appears to be a duplicate/special case of https://bugzilla.mozilla.org/show_bug.cgi?id=895131.

I'll go ahead and add the same notes here that I did there, here, though.

Safari shipped this earlier this year: https://bugs.webkit.org/show_bug.cgi?id=176825 (to much fanfare: https://calendar.perfplanet.com/2017/animated-gif-without-the-gif/)

Blink has had a lot of discussion about this recently, including a prototype implementation: https://bugs.chromium.org/p/chromium/issues/detail?id=791658&q=webp%20accept&colspec=ID%20Pri%20M%20Stars%20ReleaseBlock%20Component%20Status%20Owner%20Summary%20OS%20Modified

Anybody interested in picking it up?

Comment 2

[Paul Adenot (:padenot)]

Is there a spec I can have a look at ? Safari shipped without speccing anything afaik, but we don't do that.

Comment 3

[Eric Portis]

What part of this would you like more spec info around?

If Firefox implements this, it should begin broadcasting support for video formats in <img> requests via the `Accept` header. `Accept` is well specified: https://tools.ietf.org/html/rfc7231#section-5.3.2

As far as what happens once the WebM (or MP4 or whatever) is loaded over the network – I believe it can be handled just like animated GIFs are now, using the processing steps here: https://html.spec.whatwg.org/multipage/images.html#images-processing-model

Perhaps we could use some language about what UAs should do with "image" files that contain sound information?

Anything else?

Comment 4

[Paul Adenot (:padenot)]

This is not exhaustive, but having a quick look at the links in comment 3, I'd expect answers to the following:
- what to do with audio content, as you mention
- how to react to intrinsic video dimensions changes
- check if we need to do anything special for live-sources (`load` will probably never fire or something, and we know this to be the case ahead of time, with the Content-Length header or maybe with in-band information in the file)
- at least review the image processing model and loading section of HTML to make sure we don't see anything incompatible [0]. In particular, there is a few synchronous section that should be reviewed.

What I find the most appealing here is the fact that we could reference videos from CSS, this could be a win both in terms of bandwidth, memory and cpu usage.

[0] It _should_ be fine, but codecs sometimes have weird things. I'm thinking something like special files that require a range-request for playback, although this might just be for building a seek table, does not apply if we're just playing.

Comment 5

[Jean-Yves Avenard [:jya]]

IMHO, if you want to display a video, then use the <video> element which is precisely designed to handle this case.

Making <img> a potential replacement of <video> is outside the scope of what an "image" should be.

Just because animated gif was created a long time ago and could somehow do animations, then <video> was created to do things properly.

Let's not go back in time.

Comment 6

[Jean-Yves Avenard [:jya]]

The blog article quoted : https://calendar.perfplanet.com/2017/animated-gif-without-the-gif/#-but-we-already-have-video-tags

"Replacing GIFs with <video> is better but has perf. drawbacks: not preloaded, uses range requests"

None of this is correct, there's no condition where using an img element would have technical advantage over using a video element.
A video can be preloaded, and range-request would only be necessary if the video is too big and badly muxed.
Even if it was an image, you may have to use range-request (like an mp4 where the metadata can only be found at the end of the file). Or a webm where you can't know the duration until you've retrieved the last image and read its timestamp.

While you may have to write something different, all of what you can do with an image can be done with a video.

Comment 7

[Colin Bendell]

(In reply to Jean-Yves Avenard [:jya] from comment #6)
> "Replacing GIFs with <video> is better but has perf. drawbacks: not
> preloaded, uses range requests"
> 
> None of this is correct, there's no condition where using an img element
> would have technical advantage over using a video element.
> A video can be preloaded, and range-request would only be necessary if the
> video is too big and badly muxed.

Original author here. (read: please excuse any knee-jerk defensiveness)

I think you might be confusing 'preload' in this context. The chrome/safari/firefox don't include video tags in the preload tokenizer. Only <img> tags are included. You have to wait until the loader parses the dom. To confuse the situation, <video> tags do have a preload attribute, but it has nothing to do with the speculative parser/preloader thread. (see https://hg.mozilla.org/releases/mozilla-beta/file/tip/parser/html/nsHtml5SpeculativeLoad.cpp ) You can see only <img> and <picture> tags are included. <video> is not.

WRT range requests, this is different between browsers. Safari/WebKit always makes range request for any <video> tag. WebKit makes a 0-1 byte range request (requiring 1 RTT) to detect if the server can support range requests. Then it follows up with a second chunk ~16k. This is an optimization for long form video to prevent loading large volumes of video that may not be displayed to the user. For looping cinemagraphs / animated gifs, this is not the use case.

In the end, the advantage of the <img> tag is 1) it makes sense to developers (it is easy to adopt) 2) it has an explicit contract with the user & display - it is consistent and has a limited feature set - it is muted, you don't show the scrubber, etc and 3) it is more efficient with mobile environments since it uses the existing browser's preload/speculative loader capabilities and optimizes the congestion window growth and therefore efficiently uses the data-plane resource time allocation on a cellular tower rather than requiring multiple time schedules (caused by delays in the parser).