563574 - Create video key-frame index and cache it for future smooth seek.

Status: RESOLVED WONTFIX

(Core :: Audio/Video, enhancement)

Description

[Biju]

Create video key-frame index and cache it for future smooth seek.

One of the annoying problem with HTML5 VIDEO is the irritating throbber appearing when you seek a video. This happens even when we seek back to a section of video which was already just played.

Chris Pearce has already start working on it in Bug 519897 
It will fix all video which has already embedded index info in the media file.

But all the other video without embedded index info this wont work
So I have an alternate suggestion.

* Please maintain index information of the media being played at browser side.
  ie, create an index on the fly, soon after we fetch content to the buffer.
  This will allow seeking easier till the range we have buffed 
  and also on all the content we have already played

* Cache this index in browser cache to use next time user visit/play same file.
  Most probably user seek and watch only parts of the video.

* use a BINARY format for the cache file to reduce file size 

* index file dont have to be precise to nth byte, 
  it can just say the frame is at nth kilobyte block (or any block size)
  this will also reduce size of the cache file 

* if possible ask WHATWG to add a new attribute named "FRAMEINDEX"
  to <VIDEO/AUDIO> tag to provide pre-cooked location of the indexfile 
  in HTML source.
  (if that succeed, provide a Firefox UI for HTML authors 
    to generate indexfile of a playing video)

Comment 1

[cajbir (:cajbir)]

(In reply to comment #0)
> One of the annoying problem with HTML5 VIDEO is the irritating throbber
> appearing when you seek a video. This happens even when we seek back to a
> section of video which was already just played.

Seeking back to a section of just played video should be nearly instantaneous due to the media cache. Is it not doing this for you?

Comment 2

[Biju]

here is the steps to reproduce 

1. goto http://people.mozilla.com/~prouget/demos/mashup/video.xhtml
2. click play
3. allow to play 15 seconds
4. click on somewhere 20 sec of video
   - user see throbber
5. click on somewhere 10 sec of video
   (which was already played)
   - user see throbber again

Comment 3

[cajbir (:cajbir)]

What you're seeing there is buggy 'we need to buffer' detection code from the looks of things. We seek instantaneously and then the video thinks the download rate isn't enough to play.

Comment 4

[Biju]

dont you think all those problem will be solved if we cache the index.
Also if I remember correct video buffer also has some limit right?
ie, if the video is of huge size it will clear buffer once played.

Comment 5

[cajbir (:cajbir)]

I think building a dynamic index is a good thing if we don't already do it (cpearce would know).

I think that is a separate issue from the throbber thing you are seeing. The throbber shouldn't appear during seeks in the media cache since that should be very fast (and it is).

Comment 6

[Chris Pearce [:cpearce (Not reading bugmail)]]

Building an index is a good idea, provided we put bounds on it as we go. We don't build one at the moment, but we do determine the time ranges held in any already cached data before we seek; that's why seeking in cached data is instantaneous, we know whether ro seek inside it or whether to do a bisection search in un-downloaded data.

I agree with doublec, it looks like the "we need to buffer" logic has some problems.

Comment 7

[Biju]

(In reply to comment #6)
> but we do determine the time ranges held in any
> already cached data before we seek;

So why not save it in a cache file, for future use

Comment 8

[cajbir (:cajbir)]

(In reply to comment #7)
> So why not save it in a cache file, for future use

It's a possibility but brings in a whole lot of other questions. How do the cached entries get expired, what size should the cache be, what UI should be used to allow the user to control the caching behaviour, how do you determine the cache is incorrect due to the video being updated, etc.

If we wanted to use the standard browser cache for this then we'd need to fix the issues that prevent most videos from being cached there first.

Comment 9

[Biju]

Just my suggestion 
See it like this, ideally we want the entire video and index to be cached.
because of limitation of space we cache
1. index
2. first 10 sec of video (or by size in kb/MB)
   (should be an about:config item)
>> How do the cached entries get expired, 
so same as file expire/cache-control header  

>> what size should the cache be, 
same place/size we allocated for html/css/js/image file

>> what UI should be used to allow the user to control the caching behaviour, 
no extra UI needed

>> how do you determine the cache is incorrect due to the 
>> video being updated, etc.
When you buffer/playing video again compare it with what on index file
if they mismatched there is an error

Comment 10

[Biju]

(In reply to comment #3)
> What you're seeing there is buggy 'we need to buffer' detection code from the

created a separate Bug 563589

Comment 11

[Biju]

Here are some calculation on index file size for caching on the disk.

Assume that the video file is only less than 65MB, and plays for duration of less than 12 hour. We can split the 65MB file to 1kb blocks, now assume the key frame is there on every second. Then we can allocate 2 bytes the block number and 2 bytes for duration in seconds (which will take up to 12h)
so record length for an entry is 4 bytes

So 65MB file will have 65536 records (64k)
Hence at total size of index file = 64k * 4 bytes = 256kb

now if key frame is at consistent interval we can ignore the time part in the index file. Then the size will be only  128kb

In practical case VIDEO file size will be more that 65MB, 
but key frame will not be there in ever second
So I expect we can still bring the index file size less than 256kb

Comment 12

[Daniel C]

(In reply to comment #6)
> I agree with doublec, it looks like the "we need to buffer" logic has some
> problems.

Bug number?

Comment 13

[Chris Pearce [:cpearce (Not reading bugmail)]]

(In reply to comment #12)
> (In reply to comment #6)
> > I agree with doublec, it looks like the "we need to buffer" logic has some
> > problems.
> 
> Bug number?

bug 589626

Note that bug 519897 has landed now, so we support ogg files which contain their own embedded keyframe index, which will improve support for indexed ogg files. WebM files always contain a keyframe index.

Seeking in cached data in files without an index is pretty quick, and we recently expanded the media cache so we're unlikley to be too aggressive in ejecting stuff from the media cache, so I don't think investing the time in constructing and maintaining a keyframe index is worthwhile. Resolving wontfix.