469923 - Support X-Content-Duration header

Status: VERIFIED FIXED

(Core :: Audio/Video, defect, P2)

Description

[Chris Pearce [:cpearce (Not reading bugmail)]]

Web servers will hopefully start sending X-Content-Duration header along with Ogg streams, which denotes the duration of the stream. We should watch out for this and use it in preference to calling oggplay_get_duration() where possible.

Comment 1

[cajbir (:cajbir)]

Header spec:

http://wiki.annodex.net/wiki/HttpHeaders

Comment 2

[cajbir (:cajbir)]

Attachment: Detect the content duration metadata headers

This patch detects for the presence of X-Content-Duration. If it exists it uses this to set the media duration of an Ogg file and avoids the need to seek to find this out. X-Content-Duration is the 'standard' used by optimized Ogg servers/content management systems that Annodex promotes. oggz-chop will also use this when sending content and this is used by the major Ogg serving sites (Wikimedia, archive.org, Metavid). 

If X-Content-Duration is not found it looks for x-amz-meta-content-duration. This is for Ogg content hosted on Amazon S3 where all custom headers must be prefixed via x-amz-meta.

If neither of these are found it performs a seek via HTTP byte range requests to get the information.

I tested this against the following URLS:

Does not support any of the headers, falls back to seeking:
http://www.double.co.nz/video_test/test5.html

Supports X-Content-Duration:
http://www.tinyvid.tv/show/t5v1inpguy9t

Supports x-amz-meta-content-duration:
http://tinyvid.tv/show/207rjoij80r8v

The format of X-Content-Duration and x-amz-meta-content-duration is a floating point number containing the duration in seconds.

Comment 3

[Robert O'Callahan (:roc) (email my personal email if necessary)]

What happens if the value of X-Content-Duration is wrong?

Comment 4

[cajbir (:cajbir)]

If X-Content-Duration is wrong then wrong values will be passed through. Maybe we could check during playback if we're playing past the current duration point and then change it, sending a durationchanged event.

I don't think we use the duration for anywhere other than reporting back to the client so it won't affect the internals of playback.

Comment 5

[Justin Dolske [:Dolske]]

Maybe ignore negative values? eg

X-Content-Duration: -4.2

or

X-Content-Duration: -1

Comment 6

[Robert O'Callahan (:roc) (email my personal email if necessary)]

It doesn't really matter what we do with an incorrect value as long as we don't crash, I guess. But we should have a test for it.

Comment 7

[cajbir (:cajbir)]

Attachment: Includes tests

Checks for negative values and includes tests.

The tests need to add headers to the response so I 'inline' the video data to send. Although it's a small video it inlines quite large. Is there a way to load or include other .js files in an .sjs file? I asked on #developers and no one seemed to know. There don't seem to be any examples of doing this anywhere.

Comment 8

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Why didn't you use a ^headers^ file?
http://mxr.mozilla.org/mozilla-central/find?string=headers

+      rv = hc->GetResponseHeader(NS_LITERAL_CSTRING("x-content-duration"), contentDurationText);

It would be better if this used the preferred capitalization, X-Content-Duration.

+        rv = hc->GetResponseHeader(NS_LITERAL_CSTRING("x-amz-meta-content-duration"), metaDurationText);

Same here. I'd prefer if the code was reorganized so that we only do one ToFloat call, by checking the value of rv and checking for headers until we find one that exists.

The code itself looks fine.

Comment 9

[cajbir (:cajbir)]

> Why didn't you use a ^headers^ file?

Because I didn't know about them? How do they work? Any docs? If someone can point to or describe how to use them I'll change it.

Comment 10

[cajbir (:cajbir)]

Ok, found some docs here: https://developer.mozilla.org/En/HTTP_server_for_unit_tests

Does this mean I still need to have a duplicate copy of the video file in the repository for each of the six tests, since I want to serve the same file with different headers?

Comment 11

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Don't know, that would be unfortunate...

Comment 12

[Jeff Walden [:Waldo]]

The intent of ^headers^ was to basically provide essentially equivalent functionality to mod_cern_meta:

http://httpd.apache.org/docs/2.0/mod/mod_cern_meta.html

The intent of it is to provide simple, static header-overriding capability for individual files, not to split headers a number of different ways.  Copying the file a bunch of times is probably your best bet, but it's nearly as easy to write one SJS and have it split behavior based on the query string.

There isn't a way to load JS files within SJS, at least not in any principled manner; please file a bug on it and we can figure something out.

Comment 13

[Jeff Walden [:Waldo]]

Another possibility: does hg support symlinks well enough everywhere to add one copy of the file and make all the others symlinks?

Comment 14

[Justin Dolske [:Dolske]]

Or use some Makefile-fu to take a single media file, make copies of it, and generate ^headers^ variations?

Comment 15

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Copying the file into multiple locations at build time sounds like the best (simplest) approach. The ^headers^ variations don't need to be dynamically generated.

Comment 16

[cajbir (:cajbir)]

I like the idea of query parameters to the sjs personally. One file to maintain, no copying. Even better if I can load the binary video file from the sjs and serve that rather than inlining the array. I assume I can do that?

Comment 17

[Robert O'Callahan (:roc) (email my personal email if necessary)]

That's good if you can load a binary file, but I don't know if you can. If you can't, inlining the video really sucks :-(

Comment 18

[cajbir (:cajbir)]

Attachment: Address review comments

I went the approach of loading the binary file.

Comment 19

[cajbir (:cajbir)]

Attachment: Update to tip

Fix bitrot from recent video patch landings. Carried for reviews.

Comment 20

[cajbir (:cajbir)]

Pushed to mozilla-central:
http://hg.mozilla.org/mozilla-central/rev/23c281c1130a

Comment 21

[cajbir (:cajbir)]

Attachment: Fix rounding errors

Comment 22

[cajbir (:cajbir)]

Pushed to mozilla-1.9.1:
http://hg.mozilla.org/releases/mozilla-1.9.1/rev/003adc9f2896
http://hg.mozilla.org/releases/mozilla-1.9.1/rev/62dd7b299b40

Comment 23

[Tony Chung [:tchung]]

Chris or Roc, Can i get some help on how to test this fix?  i tried installing livehttpheaders addon, and running it against the 3 URLs in comment 2.   the results were 1 and 3 did not return any X-Content-duration headers, and only url 2, returned a partial content of "x-amz-meta-content-duration: 313.353" in the header.

I tested this on Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.2a1pre) Gecko/20090207 Minefield/3.2a1pre (Prior build to trunk checkin), and Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.2a1pre) Gecko/20090512 Minefield/3.6a1pre which should have the fix.

But it turns out both builds returned the same set of header results i listed above.  Shouldnt they differ?

Am i testing this incorrectly, or does the fix still need work?   Thanks, Tony

Comment 24

[cajbir (:cajbir)]

The URL's in comment 2 link to the page containing the URL to the video. You should see the headers in these:

http://media.tinyvid.tv/207rjoij80r8v.ogg
http://tinyvid.tv/file/3uwvr4t3wi3rm.ogg

Note that tinyvid will have a different header depending on if it is serving from Amazon S3 or locally. So the headers returned may change in the future. I've locked these files to their current location for a couple of months so these url's are valid for this bug until then.

What you should see with these URL's is no http byte range request to find the duration when first loading the file.

Comment 25

[Tony Chung [:tchung]]

Thanks for the testcases, Chris. Verified http byte range requests duration doesnt generate when videos are loading.

Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1b5pre) Gecko/20090512 Shiretoko/3.5b5pre 
and Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.2a1pre) Gecko/20090512 Minefield/3.6a1pre