Closed Bug 1262659 Opened 4 years ago Closed 4 years ago

Measure m3u8 popularity

Categories

(Core :: Audio/Video: Playback, defect, P2)

defect

Tracking

()

RESOLVED FIXED
mozilla48
Tracking Status
firefox47 --- fixed
firefox48 --- fixed

People

(Reporter: rillian, Assigned: rillian)

References

Details

Attachments

(1 file)

Add telemetry to measure how often we fail to play an HLS m3u8 source url.
Identify HLS media types (and related m3u playlists) and report
telemetry about how often we fail to play them. This is designed
to measure popularity to inform decisions about support for this.

Review commit: https://reviewboard.mozilla.org/r/45841/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/45841/
Attachment #8740634 - Flags: review?(cpearce)
Comment on attachment 8740634 [details]
MozReview Request: Bug 1262659 - Report HTTP Live Streaming playback requests. r?cpearce

Adding bsmedburg for data collection review.
Attachment #8740634 - Flags: feedback?(benjamin)
Comment on attachment 8740634 [details]
MozReview Request: Bug 1262659 - Report HTTP Live Streaming playback requests. r?cpearce

https://reviewboard.mozilla.org/r/45841/#review43525
Attachment #8740634 - Flags: review?(cpearce) → review+
Data collection review ping.
Flags: needinfo?(benjamin)
Flags: needinfo?(benjamin)
Attachment #8740634 - Flags: feedback?(benjamin) → feedback+
Thanks, bsmedberg!
https://hg.mozilla.org/mozilla-central/rev/343ae5fe88d4
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla48
Comment on attachment 8740634 [details]
MozReview Request: Bug 1262659 - Report HTTP Live Streaming playback requests. r?cpearce

Approval Request Comment
[Feature/regressing bug #]: n/a
[User impact if declined]: HLS work incorrectly prioritised
[Describe test coverage new/current, TreeHerder]: n/a
[Risks and why]: Low - compares strings, reports telemetry, doesn't change behaviour
[String/UUID change made/needed]: none
Attachment #8740634 - Flags: approval-mozilla-beta?
Hello Anthony, Ralph: I will take this uplift to Beta47 this time. But in future, could you please fill out the test coverage section? For telemetry related uplifts, it would be good to validate that the new/updated telemetry data is coming in as expected from Nightly/Aurora channel before uplifting to Beta. Thank you!
Flags: needinfo?(giles)
Flags: needinfo?(ajones)
Comment on attachment 8740634 [details]
MozReview Request: Bug 1262659 - Report HTTP Live Streaming playback requests. r?cpearce

New telemetry probe, Beta47+
Attachment #8740634 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
Thanks, Ritu. We have been getting reasonable-seeming data from 48 and 49 over the past few weeks.
Flags: needinfo?(giles)
(In reply to Ralph Giles (:rillian) needinfo me from comment #11)
> Thanks, Ritu. We have been getting reasonable-seeming data from 48 and 49
> over the past few weeks.

That's great! Thank you for the due diligence.
Flags: needinfo?(ajones)
Blocks: 1272142
The measure might be biased, for example if I know that a website doesn't work with Firefox, I won't even open it.

Have you thought about measuring m3u8 popularity by scraping websites from the Alexa top list?
(In reply to Marco Castelluccio [:marco] from comment #14)
> The measure might be biased, for example if I know that a website doesn't
> work with Firefox, I won't even open it.
> 
> Have you thought about measuring m3u8 popularity by scraping websites from
> the Alexa top list?

We've done some Alexa top-x work here, but we hope to get interesting data on the longer tail - and scraping is unreliable stuff, it's particularly hard to get this sort of thing right.
(In reply to Marco Castelluccio [:marco] from comment #14)
> Have you thought about measuring m3u8 popularity by scraping websites from
> the Alexa top list?

I scraped the homepages of Alexa top 96,000 and found 853 unique domains with ".m3u8" in at least one of the scripts (inline or external) or HTML source. That's .888% (of that dataset).

Problem is, some of those are potentially false-positives if a video player does the right thing and detects for HLS support. 

Also, as Hallvord says we want to know how far into the long tail are people using this.
Can you share the ContentType headers the .m3u8 files were being served with? I suspect we're missing some in our telemetry check.
Flags: needinfo?(miket)
Here's Content-Type from the first 25 m3u8 files:

Content-Type: application/vnd.apple.mpegurl;charset=utf-8
Content-Type: application/x-mpegURL
Content-Type: application/vnd.apple.mpegurl
Content-Type: text/plain (but, <source src="https://cdn.theguardian.tv/HLS/2016/02/24/160223BrooklynHustings.m3u8" type="video/m3u8"></source>)
Content-Type: application/vnd.apple.mpegurl
Content-Type: application/vnd.apple.mpegurl; charset=utf-8
Content-Type: application/vnd.apple.mpegurl
Content-Type: application/vnd.apple.mpegurl
Content-Type: application/x-mpegURL
Content-Type: application/vnd.apple.mpegurl
Content-Type: application/vnd.apple.mpegurl
Content-Type: application/vnd.apple.mpegurl
Content-Type: application/vnd.apple.mpegurl; charset=utf-8
Content-Type: application/vnd.apple.mpegurl
Content-Type: application/vnd.apple.mpegurl
Content-Type: text/plain (http://cloud.imncdn.com/4224.m3u8)
Content-Type: application/vnd.apple.mpegurl
Content-Type: application/octet-stream (http://tv.life.ru/index.m3u8)
Content-Type: application/vnd.apple.mpegurl
Content-Type: application/vnd.apple.mpegurl
Content-Type: application/x-mpegURL
Content-Type: application/vnd.apple.mpegurl
Content-Type: application/x-mpegURL
Content-Type: application/vnd.apple.mpegurl
Flags: needinfo?(miket)
(ok, I can't count that's only 24 ;_;)
Ok, that should be fine. Thanks!

I added application/x-mpegurl in bug 1272142, and our telemtry should be based on case-insensitive measurements of the type portion, so x-mpegURL and ;charset=utf-8 variants will still be counted. There's not much we can do about text/plain or application/octet-stream without adding sniffing.
SGTM.

("video/m3u8" was new to me, from <source src="https://cdn.theguardian.tv/HLS/2016/02/24/160223BrooklynHustings.m3u8" type="video/m3u8"></source> -- but it only shows up on theguardian pages.)
You need to log in before you can comment on or make changes to this bug.