DASH: no audio if audio encoded with track number other than 1

RESOLVED FIXED in mozilla21

Status

()

Core
Audio/Video
RESOLVED FIXED
6 years ago
5 years ago

People

(Reporter: nwgh, Assigned: sworkman)

Tracking

Trunk
mozilla21
Points:
---

Firefox Tracking Flags

(relnote-firefox -)

Details

Attachments

(1 attachment, 2 obsolete attachments)

(Reporter)

Description

6 years ago
Using the directions at http://wiki.webmproject.org/adaptive-streaming/instructions-to-playback-a-webm-dash-presentation to create a DASH video results in no audio being played. Changing the audio_track_number to 1 (from 2, the value used in the directions) results in audio being played. We should maybe ignore the audio track number when there's only one audio track available.
(Assignee)

Updated

6 years ago
Status: NEW → ASSIGNED
(Assignee)

Updated

6 years ago
Blocks: 702122
Component: Networking → Video/Audio
(Assignee)

Comment 1

6 years ago
Created attachment 693223 [details] [diff] [review]
v1.0 Use TrackNumber to reference tracks instead of their index

Currently ne_read_block fails if it reads a TrackNumber value greater than the number of tracks in the stream. With DASH-WebM, if the audio WebM file has one track encoded to track number 2, ne_read_block will fail and thus the audio will not play.

Current solution/workaround has been to encode the video and audio files with TrackNumber=1. This is fine, but it raises a possible incompatibility with DASH-WebM files encoded for Chrome/MediaSource. Instructions from http://wiki.webmproject.org/adaptive-streaming/instructions-to-playback-a-webm-dash-presentation suggest that audio and video should be encoded with different TrackNumbers, since MediaSource doesn't work if they are the same. While this may be a limitation of MediaSource and something that may need to be fixed there, I don't see a reason for the current restriction in nestegg that TrackNumbers be no greater than the total track count in a file.

Matroska Spec: http://matroska.org/technical/specs/index.html
WebM Spec: http://www.webmproject.org/docs/container/

I only see a restriction that TrackNumber can't be 0. If that's the case, then we can use any TrackNumber internally, and then get compatibility with DASH-WebM files encoded for Chrome/MediaSource.

This patch changes nestegg.c to do just that, and adds a function to get an array of TrackNumbers for external reference, i.e. WebMReader.
Attachment #693223 - Flags: review?(kinetik)
(Assignee)

Comment 2

6 years ago
Created attachment 693225 [details] [diff] [review]
v1.0 Use TrackNumbers in WebMReader to reference tracks

Uses nestegg_get_track_numbers to get reference values for audio/video tracks encoded in WebM media.
(Assignee)

Comment 4

6 years ago
Matthew, there may be other reasons to maintain nestegg the way it is - if so, let me know and I can try to work out something less dangerous.
Comment on attachment 693223 [details] [diff] [review]
v1.0 Use TrackNumber to reference tracks instead of their index

The public API is was designed to expose normalized zero-based track numbers.  I'm not sure I want to change that right now.

The check in ne_read_block is certainly bogus because there's no requirement for the track numbering to start at 1 or even be contiguous.  The current normalization simply subtracts 1 from the container's track numbers, which is bogus for the same reason.

I think the right approach for now is to establish a track number mapping inside nestegg, so that the public API remains untouched.
Attachment #693223 - Flags: review?(kinetik) → review-
(Assignee)

Comment 6

6 years ago
(In reply to Matthew Gregan [:kinetik] from comment #5)
> Comment on attachment 693223 [details] [diff] [review]
> v1.0 Use TrackNumber to reference tracks instead of their index
> 
> The public API is was designed to expose normalized zero-based track
> numbers.  I'm not sure I want to change that right now.

Okie doke - makes sense.

> The check in ne_read_block is certainly bogus because there's no requirement
> for the track numbering to start at 1 or even be contiguous.  The current
> normalization simply subtracts 1 from the container's track numbers, which
> is bogus for the same reason.

Good - I read that right then.

> I think the right approach for now is to establish a track number mapping
> inside nestegg, so that the public API remains untouched.

Sounds like a plan.
(Assignee)

Comment 7

6 years ago
Created attachment 693723 [details] [diff] [review]
v2.0 Allow values for TrackNumber > 0; map TrackNumber to external index

-- Allows values for TrackNumber other than 0 in ne_read_block.

-- Provides a new internal function ne_map_track_number_to_index to map TrackNumber to the external index/reference value. This index is then stored in the nestegg packet, which is used externally. The index should have the same value externally, with or without the patch (discounting the error cases this patch aims to fix).

I have tested it with single stream audio files, multiplexed audio/video files, and DASH audio/video files - all with at least one TrackNumber > TrackCount. All pass.

Push to try for all mochitests:
https://tbpl.mozilla.org/?tree=Try&rev=5ae7183845c2
Attachment #693223 - Attachment is obsolete: true
Attachment #693225 - Attachment is obsolete: true
Attachment #693723 - Flags: review?(kinetik)
Attachment #693723 - Flags: review?(kinetik) → review+

Comment 9

6 years ago
https://hg.mozilla.org/mozilla-central/rev/aeacf3c6231b
Status: ASSIGNED → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla21
relnote-firefox: --- → ?

Updated

5 years ago
relnote-firefox: ? → -
You need to log in before you can comment on or make changes to this bug.