804372 - DASH: no audio if audio encoded with track number other than 1

Status: RESOLVED FIXED

(Core :: Audio/Video, defect)

Description

[u408661]

Using the directions at http://wiki.webmproject.org/adaptive-streaming/instructions-to-playback-a-webm-dash-presentation to create a DASH video results in no audio being played. Changing the audio_track_number to 1 (from 2, the value used in the directions) results in audio being played. We should maybe ignore the audio track number when there's only one audio track available.

Comment 1

[Steve Workman [:sworkman] (INACTIVE)]

Attachment: v1.0 Use TrackNumber to reference tracks instead of their index

Currently ne_read_block fails if it reads a TrackNumber value greater than the number of tracks in the stream. With DASH-WebM, if the audio WebM file has one track encoded to track number 2, ne_read_block will fail and thus the audio will not play.

Current solution/workaround has been to encode the video and audio files with TrackNumber=1. This is fine, but it raises a possible incompatibility with DASH-WebM files encoded for Chrome/MediaSource. Instructions from http://wiki.webmproject.org/adaptive-streaming/instructions-to-playback-a-webm-dash-presentation suggest that audio and video should be encoded with different TrackNumbers, since MediaSource doesn't work if they are the same. While this may be a limitation of MediaSource and something that may need to be fixed there, I don't see a reason for the current restriction in nestegg that TrackNumbers be no greater than the total track count in a file.

Matroska Spec: http://matroska.org/technical/specs/index.html
WebM Spec: http://www.webmproject.org/docs/container/

I only see a restriction that TrackNumber can't be 0. If that's the case, then we can use any TrackNumber internally, and then get compatibility with DASH-WebM files encoded for Chrome/MediaSource.

This patch changes nestegg.c to do just that, and adds a function to get an array of TrackNumbers for external reference, i.e. WebMReader.

Comment 2

[Steve Workman [:sworkman] (INACTIVE)]

Attachment: v1.0 Use TrackNumbers in WebMReader to reference tracks

Uses nestegg_get_track_numbers to get reference values for audio/video tracks encoded in WebM media.

Comment 3

[Steve Workman [:sworkman] (INACTIVE)]

Try job, all mochitests:
https://tbpl.mozilla.org/?tree=Try&rev=3404e56bfa38

Comment 4

[Steve Workman [:sworkman] (INACTIVE)]

Matthew, there may be other reasons to maintain nestegg the way it is - if so, let me know and I can try to work out something less dangerous.

Comment 5

[Matthew Gregan [:kinetik]]

Comment on attachment 693223 [details] [diff] [review]
v1.0 Use TrackNumber to reference tracks instead of their index

The public API is was designed to expose normalized zero-based track numbers.  I'm not sure I want to change that right now.

The check in ne_read_block is certainly bogus because there's no requirement for the track numbering to start at 1 or even be contiguous.  The current normalization simply subtracts 1 from the container's track numbers, which is bogus for the same reason.

I think the right approach for now is to establish a track number mapping inside nestegg, so that the public API remains untouched.

Comment 6

[Steve Workman [:sworkman] (INACTIVE)]

(In reply to Matthew Gregan [:kinetik] from comment #5)
> Comment on attachment 693223 [details] [diff] [review]
> v1.0 Use TrackNumber to reference tracks instead of their index
> 
> The public API is was designed to expose normalized zero-based track
> numbers.  I'm not sure I want to change that right now.

Okie doke - makes sense.

> The check in ne_read_block is certainly bogus because there's no requirement
> for the track numbering to start at 1 or even be contiguous.  The current
> normalization simply subtracts 1 from the container's track numbers, which
> is bogus for the same reason.

Good - I read that right then.

> I think the right approach for now is to establish a track number mapping
> inside nestegg, so that the public API remains untouched.

Sounds like a plan.

Comment 7

[Steve Workman [:sworkman] (INACTIVE)]

Attachment: v2.0 Allow values for TrackNumber > 0; map TrackNumber to external index

-- Allows values for TrackNumber other than 0 in ne_read_block.

-- Provides a new internal function ne_map_track_number_to_index to map TrackNumber to the external index/reference value. This index is then stored in the nestegg packet, which is used externally. The index should have the same value externally, with or without the patch (discounting the error cases this patch aims to fix).

I have tested it with single stream audio files, multiplexed audio/video files, and DASH audio/video files - all with at least one TrackNumber > TrackCount. All pass.

Push to try for all mochitests:
https://tbpl.mozilla.org/?tree=Try&rev=5ae7183845c2

Comment 8

[Steve Workman [:sworkman] (INACTIVE)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/aeacf3c6231b

Comment 9

[Ed Morley [:emorley]]

https://hg.mozilla.org/mozilla-central/rev/aeacf3c6231b