Closed Bug 1283984 Opened 8 years ago Closed 8 years ago

impossible to decode mediasource

Categories

(Core :: Audio/Video: Playback, defect, P1)

Product:
Core
Component:
Audio/Video: Playback
Platform:
x86_64
Linux
Type:
defect

Tracking

()

Status:
RESOLVED INVALID

People

(Reporter: cristian.lorenzetto, Unassigned)

Details

User Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36

Steps to reproduce:

i m adding a video using mediasource api. 
You can verify the same error also in a public url :
http://download.tsi.telecom-paristech.fr/gpac/mp4box.js/




Actual results:

no error as in chrome.


Expected results:

the log is :


          StreamPlayer:Source openedaa:1476:22
FileStream:ready to read file trailer.mp4 , size = 4372373aa:1476:22
FileStream:read offset [0,8191] , packet bps 65536000, bandwidth 65536000aa:1476:22
FileStream:set offset 4332917aa:1476:22
FileStream:read offset [4332917,4341108] , packet bps 7281777.777777778, bandwidth 36408888.88888889aa:1476:22
StreamPlayer:starting to parse movie informationaa:1476:22
FileStream:read offset [4341109,4349300] , packet bps 65536000, bandwidth 50972444.44444445aa:1476:22
FileStream:read offset [4349301,4357492] , packet bps 65536000, bandwidth 58254222.222222224aa:1476:22
FileStream:read offset [4357493,4365684] , packet bps 65536000, bandwidth 61895111.11111111aa:1476:22
FileStream:read offset [4365685,4367188] , packet bps 12032000, bandwidth 36963555.55555555aa:1476:22
FileStream:read offset [4367189,4370196] , packet bps 24064000, bandwidth 30513777.777777776aa:1476:22
FileStream:read offset [4370197,4372372] , packet bps 17408000, bandwidth 23960888.888888888aa:1476:22
FileStream:done reading fileaa:1476:22
StreamPlayer:adding sourceBuffer #1 for type 'video/mp4; codecs="avc1.64001e"'aa:1476:22
StreamPlayer:sourceBuffer #1 - appending initial bufferaa:1476:22
StreamPlayer:adding sourceBuffer #2 for type 'video/mp4; codecs="mp4a.40.2"'aa:1476:22
StreamPlayer:sourceBuffer #2 - appending initial bufferaa:1476:22
FileStream:set offset 8192aa:1476:22
StreamPlayer:ready to segmentation ...aa:1476:22
FileStream:read offset [8192,16383] , packet bps 16384000, bandwidth 20172444.444444444aa:1476:22
FileStream:read offset [16384,24575] , packet bps 65536000, bandwidth 42854222.222222224aa:1476:22
FileStream:read offset [24576,32767] , packet bps 65536000, bandwidth 54195111.11111111aa:1476:22
FileStream:read offset [32768,40959] , packet bps 65536000, bandwidth 59865555.55555555aa:1476:22
FileStream:read offset [40960,49151] , packet bps 65536000, bandwidth 62700777.777777776aa:1476:22
FileStream:read offset [49152,57343] , packet bps 65536000, bandwidth 64118388.88888889aa:1476:22
FileStream:read offset [57344,65535] , packet bps 65536000, bandwidth 64827194.44444445aa:1476:22
FileStream:read offset [65536,73727] , packet bps 1680410.2564102565, bandwidth 33253802.350427352aa:1476:22
FileStream:read offset [73728,81919] , packet bps 65536000, bandwidth 49394901.17521368aa:1476:22
FileStream:read offset [81920,90111] , packet bps 65536000, bandwidth 57465450.58760684aa:1476:22
FileStream:read offset [90112,98303] , packet bps 65536000, bandwidth 61500725.29380342aa:1476:22
FileStream:read offset [98304,106495] , packet bps 65536000, bandwidth 63518362.64690171aa:1476:22
MP4Parser:received new segment for track 2 up to sample #100aa:1476:22
StreamPlayer:sourcebuffer#2 - adding segment 1, pending count: 1aa:1476:22
StreamPlayer:sourceBuffer #2 - appending new buffer, pending: 0aa:1476:22
StreamPlayer:sourceBuffer #2 - error aa:1476:22
StreamPlayer:sourcebuffer #2 - update endedaa:1476:22
Impossibile decodificare la risorsa multimediale mediasource:http://127.0.0.1:8080/8de79b27-bba1-4bff-a88a-b67756a8325b.aa
StreamPlayer:source closed, error: "general"aa:1476:22
FileStream:read offset [106496,114687] , packet bps 885621.6216216217, bandwidth 32201992.134261668
Severity: normal → major
OS: Unspecified → Linux
Priority: -- → P1
Hardware: Unspecified → x86_64
sorry the log is the actual result. I expected no error "Impossibile decodificare la risorsa multimediale mediasource:http://127.0.0.1:8080/8de79b27-bba1-4bff-a88a-b67756a8325b.aa"
Testing this link (it is not my case but just for understand the general stability level ... https://html5-demos.appspot.com/static/media-source.html  has problems too)
(In reply to Cristian from comment #0)
> User Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like
> Gecko) Chrome/51.0.2704.106 Safari/537.36
The above user agent is a Chrome agent (most likely from where you logged the bug). Could you please reply with the firefox version you're having problems on? (type about:support in address bar and copy paste it from there)


(In reply to Cristian from comment #2)
> Testing this link (it is not my case but just for understand the general
> stability level ... https://html5-demos.appspot.com/static/media-source.html
> has problems too)

Also, related to the above link, I get no issue running it in Firefox47- 20160604131506 but it fails on Nightly50 -20160704030211 with "InvalidStateError: An attempt was made to use an object that is not, or is no longer, usablemedia-source.html:174readChunk_/reader.onload". But I do not think this problem is the same one you reported.

So, Christian, could you please attach a testcase for this bug as well?
Flags: needinfo?(cristian.lorenzetto)
(In reply to Cristian from comment #0)
> User Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like
> Gecko) Chrome/51.0.2704.106 Safari/537.36

>>The above user agent is a Chrome agent (most likely from where you logged the bug). Could you please >>reply with the firefox version you're having problems on? (type about:support in address bar and copy >>paste it from there)

Sorry you miserunderstood. The user agent is referred to the agent reading the bug tracker page(In that moment i used chrome). The log bottom is refering instead to the firefox browser where is the problem.
You can also try this link directly by you (http://download.tsi.telecom-paristech.fr/gpac/mp4box.js/) having a similar log error. 

(In reply to Cristian from comment #2)
> Testing this link (it is not my case but just for understand the general
> stability level ... https://html5-demos.appspot.com/static/media-source.html
> has problems too)


Also, related to the above link, I get no issue running it in Firefox47- 20160604131506 but it fails on Nightly50 -20160704030211 with "InvalidStateError: An attempt was made to use an object that is not, or is no longer, usablemedia-source.html:174readChunk_/reader.onload". But I do not think this problem is the same one you reported.
 
In thi case i used firefox 48.0a2 (2016-05-25) and the problem is 

   InvalidStateError: An attempt was made to use an object that is not, or is no longer, usable

So, Christian, could you please attach a testcase for this bug as well?
The firefox using is 48.0a2 (2016-05-25) for developers
>>Also, related to the above link, I get no issue running it in Firefox47- 20160604131506 but it fails on >>Nightly50 -20160704030211 with "InvalidStateError: An attempt was made to use an object that is not, or is >>no longer, usablemedia-source.html:174readChunk_/reader.onload". But I do not think this problem is the >>same one you reported.
 
In this case i used firefox 48.0a2 (2016-05-25) and the problem is 

   InvalidStateError: An attempt was made to use an object that is not, or is no longer, usable

>>So, Christian, could you please attach a testcase for this bug as well?

the log and the link above is referred to firefox
This bug is related to https://bugzilla.mozilla.org/show_bug.cgi?id=1138992. Note that as reported in that bug, the MP4Box.js demo was working (although not optimized) in Firefox. It is not anymore. I don't know yet if this is a regression in MP4Box.js (I'm investigating) or in Firefox.

I'm using:
Name: Firefox
Version: 50.0a1
Build ID: 20160703030210
Update Channel: nightly
User Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:50.0) Gecko/20100101 Firefox/50.0
OS: Windows_NT 10.0
Multiprocess Windows: 0/1 (Disabled by accessibility tools)
Safe Mode: false
Ah ok ... so probaly is a bug in mp4box not in firefox. Compliments anyway for your knowledges in mp4&video for all :) I m learning following your previous discussion.
A suggestion : I m  developping my player and i d have to calculate the required time for download the next segments (considering the cursor point at a precise temporal offset). 
The missing info for me is the size of segments before to download them. 
There is no way for having this info on the fly  except to saving them in a initial dash file?
i was thinking i could know the size of the segment i m going to download before to download it in the header part probbaly no (it m thinking in abstract way - not only mp4 but also webm parsing )?
Christian, your question is not related to Firefox. It's an MPEG DASH Question. I suggest you ask it on DASH related forums. This one is for Firefox-related bugs only.
Regarding Firefox, I've isolated the problem. 

An example of media segments generated by MP4Box.js can be found here:
http://download.tsi.telecom-paristech.fr/gpac/mp4box.js/segments/20160705/

You can play them with this simple player provided here:
http://download.tsi.telecom-paristech.fr/gpac/mp4box.js/segments/segment-player.html

Loading the init segment does not create a problem. Loading the first media segment gives the following error message in the page: "Video can't be played because the file is corrupt".

The console says: 
"Media resource mediasource:http://127.0.0.1:8080/bb81764f-db51-4729-9188-de4c6962fd1b could not be decoded."

FYI, this plays fine in Google Chrome and MS Edge.

Is there a way to get some information about why a media segment is not decoded in Firefox? Something similar to chrome://media-internals of Google Chrome ?
Small mistake in the previous message, the console says:
"Media resource mediasource:http://download.tsi.telecom-paristech.fr/60432742-02cf-42da-854a-62a0a7557234 could not be decoded"
Thanks Cyril for the help. The scenario you provided (comment12) is reproducible for me on both Release47 and Nightly 50.0a1. Therefore, based on the above comments and to advance this bug further setting component for this bug to A/V.
Status: UNCONFIRMED → NEW
Component: Untriaged → Audio/Video
Ever confirmed: true
Flags: needinfo?(cristian.lorenzetto)
Product: Firefox → Core
Version: 48 Branch → Trunk
Flags: needinfo?(jyavenard)
Component: Audio/Video → Audio/Video: Playback
init segment is missing ftyp atom:
per spec:
https://w3c.github.io/media-source/isobmff-byte-stream-format.html#iso-init-segments

"An ISO BMFF initialization segment is defined in this specification as a single File Type Box (ftyp) followed by a single Movie Box (moov)"

when adding the first media segment, as no init segment was found, per spec:
https://w3c.github.io/media-source/index.html#sourcebuffer-segment-parser-loop

"If the first initialization segment received flag is false, then run the append error algorithm with the decode error parameter set to true and abort this algorithm."
Status: NEW → RESOLVED
Closed: 8 years ago
Flags: needinfo?(jyavenard)
Resolution: --- → INVALID
Thanks for the feedback. Few questions and remarks:
- This was indeed a regression in MP4Box.js. Now fixed, sorry for the trouble.
- Is there a way to get more details when such error happens (setting a flag, passing arguments ...)? It's very hard to know what goes wrong.
- It is very confusing to have an error when appending a media segment, when it actually concerns the initialization segment. I think you could fire an error during the IS append. Since in a conformant IS the ftyp shall be followed by a moov, the absence of ftyp could be detected by the start of the moov, and the error triggered then (see section 3 of the ISOBMFF byte stream format).
- That said, given that neither Chrome nor Edge failed on those streams and that the ftyp is purely signaling, I think the spec should probably be revised to tolerate its absence, and Firefox should probably be more robust to its absence. What do you think?
reading the spec :
3. Initialization Segments

An ISO BMFF initialization segment is defined in this specification as a single File Type Box (ftyp) followed by a single Movie Header Box (moov).

The user agent must run the end of stream algorithm with the error parameter set to "decode" if any of the following conditions are met:

A File Type Box contains a major_brand or compatible_brand that the user agent does not support.
A box or field in the Movie Header Box is encountered that violates the requirements mandated by the major_brand or one of the compatible_brands in the File Type Box.
The tracks in the Movie Header Box contain samples (i.e. the entry_count in the stts, stsc or stco boxes are not set to zero).
A Movie Extends (mvex) box is not contained in the Movie (moov) box to indicate that Movie Fragments are to be expected.
The user agent must support setting the offset from media composition time to movie presentation time by handling an Edit Box (edts) containing a single Edit List Box (elst) that contains a single edit with media rate one. This edit may have a duration of 0 (indicating that it spans all subsequent media) or may have a non-zero duration (indicating the total duration of the movie including fragments).

The user agent must support parameter sets (e.g., PPS/SPS) stored in the sample entry (as defined for avc1/avc2), and should support parameter sets stored inband in the samples themselves (as defined for avc3/avc4).

does the bytearray ftyp start with a specific magic code(different from moov)?
aligned(8) class FileTypeBox
extends Box(‘ftyp’) {
unsigned int(32) major_brand;
unsigned int(32) minor_version;
unsigned int(32) compatible_brands[];
}
aligned(8) class Box (unsigned int(32) boxtype,
optional unsigned int(8)[16] extended_type) {
unsigned int(32) size;
unsigned int(32) type = boxtype;
if (size==1) {
unsigned int(64) largesize;
} else if (size==0) {
// box extends to end of file
}
if (boxtype==‘uuid’) {
unsigned int(8)[16] usertype = extended_type;
}
}
i answered on my own , i think. Yes int32 type tell the box type. 
So the the initializing segment has no ftyp box maybe it could be added. but why has no ftyp the original file?
ah sorry i answered on my own also it observing the fix in the library :)
We go by the spec:
https://w3c.github.io/media-source/index.html#sourcebuffer-segment-parser-loop

"If the beginning of the input buffer indicates the start of an initialization segment, set the append state to PARSING_INIT_SEGMENT."

We interpret this as the beginning of an init segment is the ftyp atom signature. 

The ftyp is never seen and as such we never enter appendState == PARSING_INIT_SEGMENT

adding error because a box is missing could have the side effect of false positives. You would no longer be able to append incomplete data because the buffer appended has been truncated. 

The w3c reference test has specific test that appends just a ftyp atom and later a moov. Distinguishing this case from an error adds, I believe, extra unnecessary complexity. 

You'll find the Firefox MSE implementation is almost word for words what is found in the specs. That Chrome or Edge incorrectly implement them is no reason to have Firefox do the same wrong thing.
You may use Firefox in your regression tests instead. I'd like to think that if it works with Firefox it will work with all other browsers. 

If you believe the specs are too narrow, feel free to open a bug at w3c. 
We do enter a case here not explicitly handled in the spec, but at the end of the day, the data was invalid m: a case of garbage in, garbage out.
I was rather interpreting the following rule, which comes before the text you mention:
"If the input buffer contains bytes that violate the SourceBuffer byte stream format specification, then run the append error algorithm with the decode error parameter set to true and abort this algorithm."
But I follow your interpretation. You don't know if it violates the byte stream format if you don't know it's an init segment ... and you don't know it's an init segment until you've find the beginning ... Anyway, it was a mistake not to put the ftyp box.

It is my intention to include Firefox in the regression tests, but a way to get meaningful error messages would help in using it.
i have another error in my player ... but i m investiganting .... the tracer log is identical in chrome and firefox , no error thrown but no video visible. I m investigating ....
FYI, I've raised an issue on the MSE spec:
https://github.com/w3c/media-source/issues/110
comparing player behaviour : i found a suspected side effect:

[0:03:27.395] [MSE - SourceBuffer #1] Creation with type 'video/mp4; codecs="avc1.42c01e"'mp4box.js:3:274
[0:03:31.268] [MSE - SourceBuffer #2] Creation with type 'video/mp4; codecs="mp4a.40.2"'mp4box.js:3:274
[0:03:31.276] [MSE - SourceBuffer #1] Appending initialization datamp4box.js:3:274
[0:03:31.276] [MSE - SourceBuffer #2] Appending initialization data

instead of 

[0:03:27.395] [MSE - SourceBuffer #1] Creation with type 'video/mp4; codecs="avc1.42c01e"'mp4box.js:3:274
[0:03:31.268] [MSE - SourceBuffer #2] Creation with type 'video/mp4; codecs="mp4a.40.2"'mp4box.js:3:274
[0:03:31.276] [MSE - SourceBuffer #1] Appending initialization datamp4box.js:3:274
[0:03:31.276] [MSE - SourceBuffer #2] Appending initialization data


I m modifying player for confirming the ipothesis (in that case firefox is correct ? )
instead of 

StreamPlayer:adding sourceBuffer #1 for type 'video/mp4; codecs="avc1.64001e"'aa:41:24
StreamPlayer:sourceBuffer #1 - appending initial bufferaa:41:24
StreamPlayer:adding sourceBuffer #2 for type 'video/mp4; codecs="mp4a.40.2"'aa:41:24
StreamPlayer:sourceBuffer #2 - appending initial buffer
it is not possible to add a additional sourcebuffer in a second time when video is buffering ?
the question is anyway interesting but it is not the cause of bug. 
i found the problem.
el.poster= '';  throws a error in video component. it can be usefull for normalizing the behaviour with chrome. 


Now video is working but i saw sometimes a error 

An attempt was made to use an object that is not, or is no longer, usable when i call mediasource.endOfStream
it is very strange because mediasource should be a usable object in the page
the problem seams to appear when i go back and forward with seeking cursor many times. Some times the video is crashing.

I think there is some problems with synchronization of objects inside browser.
firefox is crashed
(In reply to Cristian from comment #32)
> firefox is crashed

Please file a separate bug for the crash with a link to the crash from about:crashes
ok :) it is probably a concurrent synchronization problem or a memory leak. 
code is bp-bdda6a7a-c8a4-4521-96d9-aaf262160712
	12/07/2016	19:27 
i will submit it in a new bug. 

can i ask a fast question? 
in the case there are 2 track video or audio with different language or codec what is possible to do?
i saw that if i add 2 sourcebuffer with 2 video track it is throws a exception. So i image it is possible to 
add/remove sourcebuffer on the fly. For example if user change language i can remove sourcebuffer lang eng and add sourcebuffer lang ita also in the half of video visualization?
(In reply to Cristian from comment #34)

> can i ask a fast question? 
> in the case there are 2 track video or audio with different language or
> codec what is possible to do?
> i saw that if i add 2 sourcebuffer with 2 video track it is throws a
> exception. So i image it is possible to 
> add/remove sourcebuffer on the fly. For example if user change language i
> can remove sourcebuffer lang eng and add sourcebuffer lang ita also in the
> half of video visualization?

none of this is currently supported.
tracks change isn't supported. The first track in a mp4/webm of each type (either audio or video) is always selected, there is no ability to select a different track.

You also can only have a single sourcebuffer of a given type (either audio or video) active, and the first one to receive an init segment will be the one used. So you can't have a sourcebuffer with mp4 video and another with webm and have them overlap. It's one or the other.

if you remove a source buffer from the video element, you can't add it later. Well, you can but it will do nothing.

feel free to open a bug if you want those features supported. so far there's been no request for it.
Hi thanks for the comment. very usefull for me. I posted a issue/new feature to w3c for this.
https://github.com/w3c/media-source/issues/120
I think you misunderstood me: the limitations are firefox limitations ; not in the MSE spec. While the behaviour isn't fully defined on how it should work if you have overlapping sourcebuffer, there's nothing in the spec preventing it from working).

it's a firefox bug you should open, not a w3c one
ah ok anyway also chrome has same limitations. maybe w3c might describe better the behaviour.
I will add a bug in firefox too:)
You need to log in before you can comment on or make changes to this bug.