Closed Bug 496684 Opened 15 years ago Closed 14 years ago

Some valid Ogg files are rejected as invalid (they don't play)

Categories

(Core :: Audio/Video, defect)

defect
Not set
normal

Tracking

()

RESOLVED FIXED
mozilla1.9.2a1
Tracking Status
status1.9.1 --- .4-fixed

People

(Reporter: kinetik, Assigned: cajbir)

References

Details

Attachments

(1 file, 1 obsolete file)

I know this is a known problem, but I can't find any existing bug for it.  Feel free to close this as a dupe if I've missed an existing bug.

When we fixed a bunch of crashers recently (e.g. bug 487519 and others), the temporary workaround we used caused some valid files to be rejected.  It's not clear how many valid files we may be rejecting.  I think it's assumed that the set is fairly small, but there was some concern in #theora that the set may be more substantial.

Some files that are current rejected are:
http://people.redhat.com/ebachalo/video/eclipse-valgrind-demo-1-au.ogg
http://whatthebert.com/ihameed/boomtss/zurie-piratesxaimusremix.ogg
http://podcasts.linux-foundation.org/ogg/LinuxTools/eclipse-oprofile-demo.ogg
Gregory Maxwell offered to run analysis against the Ogg files hosted by Wikipedia to get a feel for how many valid files we may be rejecting.  It'd be extremely unfortunate to ship Firefox 3.5 if we're unable to play a large number of Wikipedia's valid Ogg files.
Viktor had a patch in bug 487519 comment 53 but it has issues as mentioned in comment 56. Do these files play with that patch applied though?
Applying the patch mentioned in comment 3 allows the first and third examples from comment 1 to play. The video is initially garbled due to the issues mentioned about the patch.

The vorbis only files don't play. Possibly due to a/v sync timing issues from missing the first frame.
Viktor, any progress on fixing the issues with that patch?
Summary: Some valid Ogg files are rejected as invalid → Some valid Ogg files are rejected as invalid (they don't play)
Do we know what the specific issues are with those files that are causing failures?
There is nothing wrong with the files.

My understanding of the behavior is this:
Libvorbis has an init function which gets passed header information from the vorbis stream. If the init function isn't called or if it fails then subsequent vorbis decode calls are unsafe and can cause random memory scribbling. 

Some patterns of headers (i.e. stream muxing orders, existence of comment packets, etc) trigger oggplay to signal to firefox that its ready before all headers from all streams have been processed. Firefox then commands oggplay to see to position zero, which is beyond some of the headers. Then it attempts to decode audio with the non-initialized decoder.  This was causing a crash and possible security problems.

Chris Double implemented a fix that causes it to reject files which result in the vorbis decoder being called before being initialized. This removes the crash and potential security problems, but we're still left with the same root cause of making unreasonable assumptions about the mutual ordering of data from multiple streams. 

Viking implemented a fix to wait until all headers are read, but this causes video playback to miss the all-important first keyframe on some streams.

Perhaps there is some structural challenge in oggplay which makes it difficult to handle the fact that streams do not always begin or end in with particular ordering.
I found the issue with Viktor's patch that was causing the first frame to be skipped. This patch is Viktor's patch plus that fix. This allows the two video files mentioned in the first comment to play but the Vorbis files still do not play. I'll investigate to see if the problem lies in liboggplay or our code.
Assignee: nobody → chris.double
Status: NEW → ASSIGNED
This fixes the remaining issue preventing the vorbis files from playing. That issue was caused by my patch in bug 487519 which treated all non-zero fishsound results as an error which was not correct.
Attachment #382665 - Attachment is obsolete: true
Attachment #382669 - Flags: review?(wiking)
Viktor, can you review please and let me know if ok.
Flags: wanted1.9.1?
It's been suggested by <video> users that it would be very bad for this not to go in 1.9.1. The number of files that don't play is fairly large and there is no reliable way to detect if <video> will play the file or not. This makes fallback pretty much impossibly to work in this situation so page authors will just use some other mechanism that can reliably play the video (eg. Cortado).

Some test pages (http://chaos.troll.no/~tavestbo/webkit/mediaelement/) have examples of videos that don't work in Firefox due to this bug but do work in Chrome for example.

This is why I've asked for wanted1.9.1.
Chris can you help us understand risk here?
As I understood it, the "risk" is about the fact that if a lot of videos won't play, it would discourage the usage of the <video> tag. Also, as Mozilla is really going in for this one and "rewriting the web", I suggest this should be included as soon as possible.
I meant risk in the patch, not risk in not understanding the impact.
This risk is that it could introduce a bug into the granule position handling code. This is fundamental to the reading of video and audio files and would result in them not playing. 

The good news is that if this were the case our tests should pick it up and the examples linked wouldn't play at all. Note the original version of the patch triggered test failures due to these issues.

Due to the the test coverage I think the risk is low.
We're at the end game.  I'm assuming that by putting the ? on it you want it for 3.5.  Beltzner?  Shaver?
Beltzner and Shaver aren't CCed. I'm not sure what to do here, I think we really really want this patch in 1.9.1 but Viktor can't be found to review it.
Whiteboard: [needs review]
Sorry i'm just now I'm just now a in a different time zone...

On comment 16: i don't exactly see what kind of bug could it introduce? i mean this patch basically assures that any of the streams present in an Ogg container will have to follow the specs of Ogg + that it reads first all the headers during the initialization before being able to do anything else.
Viktor: can we take that as a positive review of the patch?
Comment on attachment 382669 [details] [diff] [review]
Fix for remaining audio files

I think we should take comment #20 as a positive review and get this on trunk ASAP.
Attachment #382669 - Flags: review?(wiking) → review+
Whiteboard: [needs review] → [needs landing]
http://hg.mozilla.org/mozilla-central/rev/646ba2991c1c
Status: ASSIGNED → RESOLVED
Closed: 15 years ago
Resolution: --- → FIXED
Whiteboard: [needs landing]
Target Milestone: --- → mozilla1.9.2a1
Blocks: 498855
The first and third examples in comment 0 are still rejected:

Windows XP, Firefox 3.5 RC2:
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1) Gecko/20090616 Firefox/3.5

Linux x86/64, mozilla-1.9.1 nightly:
Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1pre) Gecko/20090622 Shiretoko/3.5pre

According to comment 4 and comment 24, these should be working now.
I concur(In reply to comment #25)
> The first and third examples in comment 0 are still rejected:
> 
> Windows XP, Firefox 3.5 RC2:
> Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1) Gecko/20090616
> Firefox/3.5
> 
> Linux x86/64, mozilla-1.9.1 nightly:
> Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1pre) Gecko/20090622
> Shiretoko/3.5pre
> 
> According to comment 4 and comment 24, these should be working now.

I confirm this too.  samples 1 and 3 do not play for me.  Reopening for investigation.

Tested on: 
Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1pre) Gecko/20090623 Shiretoko/3.5pre Ubiquity/0.1.5  
and Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.2a1pre) Gecko/20090623 Minefield/3.6a1pre
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Tested on latest 1.9.1 branch on Linux

Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1pre) Gecko/20090623 Shiretoko/3.5pre

All samples in comment #0 and the single sample in comment #2 do not play. The behaviors per sample are different.

In comment #0, sample #1, I get an X with no load. Right clicking and selecting "Play" demonstrates a 0:00 begin and end time.

In comment #0, sample #2, I get a load of 2:56 total play time. The play button is shown as pause and I can not get the audio to play through contextual or icon click. Off-topic, this tab when opened has a favicon of a Windows error dialog (what the heck?)

In comment #0, sample #3, I get an X with no load. Right clicking and selecting "Play" demonstrates a 0:00 begin and end time. 

In comment #2, sample #1, I get a load of 2:45 total play time - the behavior is the same as comemnt #0, sample #2.
(In reply to comment #27)
> Tested on latest 1.9.1 branch on Linux
> 
> Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1pre) Gecko/20090623
> Shiretoko/3.5pre
> 
> All samples in comment #0 and the single sample in comment #2 do not play. The
> behaviors per sample are different.
> 
> In comment #0, sample #1, I get an X with no load. Right clicking and selecting
> "Play" demonstrates a 0:00 begin and end time.
> 
> In comment #0, sample #2, I get a load of 2:56 total play time. The play button
> is shown as pause and I can not get the audio to play through contextual or
> icon click. Off-topic, this tab when opened has a favicon of a Windows error
> dialog (what the heck?)
> 
> In comment #0, sample #3, I get an X with no load. Right clicking and selecting
> "Play" demonstrates a 0:00 begin and end time. 
> 
> In comment #2, sample #1, I get a load of 2:45 total play time - the behavior
> is the same as comemnt #0, sample #2.

Suspicious of more VM issues, I justed tested natively on a 32-bit Ubuntu install with Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1pre) Gecko/20090623 Shiretoko/3.5pre and sure enough the following play correctly:

Comment #0, sample #2 and comment #2 sample #1

The following give me an X: 

Comment #0 - Sample #1
http://people.redhat.com/ebachalo/video/eclipse-valgrind-demo-1-au.ogg

Comment #0 - Sample #3
http://podcasts.linux-foundation.org/ogg/LinuxTools/eclipse-oprofile-demo.ogg
Keywords: fixed1.9.1
Removed "fixed1.9.1". Comment #0, samples 1 and 3 do not play on latest Shiretoko on Win, Mac, Linux.
Sorry for (possibly) hijacking this bug. Is this

http://proyectofedora.org/mexico/wp-content/uploads/2009/05/boot.ogg

another example of this bug? Even with Firefox 3.5 (Fedora package) it doesn't play. There might be some problems with the file itself (totem complains about problems with demuxing), just if somebody could take a look at this.

(originally filed as https://bugzilla.redhat.com/show_bug.cgi?id=501632)
(In reply to comment #30)
> http://proyectofedora.org/mexico/wp-content/uploads/2009/05/boot.ogg
> 
> another example of this bug? Even with Firefox 3.5 (Fedora package) it doesn't
> play. There might be some problems with the file itself (totem complains about
> problems with demuxing), just if somebody could take a look at this.

MPlayer complains about being unable to open the codec (generic error message) and audio does not play.

Totem gives some more unusual results:

osterj@cn212570:~/Desktop$ totem --version
totem 2.26.1
osterj@cn212570:~/Desktop$ totem boot.ogg 
/var/lib/python-support/python2.6/gdata/tlslite/utils/cryptomath.py:9: DeprecationWarning: the sha module is deprecated; use the hashlib module instead
  import sha
Segmentation fault


My guess is there is something wrong with the OGG file.
Regarding comment 30. This is a known bug. It's a corrupted Ogg file triggering a problem with the Ogg playback library liboggplay that we use. Bug 500311.
(In reply to comment #32)
> Regarding comment 30. This is a known bug. It's a corrupted Ogg file triggering
> a problem with the Ogg playback library liboggplay that we use. Bug 500311.

Unfortunately, I am apparently not allowed to see bug 500311 :(
Depends on: 516323
Hi, 

not sure if this is the same bug, but the following ogg file: http://ks36686.kimsufi.com/tmp/2_319948258ebfd3eb0c5b362df6f9c62f.ogg does not play with the current nightly build (and does not with FF 3.5). It loads the file, "knows" it length, but does not start playback. The behavior is exactly the same with FF 3.5.
Dirk, — whatever tool you used to write that 2_319948258ebfd3eb0c5b362df6f9c62f.ogg file, don't use that. Its muxing is screwy: It's putting one vorbis packet per page which is increasing the file size by about 7% and first page of audio data has a granpos of zero, which is a bit weird.   I don't see an obvious reason that firefox should refuse to play it, but the file is certainly a bit screwy.
Hi, (In reply to comment #35)
this file was written with ffmpeg (version 0.5 built on May, 14th 2009) using libvorbis which _should_ be fine - it did not come to my mind that this could be an encoder problem. I will try to sort this out - thank you anyways. I will try with a newer snapshot.
FFmpeg's ogg framer has been known to be broken in the past; though I thought it had been fixed, I suppose not.

In this case your file is odd and inefficient. It but not invalid as far as I or the people currently active on freenode IRC #theora can determine. It's a fine test case and firefox should probably be fixed.
Depends on: 531340
The new Ogg decoder backend plays all the files listed in this bug which aren't 404ing.
Status: REOPENED → RESOLVED
Closed: 15 years ago14 years ago
Resolution: --- → FIXED
(In reply to comment #38)
> The new Ogg decoder backend plays all the files listed in this bug which aren't
> 404ing.

is this also fixed on 1.9.1? We should set status accordingly, or mark this fixed and file a new bug to fix it on 1.9.1 (since sounds like this bug fixed "something" on 1.9.1)
(In reply to comment #39)
> is this also fixed on 1.9.1? We should set status accordingly, or mark this
> fixed and file a new bug to fix it on 1.9.1 (since sounds like this bug fixed
> "something" on 1.9.1)

All the videos in comment 0 and 2 work for me in 1.9.1 [Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.8pre) Gecko/20100110 Shiretoko/3.5.8pre] and 1.9.2 [Mozilla/5.0 (Windows; U; Windows NT 6.1; en-GB; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3].

The video in comment 30 doesn't play in my 1.9.1 build, but Chris Double pointed out that that's Bug 500311.

This was probably fixed on 1.9.1/1.9.2 by the liboggplay update back in September from bug 512328.
Based on comment 40 I'm setting status1.9.1 to .4-fixed (since it's the same as bug 512328)
This bug is bogus-marked ad not yet fixed on 1.9.1 while it has been pushed to that branch and sounds like it IS fixed instead.
You need to log in before you can comment on or make changes to this bug.