Integrate Tremor for Vorbis decoding for non/slow-FPU platforms

RESOLVED FIXED

Status

()

defect
RESOLVED FIXED
10 years ago
8 years ago

People

(Reporter: vlad, Assigned: dougt)

Tracking

unspecified
ARM
All
Points:
---
Dependency tree / graph
Bug Flags:
wanted1.9.2 -

Firefox Tracking Flags

(fennec2.0+)

Details

Attachments

(3 attachments, 7 obsolete attachments)

We really should be using Tremor (the fixed point decoder) and not stock Vorbis (floating point) on ARM and any other platform that has a slow or nonexistent FPU.
Flags: wanted1.9.2?
We are currently using liboggplay which uses libfishsound for audio decoding. If it supports tremor then it would be fairly easy to use otherwise it'd require a fair amount of effort. Might be worth contacting the libfishsound developer to get their thoughts. I've cc'd them to this bug.
Tremor is afaik largely API-compatible with vorbis -- see ivorbiscodec.h, ivorbisfile.h in the tremor source repo, which correspond to codec.h and vorbisfile.h for the standard vorbis decoder.  So getting fishsound to use it seems like it would be straightforward.
I added Tremor support to a version of libfishsound that we used on Symbian devices a few years ago. The code should still be in svn.annodex.net and straightforward to merge. It added functions to decode as integer; to use these we should also modify liboggplay to call them etc.
I don't think we want to do this for 1.9.2 at this point
Flags: wanted1.9.2? → wanted1.9.2-
We probably want this for Fennec soonish, as Vorbis is probably fairly slow on ARM.  Unfortunately it's not quite API-compatible as Tremor uses libogg2, so it'd need a bit of hacking to integrate a Tremor decoder.
I don't think (Tremor) has ever been fuzzed or otherwise security checked so it might be worth checking that too.
Robin Watts also has ARM optimizations for the tremor low-mem branch, which he calls Tremolo: http://svn.xiph.org/branches/lowmem-branch/Tremolo/
tracking-fennec: --- → 2.0+
Posted patch Tremor Integration to Webm (obsolete) — Splinter Review
Comment on attachment 469415 [details] [diff] [review]
Tremor Integration to Webm

Do we really need to allocate and free the packet/reference/buffer data structure for every packet? These could be members of nsWebMReader, and, since they are all of known, fixed size, don't need separate heap allocations at all. That removes the need for UnInitOggPacket, as well, and helps minimize the code divergence.

I'd also highly recommend making ClipTo15 static inline and moving it to a header, as it gets called once per sample, which is rather a lot.

Finally, where did you get your version of Tremor from? http://svn.xiph.org/trunk/Tremor/ does not have the source code in a lib subdir, nor does it have an AUTHORS or todo.txt (and, although your update.sh copies over a todo.txt, there does not appear to be one added with the diff?), and the README and COPYING look like they come from libvorbis, not Tremor.

This looks like enough to start getting performance numbers with Tremor, even without Robin Watt's ARM optimizations. That should be your next step. At some point we will need to re-work the configure logic so that a) you can't have both Tremor and Vorbis at the same time and b) Tremor is enabled by default for just the platforms that will benefit from it.
Posted patch review comments incorporated (obsolete) — Splinter Review
Posted patch Tremor Integration to Webm (obsolete) — Splinter Review
Attachment #469415 - Attachment is obsolete: true
Attachment #469801 - Attachment is obsolete: true
Attachment #469803 - Flags: review?(tterribe)
I did a diff on http://svn.xiph.org/trunk/Tremor/ & http://svn.xiph.org/branches/lowmem-branch/Tremolo/ for arm optimizations.

But there are many changes between these two trunks.

If I get Robin Watt's ARM optimizations as a seperate patch, then that would be easy to integrate.
Comment on attachment 469803 [details] [diff] [review]
Tremor Integration to Webm

I'm r-'ing this patch because it completely breaks the Ogg backend, which is not to say you aren't making good progress, only that it can't be committed in its current form yet.


> PRBool MsToSamples(PRInt64 aMs, PRUint32 aRate, PRInt64& aOutSamples);
>-
> #endif

Whitespace only change.

>     while ((samples = vorbis_synthesis_pcmout(&mVorbisDsp, &pcm)) > 0) {
>+    	short* buffer = new short[samples * mChannels];
>+      	  for(PRUint32 i=0;i<mChannels;i++) { /* It's faster in this order */

Tabs.

>+#else
>+     float** pcm = 0;

Wrong indentation.

>+    while ((samples = vorbis_synthesis_pcmout(&mVorbisDsp, (ogg_int32_t***)&pcm)) > 0) {

This cast shouldn't be here.

>                            PRInt64 aGranulepos);
>-                     
>+
>   // Decode a nestegg packet of audio data. Push the audio data on the

Whitespace change (though this one at least is a definite improvement, so it can stay as far as I'm concerned).

>+The source from this directory was copied from the libtremor
>+subversion repository using the update.sh script. The only changes
>+made were those applied by update.sh and the addition/upate of
>+Makefile.in files for the Mozilla build system.
>+
>+The upstream release used was http://svn.xiph.org/trunk/Tremor/

This does not appear to be true. The README and COPYRIGHT contents are from libvorbis, not Tremor, as I noted before, and there are a number of include file changes for which there is no corresponding patch applied by update.sh.

I'm including the following diff against the code in lib for reference:
Index: misc.h
===================================================================
--- misc.h	(revision 17372)
+++ misc.h	(working copy)
@@ -17,8 +17,8 @@
 
 #ifndef _V_RANDOM_H_
 #define _V_RANDOM_H_
-#include "ivorbiscodec.h"
-#include "os.h"
+#include "tremor/ivorbiscodec.h"
+#include "tremor/os.h"

 #include "asm_arm.h"
 #include <stdlib.h> /* for abs() */
Index: floor0.c
===================================================================
--- floor0.c	(revision 17372)
+++ floor0.c	(working copy)
@@ -18,8 +18,8 @@
 #include <stdlib.h>
 #include <string.h>
 #include <math.h>
-#include "ogg.h"
-#include "ivorbiscodec.h"
+#include "tremor/ogg.h"
+#include "tremor/ivorbiscodec.h"
 #include "codec_internal.h"
 #include "registry.h"
 #include "codebook.h"
Index: floor1.c
===================================================================
--- floor1.c	(revision 17372)
+++ floor1.c	(working copy)
@@ -18,8 +18,8 @@
 #include <stdlib.h>
 #include <string.h>
 #include <math.h>
-#include "ogg.h"
-#include "ivorbiscodec.h"
+#include "tremor/ogg.h"
+#include "tremor/ivorbiscodec.h"
 #include "codec_internal.h"
 #include "registry.h"
 #include "codebook.h"
Index: registry.c
===================================================================
--- registry.c	(revision 17372)
+++ registry.c	(working copy)
@@ -15,7 +15,7 @@
 
  ********************************************************************/
 
-#include "ivorbiscodec.h"
+#include "tremor/ivorbiscodec.h"
 #include "codec_internal.h"
 #include "registry.h"
 #include "misc.h"
Index: mapping0.c
===================================================================
--- mapping0.c	(revision 17372)
+++ mapping0.c	(working copy)
@@ -19,8 +19,8 @@
 #include <stdio.h>
 #include <string.h>
 #include <math.h>
-#include "ogg.h"
-#include "ivorbiscodec.h"
+#include "tremor/ogg.h"
+#include "tremor/ivorbiscodec.h"
 #include "mdct.h"
 #include "codec_internal.h"
 #include "codebook.h"
Index: framing.c
===================================================================
--- framing.c	(revision 17372)
+++ framing.c	(working copy)
@@ -21,7 +21,7 @@
 
 #include <stdlib.h>
 #include <string.h>
-#include "ogg.h"
+#include "tremor/ogg.h"
 #include "misc.h"
 
 
Index: codebook.c
===================================================================
--- codebook.c	(revision 17372)
+++ codebook.c	(working copy)
@@ -18,8 +18,8 @@
 #include <stdlib.h>
 #include <string.h>
 #include <math.h>
-#include "ogg.h"
-#include "ivorbiscodec.h"
+#include "tremor/ogg.h"
+#include "tremor/ivorbiscodec.h"
 #include "codebook.h"
 #include "misc.h"
 
Index: synthesis.c
===================================================================
--- synthesis.c	(revision 17372)
+++ synthesis.c	(working copy)
@@ -17,8 +17,8 @@
  ********************************************************************/
 
 #include <stdio.h>
-#include "ogg.h"
-#include "ivorbiscodec.h"
+#include "tremor/ogg.h"
+#include "tremor/ivorbiscodec.h"
 #include "codec_internal.h"
 #include "registry.h"
 #include "misc.h"
Index: codebook.h
===================================================================
--- codebook.h	(revision 17372)
+++ codebook.h	(working copy)
@@ -18,7 +18,7 @@
 #ifndef _V_CODEBOOK_H_
 #define _V_CODEBOOK_H_
 
-#include "ogg.h"
+#include "tremor/ogg.h"
 
 /* This structure encapsulates huffman and VQ style encoding books; it
    doesn't do anything specific to either.
Index: info.c
===================================================================
--- info.c	(revision 17372)
+++ info.c	(working copy)
@@ -21,8 +21,8 @@
 #include <stdlib.h>
 #include <string.h>
 #include <ctype.h>
-#include "ogg.h"
-#include "ivorbiscodec.h"
+#include "tremor/ogg.h"
+#include "tremor/ivorbiscodec.h"
 #include "codec_internal.h"
 #include "codebook.h"
 #include "registry.h"
Index: lsp_lookup.h
===================================================================
--- lsp_lookup.h	(revision 17372)
+++ lsp_lookup.h	(working copy)
@@ -18,7 +18,7 @@
 #ifndef _V_LOOKUP_DATA_H_
 #define _V_LOOKUP_DATA_H_
 
-#include "os_types.h"
+#include "tremor/os_types.h"
 
 #define FROMdB_LOOKUP_SZ 35
 #define FROMdB2_LOOKUP_SZ 32
Index: window_lookup.h
===================================================================
--- window_lookup.h	(revision 17372)
+++ window_lookup.h	(working copy)
@@ -16,7 +16,7 @@
  ********************************************************************/
 
 
-#include "os_types.h"
+#include "tremor/os_types.h"
 
 static LOOKUP_T vwin64[32] = {
   X(0x001f0003), X(0x01168c98), X(0x030333c8), X(0x05dfe3a4),
Index: bitwise.c
===================================================================
--- bitwise.c	(revision 17372)
+++ bitwise.c	(working copy)
@@ -20,7 +20,7 @@
 
 #include <string.h>
 #include <stdlib.h>
-#include "ogg.h"
+#include "tremor/ogg.h"
 
 static unsigned long mask[]=
 {0x00000000,0x00000001,0x00000003,0x00000007,0x0000000f,
Index: res012.c
===================================================================
--- res012.c	(revision 17372)
+++ res012.c	(working copy)
@@ -18,13 +18,13 @@
 #include <stdlib.h>
 #include <string.h>
 #include <math.h>
-#include "ogg.h"
-#include "ivorbiscodec.h"
+#include "tremor/ogg.h"
+#include "tremor/ivorbiscodec.h"
 #include "codec_internal.h"
 #include "registry.h"
 #include "codebook.h"
 #include "misc.h"
-#include "os.h"
+#include "tremor/os.h"
 #include "block.h"
 
 typedef struct {
Index: sharedbook.c
===================================================================
--- sharedbook.c	(revision 17372)
+++ sharedbook.c	(working copy)
@@ -18,9 +18,9 @@
 #include <stdlib.h>
 #include <math.h>
 #include <string.h>
-#include "ogg.h"
+#include "tremor/ogg.h"
 #include "misc.h"
-#include "ivorbiscodec.h"
+#include "tremor/ivorbiscodec.h"
 #include "codebook.h"
 
 /**** pack/unpack helpers ******************************************/
Index: mdct_lookup.h
===================================================================
--- mdct_lookup.h	(revision 17372)
+++ mdct_lookup.h	(working copy)
@@ -15,7 +15,7 @@
 
  ********************************************************************/
   
-#include "os_types.h"
+#include "tremor/os_types.h"
 
 /* {sin(2*i*PI/4096), cos(2*i*PI/4096)}, with i = 0 to 512 */
 static LOOKUP_T sincos_lookup0[1026] = {
Index: mdct.c
===================================================================
--- mdct.c	(revision 17372)
+++ mdct.c	(working copy)
@@ -32,7 +32,7 @@
 
  ********************************************************************/
 
-#include "ivorbiscodec.h"
+#include "tremor/ivorbiscodec.h"
 #include "codebook.h"
 #include "misc.h"
 #include "mdct.h"
Index: block.c
===================================================================
--- block.c	(revision 17372)
+++ block.c	(working copy)
@@ -18,8 +18,8 @@
 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
-#include "ogg.h"
-#include "ivorbiscodec.h"
+#include "tremor/ogg.h"
+#include "tremor/ivorbiscodec.h"
 #include "codec_internal.h"
 
 #include "window.h"
Index: mdct.h
===================================================================
--- mdct.h	(revision 17372)
+++ mdct.h	(working copy)
@@ -18,7 +18,7 @@
 #ifndef _OGG_mdct_H_
 #define _OGG_mdct_H_
 
-#include "ivorbiscodec.h"
+#include "tremor/ivorbiscodec.h"
 #include "misc.h"
 
 #define DATA_TYPE ogg_int32_t
Attachment #469803 - Flags: review?(tterribe) → review-
It's probably best to diff Tremolo against its branch point, not trunk, which is r17159 of /branches/lowmem-branch/Tremor. You can get this kind of information from svn log, though I highly recommend using the source browser on https://trac.xiph.org/ instead.
Posted patch Tremor Integration to Webm (obsolete) — Splinter Review
Review changes incorporated and updated below files.
README, COPYING, tremor_integration.diff & update.sh.
Attachment #469803 - Attachment is obsolete: true
Attachment #470393 - Flags: review?(tterribe)
Attachment #470393 - Attachment is obsolete: true
Attachment #470393 - Flags: review?(tterribe)
I have some drive by (mostly nitty) comments.

I'm not a fan of the large #if MOZ_TREMOR blocks in nsBuiltinDecoderReader.h around SoundData just for the change of the audio data type (float vs short). This seems like it could become a maintenance problem if changes are made in one block and not the other and not tested across both vorbis and tremor.

Can you instead have a typedef for short or float depending on MOZ_TREMOR being defined and use that typedef in SoundData?

This will at least remove the duplicate in SoundData and the definition of mAudioData.

You could also use a #define or constant for the use of nsAudioStream::FORMAT_S16_LE or nsAudioStream::FORMAT_FLOAT32 and use that constant ns nsBuiltinDecoderStateMachine.cpp. 

Maybe a macro moz_vorbis_synthesis that is defined to have the extra argument added that exists in the tremor version. If all the above are defined in the header file in once place this puts most of the MOZ_TREMOR changes in one place instead of having #ifdef's spread out through multiple files.

In some places you have C style casts. eg:

+        short *dest=((short *)buffer)+i;

These should be C++ style casts (static_cast, etc) to match usage in the rest of the file. Is the short* cast even needed in the above? 'buffer' is already a short*.

Some of the code needs spaces added between operators. For example:

+        for (PRUint32 j=0;j<samples;j++) {

Notice that the standard in the existing code in the file is:

       for (PRUint32 i = 0; i < samples; ++i) {

Please change to match this.

In nsWebMReader.h you have:

   // Booleans to indicate if we have audio and/or video data
   PRPackedBool mHasVideo;
   PRPackedBool mHasAudio;
+#ifdef MOZ_TREMOR
+  // Members of ogg_packet structure
+  ogg_reference mOggRef;
+  ogg_buffer mOggBuf;
+#endif

Please change so the PRPackedBool's are last in the definition (so the mOggRef and mOggBuf should be declared before mHasVideo).
I agree with most of Chris's comments, but would suggest that it would be better to change the upstream vorbis_synthesis call to be API-compatible with libvorbis, and add a separate function to handle the extra parameter. I've started a thread on the Tremor list to discuss this and other API changes: http://lists.xiph.org/pipermail/tremor/2010-August/001650.html
I've gone ahead and made the vorbis_synthesis change as well as ripped out the libogg2 implementation from the trunk version of Tremor and made it use libogg (r17374 and r17375 in upstream svn). It should now be possible to do a version of this patch which does not completely break the Ogg backend.
Blocks: 570335
Since Kishore never did any performance tests of his patch, I did at least do a simple test comparing libvorbis's vobisfile_example vs. Tremor's ivorbisfile_example (latest svn trunk) on a 3:08.7 minute, 128 kbps test file on an N900, and even without any of Robin Watts's optimizations, it showed a 70.5% speed-up (26.8s vs. 7.91s to decode).
Assignee: nobody → tterribe
Posted patch Tremor integration take 5 (obsolete) — Splinter Review
I reviewed every commit from the last 7 years of libvorbis development, and ported all of the decoder bug fixes to Tremor which had not already been ported. This can be found in upstream commits r17509, r17510, r17511, r17512, r17513, r17514, r17517, r17518, r17519, r17520, r17521, r17522, r17523, r17524, r17525, r17526, r17527, r17528, r17529, r17530, and r17531. Monty also ported over a fix in r17534. Two additional bugs were discovered and fixed in r17536 and r17538. The latter also affected libvorbis, but was not exploitable. In Tremor it might have been exploitable, since Tremor was using uninitialized values in a lookup table (libvorbis was not).

I also un-bitrotted Kishore's patch and applied all of Chris Double's review comments. The un-bitrotting unfortunately has a good deal of interaction with the AudioAvailable events from the new Audio API, which returns audio data as floats. Right now, this patch converts the integer data output by Tremor to floats when the data is copied to be dispatched. Unfortunately, this happens before the check to see if there are any listeners for these events, so it is always done. I spoke with David Humphrey, and he says he has a plan to get rid of both the unnecessary copy, and with it the unnecessary conversion, but that will be part of a separate bug.
Attachment #483381 - Flags: review?(chris.double)
By Chris Double's request, I've split the main code changes from the build system changes to make it easier to review.

Greg Maxwell and I also fuzz tested the library over the weekend, and identified 6 issues which Monty fixed upstream in commits r17541, r17542, r17543, r17544, r17545, r17546. I identified one additional issue the fuzzer didn't catch, which Monty fixed in r17547. This patch has been updated to include those fixes. I believe all of these issues also affect libvorbis, which I'll be filing under a separate bug, but is at least mildly disturbing, considering how much fuzzing it has undergone already.

Greg Maxwell also verified that these changes did not change the decoded md5sum of his test set, derived from the unique header configurations found in around 900,000 valid files from Wikipedia. The previous patch had been verified against the full 900,000-some files.
Attachment #483381 - Attachment is obsolete: true
Attachment #483932 - Flags: review?(chris.double)
Attachment #483381 - Flags: review?(chris.double)
Split-out build system changes. Requires the previous patch.
Comment on attachment 483932 [details] [diff] [review]
Tremor integration: library and backend changes

Very nice.
Attachment #483932 - Flags: review?(chris.double) → review+
Attachment #483933 - Attachment is patch: true
Attachment #483933 - Attachment mime type: application/octet-stream → text/plain
Comment on attachment 483933 [details] [diff] [review]
Tremor integration: build system changes

>@@ -6042,16 +6058,21 @@ if test -n "$MOZ_SYDNEYAUDIO"; then
> if test -n "$MOZ_VORBIS"; then
>     AC_DEFINE(MOZ_VORBIS)
>+    MOZ_TREMOR=
>+fi
>+
>+if test -n "$MOZ_TREMOR"; then
>+    AC_DEFINE(MOZ_TREMOR)
> fi

Instead of just silently suppressing MOZ_TREMOR if MOZ_VORBIS is defined, can we replace it with
> if test -n "$MOZ_VORBIS" -a -n "$MOZ_TREMOR"; then
>     AC_MSG_ERROR([MOZ_VORBIS and MOZ_TREMOR are mutually exclusive!  <insert message about how this shouldn't happen and they should file a bug>])
> fi

>diff --git a/media/libtremor/include/Makefile.in b/media/libtremor/include/Makefile.in
>new file mode 100644
>--- /dev/null
>+++ b/media/libtremor/include/Makefile.in
>@@ -0,0 +1,47 @@
>+# ***** BEGIN LICENSE BLOCK *****
>+# Version: MPL 1.1/GPL 2.0/LGPL 2.1
>+#
>+# The contents of this file are subject to the Mozilla Public License Version
>+# 1.1 (the "License"); you may not use this file except in compliance with
>+# the License. You may obtain a copy of the License at
>+# http://www.mozilla.org/MPL/
>+#
>+# Software distributed under the License is distributed on an "AS IS" basis,
>+# WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License
>+# for the specific language governing rights and limitations under the
>+# License.
>+#
>+# The Original Code is Mozilla code.
>+#
>+# The Initial Developer of the Original Code is the Mozilla Corporation.
>+# Portions created by the Initial Developer are Copyright (C) 2007
>+# the Initial Developer. All Rights Reserved.
>+#
>+# Contributor(s):
>+#  Chris Double <chris.double@double.co.nz>
>+#
>+# Alternatively, the contents of this file may be used under the terms of
>+# either the GNU General Public License Version 2 or later (the "GPL"), or
>+# the GNU Lesser General Public License Version 2.1 or later (the "LGPL"),
>+# in which case the provisions of the GPL or the LGPL are applicable instead
>+# of those above. If you wish to allow use of your version of this file only
>+# under the terms of either the GPL or the LGPL, and not to allow others to
>+# use your version of this file under the terms of the MPL, indicate your
>+# decision by deleting the provisions above and replace them with the notice
>+# and other provisions required by the GPL or the LGPL. If you do not delete
>+# the provisions above, a recipient may use your version of this file under
>+# the terms of any one of the MPL, the GPL or the LGPL.
>+#
>+# ***** END LICENSE BLOCK *****
>+
>+DEPTH  = ../../..
>+topsrcdir = @top_srcdir@
>+srcdir  = @srcdir@
>+VPATH  = @srcdir@
>+
>+include $(DEPTH)/config/autoconf.mk
>+
>+MODULE  = tremor
>+DIRS  = tremor
>+
>+include $(topsrcdir)/config/rules.mk

Is there anything in this directory?  If not, can we move ./tremor into this directory or set media/libtremor/Makefile.in's DIRS to use include/tremor?

Also, all of these new makefiles should have the initial developer listed as the Mozilla Foundation, not the Corporation.

Other than that, r=me.
Attachment #483933 - Flags: review+
Addressed comments from review.

Carrying forward r=khuey.
Attachment #483933 - Attachment is obsolete: true
Attachment #484047 - Flags: review+
pushing to try.  if green, i will push to mc
Assignee: tterribe → doug.turner
Build bustage on Android:

http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTry/1287435339.1287437815.19155.gz&fulltext=1

The problem appears to be that the inline ARM asm is incompatible with Thumb mode. This updated patch should disable it when MOZ_THUMB2 is set.
Attachment #484047 - Attachment is obsolete: true
http://hg.mozilla.org/mozilla-central/rev/d3fac77dac26
http://hg.mozilla.org/mozilla-central/rev/3e10efdc460f
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
just to be clear - it busted thumb only.  non-thumb did compile.
Attachment #484346 - Flags: review?(tterribe)
Comment on attachment 484346 [details] [diff] [review]
fixes build bustage on android

The last hunk is not a good permanent fix, since it makes the default endianess little-endian, and hides the bug if things are not set, but this is an issue which needs to be resolved upstream. For now this should be fine.
Attachment #484346 - Flags: review?(tterribe) → review+
FWIW, the correct solution appears to be to manually #include <endian.h> in os.h. This gets pulled in by <sys/types.h> on most platforms, but apparently not on Android. I'll take this upstream, and we can pull it in the next time we update.
Status: RESOLVED → UNCONFIRMED
Ever confirmed: false
Resolution: FIXED → ---
Status: UNCONFIRMED → RESOLVED
Closed: 9 years ago9 years ago
Resolution: --- → FIXED
Comment on attachment 484148 [details] [diff] [review]
Tremor integration: build system changes (Android bustage fix)

>diff --git a/toolkit/toolkit-makefiles.sh b/toolkit/toolkit-makefiles.sh
>+MAKEFILES_libtremor="
>+  media/libtremor/Makefile
>+  media/libtremor/lib/Makefile
>+  media/libtremor/include/Makefile
>+  media/libtremor/include/tremor/Makefile


Looks like that second-to-last line was a typo -- no file "media/libtremor/include/Makefile.in" was added in this bug, so we hit this build warning:
> can't read /scratch/work/builds/fennec/mozilla/media/libtremor/include/Makefile.in: No such file or directory

Pushed a fix to delete that line, with the standing rs=ted on fixing toolkit-makefiles.sh issues:
 http://hg.mozilla.org/integration/mozilla-inbound/rev/c83fb1dd0e78
You need to log in before you can comment on or make changes to this bug.