804205 - Crash in LibTheora::huffdec.c (oc_huff_tree_collapse) with VS2012/PGO

Status: RESOLVED DUPLICATE of bug 703135

(Core :: Audio/Video, defect)

Description

[Mark Straver]

When building Firefox on VS2012 using PGO (x86 build), the resulting binary crashes in gkmedias.dll when trying to play certain videos. The VS debugger pointed at a crash point in huffdec.c in libtheora:

===
static size_t oc_huff_tree_collapse(ogg_int16_t *_tree,
 unsigned char _tokens[][2],int _ntokens){
  ogg_int16_t   node[34];
  unsigned char depth[34];
  unsigned char last[34];
  size_t        ntree;
  int           ti;
  int           l;
  depth[0]=0;
  last[0]=(unsigned char)(_ntokens-1);
  ntree=0;
  ti=0;
  l=0;
  do{
    int nbits;
    nbits=oc_huff_tree_collapse_depth(_tokens+ti,last[l]+1-ti,depth[l]);
    node[l]=(ogg_int16_t)ntree;
    ntree+=oc_huff_node_size(nbits);
>>  if(_tree!=NULL)_tree[node[l]++]=(ogg_int16_t)nbits; << *CRASH*
    do{
      while(ti<=last[l]&&_tokens[ti][1]<=depth[l]+nbits){
        if(_tree!=NULL){
          ogg_int16_t leaf;
          int         nentries;
===

Building from the same tree with the same parameters but without PGO does not cause this crash - Possibly a compiler bug? I've tried several builds with different optimizations flags and they consistently display this behavior in this lib.
Easy to reproduce with some of the W3schools pages, e.g. using the tryit editor from the <video> tag page. The crash doesn't seem to happen on all HTML5 videos through.

Comment 1

[Ioana (away)]

Does this issue reproduce when using a clean profile with your build?
http://support.mozilla.org/en-US/kb/profile-manager-create-and-remove-firefox-profiles

Comment 2

[Timothy B. Terriberry (:derf)]

This sounds an awful lot like bug 703135.

Comment 3

[Mark Straver]

Yes, it occurs on a fresh install with a clean profile.

I've tested a build side-by-side by now excluding huffdec.c from PGO with -GL- (from the makefile) on the same sites/videos and both a blank and populated profile, and the -GL- build has no issues, while the original build will crash with an access violation (c0000005 or whichever amount of 0's it is again)

Comment 4

[Mark Straver]

FYI, the makefile addition is (at the bottom of the file):

media/libtheora/lib/Makefile.in

ifdef _MSC_VER
ifeq ($(CPU_ARCH),x86)
# Workaround compiler bug on PGO (Bug 804205)
huffdec.$(OBJ_SUFFIX): CFLAGS += -GL-
endif
endif # _MSC_VER 

I'm not familiar with the method you prefer to submit patches to bugzilla, so if someone can do that properly, that would be nice ;)

Comment 5

[Carter Wang]

I also reproduced this even without PGO with VS2012 x86.

Comment 6

[Chris Pearce [:cpearce (Not reading bugmail)]]

I'm going to assume this is bug 703135. Please reopen if the patch in bug 703135 doesn't fix this for you.