924039 - crash in arena_dalloc via mozilla::nsGonkCameraControl::SetParameter()

Status: RESOLVED DUPLICATE of bug 928856

(Firefox OS Graveyard :: Gaia::Camera, defect)

Description

[bhargavg1]

Attachment: 05a20d0f-13ae-5df0-24a3d2fa-6457eb28.txt

When the following tests are run continuously overnight

Call -> Camera -> Camcorder -> Video -> Music -> Airplance

seeing a crash in mozilla::DOMCameraPreview::Start

BuildID: 20130930193354 
Version: 26.0a2

Comment 1

[Mike Habicher [:mikeh] (high bugzilla latency)]

Looks like we're failing this assertion:
http://mxr.mozilla.org/mozilla-central/source/memory/mozjemalloc/jemalloc.c#3314

Comment 2

[Mike Habicher [:mikeh] (high bugzilla latency)]

jemalloc has lots of inlines; it looks like the callstack is:
- free
- arena_dalloc
- arena_dalloc_small
- arena_run_reg_dalloc
- RELEASE_ASSERT
- jemalloc_crash
- MOZ_CRASH

Comment 3

[Mike Habicher [:mikeh] (high bugzilla latency)]

I'm pretty sure this isn't a Gaia bug. My first-pass guess is memory corruption, or a double-free.

Comment 4

[Gregor Wagner [:gwagner]]

(In reply to Mike Habicher [:mikeh] from comment #3)
> I'm pretty sure this isn't a Gaia bug. My first-pass guess is memory
> corruption, or a double-free.

So move it to DOM:DeviceInterfaces?

Comment 5

[Mike Habicher [:mikeh] (high bugzilla latency)]

What's wrong with B2G:General?

Comment 6

[Gregor Wagner [:gwagner]]

(In reply to Mike Habicher [:mikeh] from comment #5)
> What's wrong with B2G:General?

It doesn't show up in the queries for functional teams and no functional team is triaging this component.
So nobody really tracks this bug.

Comment 7

[Mike Habicher [:mikeh] (high bugzilla latency)]

(In reply to Gregor Wagner [:gwagner] from comment #6)
> 
> It doesn't show up in the queries for functional teams and no functional
> team is triaging this component.
> So nobody really tracks this bug.

I didn't realize that. :( Okay, so it's Camera-related, but not Gaia. But it's also B2G-specific, which doesn't unlock DOM:Interfaces.

Comment 8

[Jason Smith [:jsmith]]

(In reply to Mike Habicher [:mikeh] from comment #7)
> (In reply to Gregor Wagner [:gwagner] from comment #6)
> > 
> > It doesn't show up in the queries for functional teams and no functional
> > team is triaging this component.
> > So nobody really tracks this bug.
> 
> I didn't realize that. :( Okay, so it's Camera-related, but not Gaia. But
> it's also B2G-specific, which doesn't unlock DOM:Interfaces.

If we really want to get this into a b2g query, we could create a camera component in b2g and have the media team triage it.

Comment 9

[Jason Smith [:jsmith]]

i.e. Product = BootToGecko, Component = Camera

Comment 10

[Preeti Raghunath(:Preeti)]

Hema

Please review as this seems to be a camera issue.

Comment 11

[Mike Habicher [:mikeh] (high bugzilla latency)]

Until comment 9 becomes real, go with comment 6.

Comment 12

[Preeti Raghunath(:Preeti)]

Mike,

Assigning this bug to you since you were actively working on this bug.

Comment 13

[Mike Habicher [:mikeh] (high bugzilla latency)]

I wonder if this could be an allocator mismatch. mwu, do we (still) have both jemalloc and dlmalloc in our builds?

Comment 14

[Michael Wu [:mwu]]

Yup. However, jemalloc overrides dlmalloc in 99.9% of cases. There remain a few odd cases where you might accidentally use dlmalloc, especially if bionic is calling the allocator.

Comment 15

[Mike Habicher [:mikeh] (high bugzilla latency)]

I don't suppose we have any easy, non-crashy way to detect a mismatch without Valgrind? :)

Comment 16

[Mike Habicher [:mikeh] (high bugzilla latency)]

Bhargav, just to confirm: your DUT actually has a flash?

Comment 17

[bhargavg1]

(In reply to Mike Habicher [:mikeh] from comment #16)
> Bhargav, just to confirm: your DUT actually has a flash?

Yes it has

Comment 19

[Hema Koka [:hema]]

Mikeh,  Were you able to reproduce this. Please update with the latest info on this bug.

Comment 20

[bhargavg1]

Didnt see this crash again, closing for now. Shall re-open again if seen again

Comment 21

[Michael Wu [:mwu]]

FWIW, I don't know of any easy way of easily detecting an allocator mismatch, though the crash usually makes it quite obvious.

Comment 22

[bhargavg1]

We are seeing this crash again in the latest builds, could reproduce 4 times in the recent build when run with the similar steps

Gecko: 0770a5a5f305b23896d7805f6b2637e3a2919cee
Gaia: cb981e2f47bc644b4d178d54378c3676c946facf

Comment 23

[Mike Habicher [:mikeh] (high bugzilla latency)]

More detailed STR:
- Make a call & end call
- Take 4/5 snapshots through camera
- Record a video through camcorder
- Play a video in video app
- Play music in music app

Comment 24

[Mike Habicher [:mikeh] (high bugzilla latency)]

I have not been able to reproduce this problem locally with any of the devices I have.

Bhargav, can you rerun your test with a DEBUG build and enable camera logging on the build?
1. B2G_DEBUG=1 ./build.sh gecko
   --- you'll need to |rm -rf| your objdir-gecko before you build
2. add |export NSPR_LOG_MODULES=Camera:4| to /system/bin/b2g.sh on the device, just before the 'exec' line

If you can reproduce with this, it will at least tell us what state the camera is in when the crash occurs.

Comment 25

[Marcia Knous [:marcia]]

Testing with Buri device, using:

Gaia   2140c987fdde1c99097018f7e93b0bbd43d2125d
SourceStamp 6a831fcb96f4
BuildID 20131106004004
Version 26.0

I can get the camera in state where it doesn't work, but I don't get a crash. There is a comment about the DUT having a flash in this bug, but of course Buri and the ZTE device which is showing up in crash stats [@ mozilla::DOMCameraPreview::Start ] don't have a flash.

Will keep trying to repro and report back again - - I did try the steps in Comment 23 several times with no luck yet.

Comment 26

[Mike Habicher [:mikeh] (high bugzilla latency)]

(In reply to Marcia Knous [:marcia - use needinfo] from comment #25)
> 
> I can get the camera in state where it doesn't work, but I don't get a
> crash.

Marcia, please file a bug for the new "doesn't work" state. Thanks!

Comment 27

[Mike Habicher [:mikeh] (high bugzilla latency)]

Attachment: Dump gaia stacktrace before setting .flashMode

Bhargav, as an alternative to comment 24, can you reproduce this bug with this patch applied to gaia? This will dump the JS stack leading up to setting the flashMode and help us narrow the conditions of the crash.

Comment 28

[Mike Habicher [:mikeh] (high bugzilla latency)]

Bhargav, please take a look at bug 928856. We think these bugs have the same root cause, and would like to close this one if so.

Comment 29

[bhargavg1]

(In reply to Mike Habicher [:mikeh] from comment #28)
> Bhargav, please take a look at bug 928856. We think these bugs have the same
> root cause, and would like to close this one if so.


Just to be clear will make run with the following patches and then see if we can get still

patch from comment 27 
patch from Bug 928856 comment 38

Comment 30

[Mike Habicher [:mikeh] (high bugzilla latency)]

Yes, you can run with both patches. Thank you!

Comment 31

[bhargavg1]

(In reply to Mike Habicher [:mikeh] from comment #26)
> (In reply to Marcia Knous [:marcia - use needinfo] from comment #25)
> > 
> > I can get the camera in state where it doesn't work, but I don't get a
> > crash.
> 
> Marcia, please file a bug for the new "doesn't work" state. Thanks!

Do we have a bug for this, we are seeing a similar issue

Comment 32

[bhargavg1]

(In reply to bhargavg1 from comment #31)
> (In reply to Mike Habicher [:mikeh] from comment #26)
> > (In reply to Marcia Knous [:marcia - use needinfo] from comment #25)
> > > 
> > > I can get the camera in state where it doesn't work, but I don't get a
> > > crash.
> > 
> > Marcia, please file a bug for the new "doesn't work" state. Thanks!
> 
> Do we have a bug for this, we are seeing a similar issue

Raised bug 937758

Comment 33

[Marcia Knous [:marcia]]

(In reply to Mike Habicher [:mikeh] from comment #26)
> (In reply to Marcia Knous [:marcia - use needinfo] from comment #25)
> > 
> > I can get the camera in state where it doesn't work, but I don't get a
> > crash.
> 
> Marcia, please file a bug for the new "doesn't work" state. Thanks!

Bug 937928

Comment 34

[Preeti Raghunath(:Preeti)]

Hema

Please check with Mike H if this is reproducible.

Comment 35

[Hema Koka [:hema]]

The root cause for this is same as bug 928856. Mike's patch on that bug will fix some of the known rare crashes on camera

Comment 36

[Mike Habicher [:mikeh] (high bugzilla latency)]

If this issue reappears, please open a new bug.