978458 - gaia-ui tests crashing in automation

Status: RESOLVED DUPLICATE of bug 978450

(Firefox OS Graveyard :: Gaia::UI Tests, defect)

Description

[Ben Kelly [:bkelly, not reviewing]]

It appears gaia-ui tests are crashing in automation.  We're not quite sure what's happening yet.

Logcat is available here:

  https://app.box.com/s/uz12m88njyljvupk4fsi

Dave Huseby noted in #fxos-perf last night that he was seeing crashes with Marionette enabled, but things seemed ok with it disabled.  Not sure if that is the same problem, but just wanted to mention it.

It would probably be helpful if we could get one of these crash reports off the device:

I/GeckoDump(  136): Crash reporter : Deleting pending crash: 25bd5c26-e178-6f67-111eaeee-103aa4b8

Comment 1

[Zac C (:zac)]

Ben is this on the Jenkins (device) CI? or some other?

A console trace is very useful because it will tell you the test in which it failed and then you can search for that test start name in the logcat which helps to narrow down the huge logcat.

To get the crash report we have to get the device before the next test (or job) starts using the device because when it does it will wipe the crash report.

I will follow up on this tomorrow when I'm in the office.

Comment 2

[Dave Huseby [:huseby]]

So I looked at this more over the weekend.  It looks like the problem only happens when you build with MOZ_PROFILING=1 and ENABLE_MARIONETTE=1 at the same time.  The logcat I'm seeing from the crashes had these lines in them:

E/Profiler(  140): BPUnw: [8 total] thread_register_for_profiling(me=0x4beec0, stacktop=0x49b75dd7)
I/Gecko   (  140): [Parent 140] WARNING: waitpid failed pid:829 errno:10: file /opt/B2G-Hamachi/gecko/ipc/chromium/src/base/process_util_posix.cc, line 254
I/Gecko   (  140): [Parent 140] WARNING: waitpid failed pid:829 errno:10: file /opt/B2G-Hamachi/gecko/ipc/chromium/src/base/process_util_posix.cc, line 254
I/Gecko   (  140): [Parent 140] WARNING: Failed to deliver SIGKILL to 829!(3).: file /opt/B2G-Hamachi/gecko/ipc/chromium/src/chrome/common/process_watcher_posix_sigchld.cc, line 118


Something about the thread register in the profiler doesn't mix well with marionette.  To confirm this, I rebuilt with just ENABLE_MARIONETTE=1 and the crashes no longer happen.

Comment 3

[Dave Huseby [:huseby]]

This is on a Hamachi/Buri building master tip.(In reply to Zac C (:zac) from comment #1)
> Ben is this on the Jenkins (device) CI? or some other?

This is on a Hamachi/Buri building master tip.

Comment 4

[Zac C (:zac)]

(In reply to Dave Huseby [:huseby] from comment #3)
> This is on a Hamachi/Buri building master tip.(In reply to Zac C (:zac) from
> comment #1)
> > Ben is this on the Jenkins (device) CI? or some other?
> 
> This is on a Hamachi/Buri building master tip.

All our on-device CI/Automation has been affected over the weekend too :(

Comment 5

[Ben Kelly [:bkelly, not reviewing]]

Just to clarify, I have personally not seen this.  I was mainly filing the bug for Stephen since he had to run to catch a plane.

Comment 6

[Jason Smith [:jsmith]]

Might be related to the problem seen in bug 978450. We need a crash stack to confirm this.

Comment 7

[Zac C (:zac)]

(In reply to Jason Smith [:jsmith] from comment #6)
> Might be related to the problem seen in bug 978450. We need a crash stack to
> confirm this.

Yes, agree, the timing of failures match up.

Comment 8

[Jason Smith [:jsmith]]

Okay - I'm going to dupe this to bug 978450 for now then. If this ends up reproducing on tomorrow's build again, then reopen this.