1039626 - Add crash reporting to gaiatest

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: Gaia::UI Tests, defect)

Description

[Dave Hunt [:davehunt] [he/him] ⌚BST]

I've been looking into this today, and managed to get something working. It required me to use mozrunner's B2GDeviceRunner, which changes a few things in gaiatest (such as forcing a reboot, making the --restart command line option somewhat redundant for device testing).

I'll investigation continue over on bug 1038868.

Comment 1

[Paul Yang [:pyang] (away)]

Hi Dave, I just read wip patches in bug1038868 and am curious for scope of this bug. are we going to implement submitting crash report and other function, or only add check_for_crashes in gaiatest?

Comment 2

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Hey Paul, at the moment I'm just focusing on getting the crash information reported to the console using check_for_crashes. I suspect there will be more necessary to make this useful. For example, I think Stephen is hoping to have the crash details in the HTML report. I don't believe we'll be automatically submitting the crashes though.

Comment 3

[Paul Yang [:pyang] (away)]

Since mtbf uses only pvt build we'd like to leverage with https://crash-stats.mozilla.com/home/products/Firefox and need not to have a portal.
But check_for_crashes return boolean instead of crash number (whether crash report was submitted).  We do need this because it is parameter of formula but don't know if it is general case.

Comment 4

[Dave Hunt [:davehunt] [he/him] ⌚BST]

(In reply to Paul Yang [: pyang] from comment #3)
> Since mtbf uses only pvt build we'd like to leverage with
> https://crash-stats.mozilla.com/home/products/Firefox and need not to have a
> portal.
> But check_for_crashes return boolean instead of crash number (whether crash
> report was submitted).  We do need this because it is parameter of formula
> but don't know if it is general case.

I'm not sure I follow. The check_for_crashes returns true if a crash was detected, and as I understand it works by starting B2G on the device in a mode that will store crashes instead of submitting them. If they were submitted then it's unlikely that we will be able to detect if a crash has occurred.

I think it may be worth opening a new bug with your crash handling requirements for MTBF so that we can address those. I suspect this bug will be limited to detecting a crash has occurred and logging it to the console.

Comment 5

[Martijn Wargers (dead)]

(In reply to Dave Hunt (:davehunt) from comment #4)
> (In reply to Paul Yang [: pyang] from comment #3)
> I'm not sure I follow. The check_for_crashes returns true if a crash was
> detected, and as I understand it works by starting B2G on the device in a
> mode that will store crashes instead of submitting them. If they were
> submitted then it's unlikely that we will be able to detect if a crash has
> occurred.

But submitted crashes are stored here, afaik: /data/b2g/mozilla/Crash\ Reports/submitted/ (from: adb shell ls -l /data/b2g/mozilla/Crash\ Reports/submitted/  )
That would allow one to know if a crash has occured, no? 

I just saw an app (browser) crash occuring in a Jenkins report in test_browser_cell_data.py: http://jenkins1.qa.scl3.mozilla.com/job/flame-kk-319.b2g-inbound.tinderbox.ui.functional.smoke/4361/HTML_Report/

Comment 6

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Yes, this is effectively what B2GDeviceRunner already does. Unfortunately we're blocked because B2GDeviceRunner does not support remote device (bug 1143834). This is blocked by a mozdevice enhancement, which is blocked by an intermittent Gij issue (bug 1159200) that the mozdevice patch for some reason causes to become much more frequent. I'm still keen to finish up this work, but until the Gij issue is solved I'm not sure how much I can do.

Comment 7

[Martijn Wargers (dead)]

Attachment: all.diff

This would submit the crashes to crash-stats.mozilla.com, at least if there's wifi.

Comment 8

[Dave Hunt [:davehunt] [he/him] ⌚BST]

I haven't been working on this for some time now.

Comment 9

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Gaia is no longer an active project.