1299892 - talos could handle crashes better

Status: RESOLVED INACTIVE

(Testing :: Talos, defect)

Description

[Ben Kelly [:bkelly, not reviewing]]

While working on a new talos test in bug 1299271 I encountered a crash (bug 1299887).  I spent about half a day debugging talos, though, because it simply exhibited one of two behaviors:

1) If the crash occurred while there were more entries in manifest it simply hung waiting for the next test to start.

2) If it was the last entry in the manifest it would complain with this error:

>    07:51:08     INFO -  PROCESS | 37503 | Cycle 1(1): loaded http://localhost:59410/tests/service_worker/images.html?images=25 (next: http://localhost:59410/tests/service_worker/images.html?images=25)
>    07:51:08     INFO -  Terminating psutil.Process(pid=37503, name='firefox')
>    07:51:08     INFO -  TEST-INFO | 37503: exit 11
>    07:51:08     INFO -  TEST-UNEXPECTED-ERROR | tservice_worker | Could not find report in browser output: [('tsformat', ('__start_report', '__end_report')), ('tpformat', ('__start_tp_report', '__end_tp_report'))]
>    07:51:08    ERROR -  Traceback (most recent call last):
>    07:51:08     INFO -    File "/srv/mozilla-central/testing/talos/talos/run_tests.py", line 202, in run_tests
>    07:51:08     INFO -      talos_results.add(mytest.runTest(browser_config, test))
>    07:51:08     INFO -    File "/srv/mozilla-central/testing/talos/talos/ttest.py", line 70, in runTest
>    07:51:08     INFO -      return self._runTest(browser_config, test_config, setup)
>    07:51:08     INFO -    File "/srv/mozilla-central/testing/talos/talos/ttest.py", line 232, in _runTest
>    07:51:08     INFO -      else None)
>    07:51:08     INFO -    File "/srv/mozilla-central/testing/talos/talos/results.py", line 94, in add
>    07:51:08     INFO -      global_counters=self.global_counters
>    07:51:08     INFO -    File "/srv/mozilla-central/testing/talos/talos/results.py", line 311, in __init__
>    07:51:08     INFO -      self.parse()
>    07:51:08     INFO -    File "/srv/mozilla-central/testing/talos/talos/results.py", line 338, in parse
>    07:51:08     INFO -      % self.report_tokens)
>    07:51:08     INFO -    File "/srv/mozilla-central/testing/talos/talos/results.py", line 322, in error
>    07:51:08     INFO -      raise utils.TalosError(message)
>    07:51:08     INFO -  TalosError: Could not find report in browser output: [('tsformat', ('__start_report', '__end_report')), ('tpformat', ('__start_tp_report', '__end_tp_report'))]
>    07:51:08     INFO -  TEST-INFO took 18174ms 

It would be nice if talos could detect the crash and ideally show the breakpad stack trace when this happens.