Closed Bug 790602 Opened 8 years ago Closed 8 years ago

Make TBPL recognise talos failures prefixed with "talosError:"

Categories

(Tree Management Graveyard :: TBPL, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: emorley, Assigned: emorley)

References

(Blocks 1 open bug)

Details

(Whiteboard: [sheriff-want])

Attachments

(1 file)

eg:
{
Failed tdhtml: 
		Stopped Tue, 11 Sep 2012 23:13:57
Traceback (most recent call last):
  File "run_tests.py", line 250, in run_tests
    talos_results.add(mytest.runTest(browser_config, test))
  File "/builds/tegra-231/talos-data/talos/ttest.py", line 378, in runTest
    test_results.add(browser_log_filename, counter_results=counter_results)
  File "/builds/tegra-231/talos-data/talos/results.py", line 119, in add
    results = BrowserLogResults(filename=results, counter_results=counter_results, global_counters=self.global_counters).results()
  File "/builds/tegra-231/talos-data/talos/results.py", line 312, in __init__
    self.parse()
  File "/builds/tegra-231/talos-data/talos/results.py", line 349, in parse
    self.error("Could not find %s in browser output: (tokens: %s)" % (attr, tokens))
  File "/builds/tegra-231/talos-data/talos/results.py", line 324, in error
    raise utils.talosError(message)
talosError: "Could not find beforeLaunchTime in browser output: (tokens: ('__startBeforeLaunchTimestamp', '__endBeforeLaunchTimestamp')) [browser_output.txt]"
FAIL: Busted: tdhtml
FAIL: Could not find beforeLaunchTime in browser output: (tokens: ('__startBeforeLaunchTimestamp', '__endBeforeLaunchTimestamp')) [browser_output.txt]
Traceback (most recent call last):
  File "run_tests.py", line 298, in <module>
    main()
  File "run_tests.py", line 295, in main
    run_tests(parser)
  File "run_tests.py", line 259, in run_tests
    raise e
utils.talosError: "Could not find beforeLaunchTime in browser output: (tokens: ('__startBeforeLaunchTimestamp', '__endBeforeLaunchTimestamp')) [browser_output.txt]"
}
(bug 781106)

{
Failed tcheck2: 
		Stopped Tue, 11 Sep 2012 23:09:45
Traceback (most recent call last):
  File "run_tests.py", line 250, in run_tests
    talos_results.add(mytest.runTest(browser_config, test))
  File "/builds/tegra-099/talos-data/talos/ttest.py", line 378, in runTest
    test_results.add(browser_log_filename, counter_results=counter_results)
  File "/builds/tegra-099/talos-data/talos/results.py", line 119, in add
    results = BrowserLogResults(filename=results, counter_results=counter_results, global_counters=self.global_counters).results()
  File "/builds/tegra-099/talos-data/talos/results.py", line 309, in __init__
    self.error(match.group(1))
  File "/builds/tegra-099/talos-data/talos/results.py", line 324, in error
FAIL: Busted: tcheck2
FAIL: browser non-zero return code (1) [browser_output.txt]
    raise utils.talosError(message)
talosError: 'browser non-zero return code (1) [browser_output.txt]'
Traceback (most recent call last):
  File "run_tests.py", line 298, in <module>
    main()
  File "run_tests.py", line 295, in main
    run_tests(parser)
  File "run_tests.py", line 259, in run_tests
    raise e
utils.talosError: 'browser non-zero return code (1) [browser_output.txt]'
program finished with exit code 1
}
(bug 686245)

{
Traceback (most recent call last):
  File "C:\talos-slave\talos-data\talos\bcontroller.py", line 221, in ?
    sys.exit(main())
  File "C:\talos-slave\talos-data\talos\bcontroller.py", line 218, in main
    bcontroller.run()
  File "C:\talos-slave\talos-data\talos\bcontroller.py", line 161, in run
    results_file = open(self.browser_log, "a")
IOError: [Errno 13] Permission denied: 'browser_output.txt'
Failed tdhtmlr: 
		Stopped Fri, 07 Sep 2012 03:45:22
FAIL: Busted: tdhtmlr
FAIL: timeout exceeded
Traceback (most recent call last):
  File "run_tests.py", line 250, in run_tests
    talos_results.add(mytest.runTest(browser_config, test))
  File "C:\talos-slave\talos-data\talos\ttest.py", line 366, in runTest
    raise talosError("timeout exceeded")
talosError: 'timeout exceeded'
Traceback (most recent call last):
  File "run_tests.py", line 298, in ?
    main()
  File "run_tests.py", line 295, in main
    run_tests(parser)
  File "run_tests.py", line 259, in run_tests
    raise e
utils.talosError: 'timeout exceeded'
program finished with exit code 1
}
(bug 572127)

{
Failed tcheckerboard: 
		Stopped Fri, 07 Sep 2012 20:31:56
Traceback (most recent call last):
  File "run_tests.py", line 250, in run_tests
    talos_results.add(mytest.runTest(browser_config, test))
  File "/builds/tegra-164/talos-data/talos/ttest.py", line 263, in runTest
    self.initializeProfile(profile_dir, browser_config)
  File "/builds/tegra-164/talos-data/talos/ttest.py", line 137, in initializeProfile
    if not self._ffsetup.InitializeNewProfile(profile_dir, browser_config):
  File "/builds/tegra-164/talos-data/talos/ffsetup.py", line 292, in InitializeNewProfile
    raise talosError("initialization timed out")
talosError: 'initialization timed out'
Traceback (most recent call last):
  File "run_tests.py", line 298, in <module>
    main()
  File "run_tests.py", line 295, in main
    run_tests(parser)
  File "run_tests.py", line 259, in run_tests
    raise e
utils.talosError: 'initialization timed out'
FAIL: Busted: tcheckerboard
FAIL: initialization timed out
program finished with exit code 1
}
(bug 690176)

{
Failed tdhtml: 
		Stopped Sun, 09 Sep 2012 17:11:36
Traceback (most recent call last):
  File "run_tests.py", line 250, in run_tests
    talos_results.add(mytest.runTest(browser_config, test))
  File "/builds/tegra-056/talos-data/talos/ttest.py", line 263, in runTest
    self.initializeProfile(profile_dir, browser_config)
  File "/builds/tegra-056/talos-data/talos/ttest.py", line 141, in initializeProfile
    raise talosError("browser failed to close after being initialized")
talosError: 'browser failed to close after being initialized'
Traceback (most recent call last):
  File "run_tests.py", line 298, in <module>
FAIL: Busted: tdhtml
FAIL: browser failed to close after being initialized
    main()
  File "run_tests.py", line 295, in main
    run_tests(parser)
  File "run_tests.py", line 259, in run_tests
    raise e
utils.talosError: 'browser failed to close after being initialized'
program finished with exit code 1
}
(bug 782189)

{
Failed ts: 
		Stopped Tue, 11 Sep 2012 20:03:37
Traceback (most recent call last):
  File "run_tests.py", line 250, in run_tests
    talos_results.add(mytest.runTest(browser_config, test))
  File "/builds/tegra-192/talos-data/talos/ttest.py", line 263, in runTest
    self.initializeProfile(profile_dir, browser_config)
  File "/builds/tegra-192/talos-data/talos/ttest.py", line 138, in initializeProfile
    raise talosError("failed to initialize browser")
talosError: 'failed to initialize browser'
Traceback (most recent call last):
  File "run_tests.py", line 298, in <module>
FAIL: Busted: ts
FAIL: failed to initialize browser
    main()
  File "run_tests.py", line 295, in main
    run_tests(parser)
  File "run_tests.py", line 259, in run_tests
    raise e
utils.talosError: 'failed to initialize browser'
program finished with exit code 1
}
(bug 686085)

{
Failed tspaint_places_generated_max: 
		Stopped Wed, 12 Sep 2012 02:49:24
FAIL: Busted: tspaint_places_generated_max
FAIL: initialization timed out
Traceback (most recent call last):
  File "run_tests.py", line 250, in run_tests
    talos_results.add(mytest.runTest(browser_config, test))
  File "c:\talos-slave\talos-data\talos\ttest.py", line 263, in runTest
    self.initializeProfile(profile_dir, browser_config)
  File "c:\talos-slave\talos-data\talos\ttest.py", line 137, in initializeProfile
    if not self._ffsetup.InitializeNewProfile(profile_dir, browser_config):
  File "c:\talos-slave\talos-data\talos\ffsetup.py", line 292, in InitializeNewProfile
    raise talosError("initialization timed out")
talosError: 'initialization timed out'
Traceback (most recent call last):
  File "run_tests.py", line 298, in ?
    main()
  File "run_tests.py", line 295, in main
    run_tests(parser)
  File "run_tests.py", line 259, in run_tests
    raise e
utils.talosError: 'initialization timed out'
program finished with exit code 1
}
(bug 739089)

{
Failed ts: 
		Stopped Tue, 11 Sep 2012 11:40:43
Traceback (most recent call last):
  File "run_tests.py", line 250, in run_tests
    talos_results.add(mytest.runTest(browser_config, test))
  File "/builds/tegra-216/talos-data/talos/ttest.py", line 366, in runTest
    raise talosError("timeout exceeded")
talosError: 'timeout exceeded'
FAIL: Busted: ts
FAIL: timeout exceeded
Traceback (most recent call last):
  File "run_tests.py", line 298, in <module>
    main()
  File "run_tests.py", line 295, in main
    run_tests(parser)
  File "run_tests.py", line 259, in run_tests
    raise e
utils.talosError: 'timeout exceeded'
program finished with exit code 1
}
(bug 675297)

{
Failed ts: 
		Stopped Wed, 12 Sep 2012 02:03:22
Traceback (most recent call last):
  File "run_tests.py", line 250, in run_tests
    talos_results.add(mytest.runTest(browser_config, test))
  File "/builds/tegra-227/talos-data/talos/ttest.py", line 374, in runTest
    test_results.add(browser_log_filename, counter_results=counter_results)
  File "/builds/tegra-227/talos-data/talos/results.py", line 118, in add
    results = BrowserLogResults(filename=results, counter_results=counter_results, global_counters=self.global_counters).results()
  File "/builds/tegra-227/talos-data/talos/results.py", line 311, in __init__
    self.parse()
  File "/builds/tegra-227/talos-data/talos/results.py", line 338, in parse
    self.error("Could not find report in browser output: %s" % self.report_tokens)
  File "/builds/tegra-227/talos-data/talos/results.py", line 323, in error
    raise utils.talosError(message)
talosError: "Could not find report in browser output: [('tsformat', ('__start_report', '__end_report')), ('tpformat', ('__start_tp_report', '__end_tp_report'))] [browser_output.txt]"
FAIL: Busted: ts
FAIL: Could not find report in browser output: [('tsformat', ('__start_report', '__end_report')), ('tpformat', ('__start_tp_report', '__end_tp_report'))] [browser_output.txt]
Traceback (most recent call last):
  File "run_tests.py", line 298, in <module>
    main()
  File "run_tests.py", line 295, in main
    run_tests(parser)
  File "run_tests.py", line 259, in run_tests
    raise e
utils.talosError: "Could not find report in browser output: [('tsformat', ('__start_report', '__end_report')), ('tpformat', ('__start_tp_report', '__end_tp_report'))] [browser_output.txt]"
program finished with exit code 1
}
(bug 781107)
Blocks: 781107
Assignee: nobody → bmo
Status: NEW → ASSIGNED
Actually, they all consistently prefix "talosError: " already (only obvious once I pasted them all in one comment), so we should just add support for this to TBPL.
Component: Talos → Tinderboxpushlog
Product: Testing → Webtools
Summary: Talos should output failures in format 'TEST-UNEXPECTED-FAIL' or 'Automation Error:' so the TBPL parser can recognise them → Make TBPL recognise talos failures prefixed with "talosError:"
Hmmm... or maybe we should s/FAIL:/TALOS-FAIL:/ in talos itself, that way we get the "FAIL: Busted: tdhtml" line (that displays in the lower panel summary) as well?

That would then make this redundant (in MachineResult.js):
>    var failLines = $(scrape).map(function parseFailedTalosRunScrapeLine() {
>      return this.match(/FAIL\:(.*)/) && { name: this };
>    }).filter(function filterNull() { return this; }).get();
Will go with this for now.

As a followup we can decide if the "FAIL: Busted: tdhtml" is useful, if so:
* s/FAIL:/<something_useful>/ in talos
* update the line I'm adding here to the new string

and then either way remove the now redundant |var failLines = ...| in MachineResult.js
Attachment #660445 - Flags: review?(philringnalda)
Attachment #660445 - Flags: review?(philringnalda) → review+
Ta :-)

http://hg.mozilla.org/users/mstange_themasta.com/tinderboxpushlog/rev/f9c6bbe2efcc

(Are you fussed about the "FAIL: Busted: tdhtml" part, or shall we just drop that?)
Depends on: 790874
In production :-)
Status: ASSIGNED → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Blocks: 790963
Blocks: 794895
Product: Webtools → Tree Management
Product: Tree Management → Tree Management Graveyard
You need to log in before you can comment on or make changes to this bug.