419367 - Standalone Talos should output results as it runs

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Justin Dolske [:Dolske]]

I've been using standalone talos for generating some perf data. It seems to only put results in the output/ directory after after *all* tests have completed successfully. This means you can't look at incremental results while it's running, and if one test fails/crashes you lose everything. [Eg, it seems Tgfx gets stuck on 1.8, and quitting the browser makes Talos quit without any results.]

Comment 1

[(not currently active) Ted Mielczarek]

That was stupid.

Comment 2

[Johnathan Nightingale [:johnath]]

Attachment: Dump results from each test after running (for CSV only)

This is the naive approach, I guess - I didn't try to re-write send_to_csv to take single tests or anything.  But I agree with dolske that this would be handy!

Comment 3

[alice nodelman [:alice] [:anode]]

Comment on attachment 352728 [details] [diff] [review]
Dump results from each test after running (for CSV only)

This looks like an old version of the code - are you working from the copy in hg?  The copy in cvs is the one being maintained, the one in hg was created accidentally and is not in use.

Comment 4

[Justin Dolske [:Dolske]]

Eep. Can you remove the Hg version, so that people don't stumble aross it accidentally?

Comment 5

[alice nodelman [:alice] [:anode]]

Moving to correct talos category.

Comment 6

[Johnathan Nightingale [:johnath]]

(In reply to comment #3)
> (From update of attachment 352728 [details] [diff] [review])
> This looks like an old version of the code - are you working from the copy in
> hg?  The copy in cvs is the one being maintained, the one in hg was created
> accidentally and is not in use.

Oh, whoops - this was done against the standalone v1.3.1 zip on the wiki page: 

https://wiki.mozilla.org/StandaloneTalos#Prerequisites

I assume I should be using:

http://mxr.mozilla.org/mozilla/source/testing/performance/talos/run_tests.py

Comment 7

[[e-mail] (:bhearsum)]

Looks like you're working on this Johnath, toss it back if this is incorrect.

Comment 8

[Johnathan Nightingale [:johnath]]

Attachment: Patch against cvs

Sorry, I've had this kicking around since the holidays and forgot to attach it.  This is against the current CVS source, but I haven't tested it, since I don't know if it's compatible with standalone configs.  Still, it's a pretty simple change.

Comment 9

[Johnathan Nightingale [:johnath]]

Checking in talos/run_tests.py;
/cvsroot/mozilla/testing/performance/talos/run_tests.py,v  <--  run_tests.py
new revision: 1.35; previous revision: 1.34
done

Do you want me to resolve this now, or wait until we have an updated zip for the standalone talos wiki page?

Comment 10

[alice nodelman [:alice] [:anode]]

This can be resolved, I've added a dependent bug for releasing a new version of standalone talos.