Standalone Talos should output results as it runs

RESOLVED FIXED

Status

RESOLVED FIXED
11 years ago
5 years ago

People

(Reporter: Dolske, Assigned: johnath)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment, 1 obsolete attachment)

(Reporter)

Description

11 years ago
I've been using standalone talos for generating some perf data. It seems to only put results in the output/ directory after after *all* tests have completed successfully. This means you can't look at incremental results while it's running, and if one test fails/crashes you lose everything. [Eg, it seems Tgfx gets stuck on 1.8, and quitting the browser makes Talos quit without any results.]
Component: Testing → Release Engineering: Talos
Product: Core → mozilla.org
QA Contact: testing → release
Version: Trunk → other
That was stupid.
Component: Release Engineering: Talos → Talos
Product: mozilla.org → Testing
QA Contact: release → talos
Version: other → unspecified
(Assignee)

Comment 2

10 years ago
Created attachment 352728 [details] [diff] [review]
Dump results from each test after running (for CSV only)

This is the naive approach, I guess - I didn't try to re-write send_to_csv to take single tests or anything.  But I agree with dolske that this would be handy!
Attachment #352728 - Flags: review?(anodelman)
Comment on attachment 352728 [details] [diff] [review]
Dump results from each test after running (for CSV only)

This looks like an old version of the code - are you working from the copy in hg?  The copy in cvs is the one being maintained, the one in hg was created accidentally and is not in use.
Attachment #352728 - Flags: review?(anodelman) → review-
(Reporter)

Comment 4

10 years ago
Eep. Can you remove the Hg version, so that people don't stumble aross it accidentally?
Moving to correct talos category.
Component: Talos → Release Engineering: Talos
Product: Testing → mozilla.org
Version: unspecified → other
(Assignee)

Comment 6

10 years ago
(In reply to comment #3)
> (From update of attachment 352728 [details] [diff] [review])
> This looks like an old version of the code - are you working from the copy in
> hg?  The copy in cvs is the one being maintained, the one in hg was created
> accidentally and is not in use.

Oh, whoops - this was done against the standalone v1.3.1 zip on the wiki page: 

https://wiki.mozilla.org/StandaloneTalos#Prerequisites

I assume I should be using:

http://mxr.mozilla.org/mozilla/source/testing/performance/talos/run_tests.py
Looks like you're working on this Johnath, toss it back if this is incorrect.
Assignee: nobody → johnath
(Assignee)

Comment 8

10 years ago
Created attachment 357036 [details] [diff] [review]
Patch against cvs

Sorry, I've had this kicking around since the holidays and forgot to attach it.  This is against the current CVS source, but I haven't tested it, since I don't know if it's compatible with standalone configs.  Still, it's a pretty simple change.
Attachment #352728 - Attachment is obsolete: true
Attachment #357036 - Flags: review?(anodelman)
Attachment #357036 - Flags: review?(anodelman) → review+
(Assignee)

Comment 9

10 years ago
Checking in talos/run_tests.py;
/cvsroot/mozilla/testing/performance/talos/run_tests.py,v  <--  run_tests.py
new revision: 1.35; previous revision: 1.34
done

Do you want me to resolve this now, or wait until we have an updated zip for the standalone talos wiki page?
(Assignee)

Updated

10 years ago
Status: NEW → ASSIGNED
This can be resolved, I've added a dependent bug for releasing a new version of standalone talos.
Status: ASSIGNED → RESOLVED
Last Resolved: 10 years ago
Resolution: --- → FIXED

Updated

10 years ago
Component: Release Engineering: Talos → Release Engineering
QA Contact: talos → release
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.