default format for tp data in talos logs hard to read

VERIFIED FIXED

Status

Release Engineering
General
VERIFIED FIXED
11 years ago
4 years ago

People

(Reporter: bz, Assigned: alice)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

The data Tp2 puts in the tinderbox log is completely unreadable -- it's all on one line, with no indication of what the numbers mean.

Compare what the Tp output looks like...

Comment 1

11 years ago
I think there are two options here:

1) Rig up a way to access the pretty HTML page Tp2 produces for each run from Tinderbox.
2) Write some code that produces useful plain text output instead of a one-line dump.

I guess 2) would probably be easier, but it would be nice to use the existing HTML page that's already produced if we could make that accessible somehow.

Updated

11 years ago
Duplicate of this bug: 348376
(In reply to comment #1)
> I think there are two options here:
> 
> 1) Rig up a way to access the pretty HTML page Tp2 produces for each run from
> Tinderbox.
> 2) Write some code that produces useful plain text output instead of a one-line
> dump.
> 
> I guess 2) would probably be easier, but it would be nice to use the existing
> HTML page that's already produced if we could make that accessible somehow.


I think option 2 is better.. we could even just drop the useful HTML from that report into the Tinderbox log if we want, tbox will display it as HTML after all. 

Also note that this test appears in two places in the tree, let's get them synchronized too:

http://mxr.mozilla.org/mozilla/source/testing/performance/talos/page_load_test/
http://mxr.mozilla.org/mozilla/source/tools/performance/pageload/

I think that we should do this work in the "talos" version, and remove the second one from CVS.
OS: Linux → All
To clarify:
http://mxr.mozilla.org/mozilla/source/tools/performance/pageload/

^ This version is used by the bl-bld* servers on the Firefox and Mozilla1.8 tree

http://mxr.mozilla.org/mozilla/source/testing/performance/talos/page_load_test/
^ This is used by the Talos framework, currently qm-pxp01,3 on MozillaTest
Blocks: 386081

Comment 5

11 years ago
Registering a vote for option #2 since that will likely be much easier.  Just putting newlines in the right place will do wonders here..
(Assignee)

Comment 6

10 years ago
Tp2 itself offers to dump data in the tinderbox format (--tpformat option), and talos machines send the information to the graphed on the graph server.

Closing.  Re-open if there's still something missing.
Status: NEW → RESOLVED
Last Resolved: 10 years ago
Resolution: --- → FIXED
What was originally missing is still missing: readable output of the per-page performance data in the Tinderbox log for our default performance test machines.  Please do compare the current output to the output that Tp produced, as I suggest in comment 0.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
(Assignee)

Comment 8

10 years ago
The current output for tp2 results is appearing in the log as:

<snip>
NOISE: _x_x_mozilla_page_load,138.05,NaN,NaN
NOISE: _x_x_mozilla_page_load_details,avgmedian|138.05|average|132.76|minimum|NaN|maximum|NaN|stddev|NaN|0;bugzilla.mozilla.org/index.html;152.5;145.5;138;173;173;139;151;138;154
NOISE: |1;espn.go.com/index.html;147.5;141.75;135;249;249;135;145;137;150
NOISE: |2;home.netscape.com/index.html;139;134.5;128;175;175;132;144;128;134
NOISE: |3;hotwired.lycos.com/index.html;123.5;120.75;115;151;151;115;122;121;125
NOISE: |4;lxr.mozilla.org/index.html;260.5;240.75;221;278;278;221;265;221;256
NOISE: |5;my.netscape.com/index.html;114.5;109.75;105;142;142;105;115;105;114
NOISE: |6;news.cnet.com/index.html;135;128.75;120;165;165;120;136;125;134
NOISE: |7;slashdot.org/index.html;134.5;120.5;100;197;197;113;147;100;122
NOISE: |8;vanilla-page/index.html;61;60.5;57;374;63;57;374;61;61
NOISE: |9;web.icq.com/index.html;177;167.25;156;222;222;156;177;159;177
NOISE: |10;www.altavista.com/index.html;109.5;101.5;96;123;123;96;104;96;110
NOISE: |11;www.amazon.com/index.html;124;122.25;120;160;160;123;125;120;121
NOISE: |12;www.aol.com/index.html;128.5;125.5;119;171;171;119;127;126;130
NOISE: |13;www.apple.com/index.html;90.5;94.5;90;110;110;90;100;97;91

</snip>

I don't have an example of the old format - but this does seem better than the 'all info on a single line' issue.
(Assignee)

Updated

10 years ago
Status: REOPENED → NEW
Component: Testing → Release Engineering: Future
Product: Core → mozilla.org
QA Contact: testing → release
Version: Trunk → other
The old format made it a little clearer which numbers were medians/averages (and listed both) and which were raw data.  I'm not sure which numbers in the new format are which.
(Assignee)

Comment 10

10 years ago
So, if what we have now was tweaked we'd be in a pretty good place?

We're trying to ramp up work on the performance issues that plague the graph server so that this data will be available there - so a quick fix on this side would be ideal.
Summary: Getting useful data out of Tp2 is nearly impossible → default format for tp data in talos logs hard to read
Yeah, I'm not asking for major changes here.  Just adding a line explaining the format of the per-site lines, and a linebreak before that first site, would be great.  If those lines don't include medians/averages, adding those would be pretty nice too, of course.
(Assignee)

Comment 12

10 years ago
Created attachment 333630 [details] [diff] [review]
[Checked in]pageloader output format tweaks (newline, table key)

A couple of small fixes:
- added new line so that first line of per-page information is readable
- added table key so that per-page information can be deciphered

Haven't found an appropriate reviewer for this yet.
Assignee: nobody → anodelman
Status: NEW → ASSIGNED
(Assignee)

Updated

10 years ago
Attachment #333630 - Flags: review?(rcampbell)
Comment on attachment 333630 [details] [diff] [review]
[Checked in]pageloader output format tweaks (newline, table key)

this looks like it should do, inscrutable tinderbox reasons for the NaNs not-withstanding.
Attachment #333630 - Flags: review?(rcampbell) → review+
(Assignee)

Comment 14

10 years ago
Comment on attachment 333630 [details] [diff] [review]
[Checked in]pageloader output format tweaks (newline, table key)

Checking in report.js;
/cvsroot/mozilla/layout/tools/pageloader/report.js,v  <--  report.js
new revision: 1.8; previous revision: 1.7
done
Attachment #333630 - Attachment description: pageloader output format tweaks (newline, table key) → [Checked in]pageloader output format tweaks (newline, table key)
(Assignee)

Comment 15

10 years ago
New output of talos logs:

<snip>
NOISE: _x_x_mozilla_page_load,307.8,NaN,NaN
NOISE: _x_x_mozilla_page_load_details,avgmedian|307.8|average|303.84|minimum|NaN|maximum|NaN|stddev|NaN
NOISE: |i|pagename|median|mean|min|max|runs|
NOISE: |0;3d-cube.html;366.5;370.25;365;1619;1619;383;365;365;368
NOISE: |1;3d-morph.html;332.5;323.5;313;343;330;335;313;316;343
NOISE: |2;3d-raytrace.html;341.5;338.25;333;344;343;333;337;340;344
NOISE: |3;access-binary-trees.html;176.5;174.25;172;188;188;178;172;172;175
NOISE: |4;access-fannkuch.html;385.5;385.75;376;1194;1194;379;396;392;376
NOISE: |5;access-nbody.html;418;415.5;413;430;430;413;419;413;417
NOISE: |6;access-nsieve.html;156.5;154.25;151;159;159;153;157;151;156
NOISE: |7;bitops-3bit-bits-in-byte.html;149.5;148;146;157;147;148;157;146;151
NOISE: |8;bitops-bits-in-byte.html;184.5;182.5;180;195;181;184;195;180;185

</snip>
Status: ASSIGNED → RESOLVED
Last Resolved: 10 years ago10 years ago
Resolution: --- → FIXED
Verified.  That looks great, Alice.  Thanks!
Status: RESOLVED → VERIFIED
Moving closed Future bugs into Release Engineering in preparation for removing the Future component.
Component: Release Engineering: Future → Release Engineering
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.