378430 - default format for tp data in talos logs hard to read

Status: VERIFIED FIXED

(Release Engineering :: General, defect)

Description

[Boris Zbarsky [:bzbarsky]]

The data Tp2 puts in the tinderbox log is completely unreadable -- it's all on one line, with no indication of what the numbers mean.

Compare what the Tp output looks like...

Comment 1

[Adam Guthrie]

I think there are two options here:

1) Rig up a way to access the pretty HTML page Tp2 produces for each run from Tinderbox.
2) Write some code that produces useful plain text output instead of a one-line dump.

I guess 2) would probably be easier, but it would be nice to use the existing HTML page that's already produced if we could make that accessible somehow.

Comment 3

[Robert Helmer [:rhelmer]]

(In reply to comment #1)
> I think there are two options here:
> 
> 1) Rig up a way to access the pretty HTML page Tp2 produces for each run from
> Tinderbox.
> 2) Write some code that produces useful plain text output instead of a one-line
> dump.
> 
> I guess 2) would probably be easier, but it would be nice to use the existing
> HTML page that's already produced if we could make that accessible somehow.


I think option 2 is better.. we could even just drop the useful HTML from that report into the Tinderbox log if we want, tbox will display it as HTML after all. 

Also note that this test appears in two places in the tree, let's get them synchronized too:

http://mxr.mozilla.org/mozilla/source/testing/performance/talos/page_load_test/
http://mxr.mozilla.org/mozilla/source/tools/performance/pageload/

I think that we should do this work in the "talos" version, and remove the second one from CVS.

Comment 4

[Robert Helmer [:rhelmer]]

To clarify:
http://mxr.mozilla.org/mozilla/source/tools/performance/pageload/

^ This version is used by the bl-bld* servers on the Firefox and Mozilla1.8 tree

http://mxr.mozilla.org/mozilla/source/testing/performance/talos/page_load_test/
^ This is used by the Talos framework, currently qm-pxp01,3 on MozillaTest

Comment 5

[Mike Schroepfer]

Registering a vote for option #2 since that will likely be much easier.  Just putting newlines in the right place will do wonders here..

Comment 6

[alice nodelman [:alice] [:anode]]

Tp2 itself offers to dump data in the tinderbox format (--tpformat option), and talos machines send the information to the graphed on the graph server.

Closing.  Re-open if there's still something missing.

Comment 7

[Boris Zbarsky [:bzbarsky]]

What was originally missing is still missing: readable output of the per-page performance data in the Tinderbox log for our default performance test machines.  Please do compare the current output to the output that Tp produced, as I suggest in comment 0.

Comment 8

[alice nodelman [:alice] [:anode]]

The current output for tp2 results is appearing in the log as:

<snip>
NOISE: _x_x_mozilla_page_load,138.05,NaN,NaN
NOISE: _x_x_mozilla_page_load_details,avgmedian|138.05|average|132.76|minimum|NaN|maximum|NaN|stddev|NaN|0;bugzilla.mozilla.org/index.html;152.5;145.5;138;173;173;139;151;138;154
NOISE: |1;espn.go.com/index.html;147.5;141.75;135;249;249;135;145;137;150
NOISE: |2;home.netscape.com/index.html;139;134.5;128;175;175;132;144;128;134
NOISE: |3;hotwired.lycos.com/index.html;123.5;120.75;115;151;151;115;122;121;125
NOISE: |4;lxr.mozilla.org/index.html;260.5;240.75;221;278;278;221;265;221;256
NOISE: |5;my.netscape.com/index.html;114.5;109.75;105;142;142;105;115;105;114
NOISE: |6;news.cnet.com/index.html;135;128.75;120;165;165;120;136;125;134
NOISE: |7;slashdot.org/index.html;134.5;120.5;100;197;197;113;147;100;122
NOISE: |8;vanilla-page/index.html;61;60.5;57;374;63;57;374;61;61
NOISE: |9;web.icq.com/index.html;177;167.25;156;222;222;156;177;159;177
NOISE: |10;www.altavista.com/index.html;109.5;101.5;96;123;123;96;104;96;110
NOISE: |11;www.amazon.com/index.html;124;122.25;120;160;160;123;125;120;121
NOISE: |12;www.aol.com/index.html;128.5;125.5;119;171;171;119;127;126;130
NOISE: |13;www.apple.com/index.html;90.5;94.5;90;110;110;90;100;97;91

</snip>

I don't have an example of the old format - but this does seem better than the 'all info on a single line' issue.

Comment 9

[Boris Zbarsky [:bzbarsky]]

The old format made it a little clearer which numbers were medians/averages (and listed both) and which were raw data.  I'm not sure which numbers in the new format are which.

Comment 10

[alice nodelman [:alice] [:anode]]

So, if what we have now was tweaked we'd be in a pretty good place?

We're trying to ramp up work on the performance issues that plague the graph server so that this data will be available there - so a quick fix on this side would be ideal.

Comment 11

[Boris Zbarsky [:bzbarsky]]

Yeah, I'm not asking for major changes here.  Just adding a line explaining the format of the per-site lines, and a linebreak before that first site, would be great.  If those lines don't include medians/averages, adding those would be pretty nice too, of course.

Comment 12

[alice nodelman [:alice] [:anode]]

Attachment: [Checked in]pageloader output format tweaks (newline, table key)

A couple of small fixes:
- added new line so that first line of per-page information is readable
- added table key so that per-page information can be deciphered

Haven't found an appropriate reviewer for this yet.

Comment 13

[Rob Campbell [:rc] (:robcee)]

Comment on attachment 333630 [details] [diff] [review]
[Checked in]pageloader output format tweaks (newline, table key)

this looks like it should do, inscrutable tinderbox reasons for the NaNs not-withstanding.

Comment 14

[alice nodelman [:alice] [:anode]]

Comment on attachment 333630 [details] [diff] [review]
[Checked in]pageloader output format tweaks (newline, table key)

Checking in report.js;
/cvsroot/mozilla/layout/tools/pageloader/report.js,v  <--  report.js
new revision: 1.8; previous revision: 1.7
done

Comment 15

[alice nodelman [:alice] [:anode]]

New output of talos logs:

<snip>
NOISE: _x_x_mozilla_page_load,307.8,NaN,NaN
NOISE: _x_x_mozilla_page_load_details,avgmedian|307.8|average|303.84|minimum|NaN|maximum|NaN|stddev|NaN
NOISE: |i|pagename|median|mean|min|max|runs|
NOISE: |0;3d-cube.html;366.5;370.25;365;1619;1619;383;365;365;368
NOISE: |1;3d-morph.html;332.5;323.5;313;343;330;335;313;316;343
NOISE: |2;3d-raytrace.html;341.5;338.25;333;344;343;333;337;340;344
NOISE: |3;access-binary-trees.html;176.5;174.25;172;188;188;178;172;172;175
NOISE: |4;access-fannkuch.html;385.5;385.75;376;1194;1194;379;396;392;376
NOISE: |5;access-nbody.html;418;415.5;413;430;430;413;419;413;417
NOISE: |6;access-nsieve.html;156.5;154.25;151;159;159;153;157;151;156
NOISE: |7;bitops-3bit-bits-in-byte.html;149.5;148;146;157;147;148;157;146;151
NOISE: |8;bitops-bits-in-byte.html;184.5;182.5;180;195;181;184;195;180;185

</snip>

Comment 16

[Boris Zbarsky [:bzbarsky]]

Verified.  That looks great, Alice.  Thanks!

Comment 17

[Chris Cooper [:coop] (he/him)]

Moving closed Future bugs into Release Engineering in preparation for removing the Future component.