Closed Bug 386995 Opened 14 years ago Closed 13 years ago

Add Tdhtml to Talos

Categories

(Release Engineering :: General, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: mtschrep, Assigned: anodelman)

References

()

Details

Attachments

(2 files, 1 obsolete file)

Blocks: 386081
Depends on: talos-generalize
Depends on: 397004
Depends on: 397126
From the work that vlad did to make the existing dhtml/gfx/svg tests work with the pageloader, I've made the integration of these tests into talos.

I'm currently not doing any memory/cpu monitoring while the tests are run, simply recording the time that each tests takes to run.

Works on windows and mac, linux has not yet been tested.

This patch includes just the changes made to talos, more attachments to follow.
Assignee: nobody → anodelman
Status: NEW → ASSIGNED
Attachment #282336 - Flags: review?(bhearsum)
The tests themselves don't make much sense as a patch (full of images, also huge) so I created a zip at 

http://people.mozilla.org/~anodelman/new-tests.zip

unzip in page_load_test directory to run the tests.
Comment on attachment 282336 [details] [diff] [review]
integrates dhtml/svg/gfx tests into talos

r+ to this.

I'm not sure I agree about keeping the tests out of the tree. As long as there isn't a really compelling reason to keep them out (such as copyright issues like with the pageset) I think they should be put into the tree. Keeping them outside puts an additional burden on configuration. For Buildbot setup and standalone Talos I think the fewer places to download/checkout from, the better. As it stands now Talos must be checked out and a the pageset installed. Adding a "download x" requirement is going to be a pain.

I'm not too familiar with the nature of the tests, whether they are Talos specific or not, etc. So, there may be a reason to keep them out of Talos that's gone over my head.
Attachment #282336 - Flags: review?(bhearsum) → review+
Ah - I've gone and been confusing.  Yes, the tests will be checked in to the tree in talos/page_load_test.  I didn't want to submit them as a patch because they aren't really a change set and they are also full of binary files (images and such).
Ah, ok. Well, everything sounds good to me then!
The -tprender flag is required for the correct execution of the tgfx test.
Attachment #282336 - Attachment is obsolete: true
Attachment #282441 - Flags: review?(bhearsum)
Attachment #282441 - Flags: review?(bhearsum) → review+
Checking in run_tests.py;
/cvsroot/mozilla/testing/performance/talos/run_tests.py,v  <--  run_tests.py
new revision: 1.12; previous revision: 1.11
done
Checking in sample.config;
/cvsroot/mozilla/testing/performance/talos/sample.config,v  <--  sample.config
new revision: 1.8; previous revision: 1.7
done

Also checked in
page_load_test/dhtml
page_load_test/svg
page_load_test/gfx
Status: ASSIGNED → RESOLVED
Closed: 13 years ago
Resolution: --- → FIXED
There's a problem with the -tprender flag throwing off how the data from the browser is examined as it adds a 'Render report' to the end of the pageloader results.  This causes an exception and crashes talos.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Depends on: 397831
This should fix the -tprender issue in combination with bug 397831
Attachment #282617 - Flags: review?(bhearsum)
Comment on attachment 282617 [details] [diff] [review]
make talos ignore incorrectly formatted browser data

>-        for mypage in page_results[2:]:
>+        for mypage in page_results:

Why are we ignoring the first two page_results now?

Other than that, looks fine.
The first two lines are of the format:

    report += "_x_x_mozilla_page_load,"+avgmed+",NaN,NaN\n";
    report += "_x_x_mozilla_page_load_details,avgmedian|"+avgmed+"|average|"+avg.toFixed(2)+"|minimum|NaN|maximum|NaN|stddev|NaN";

They do not contain information that we are interested in.  For the render report there is an additional "***** Render report *****" that also needs to be ignored.  With the new solution I just ignored all lines that do not have numerical results in them - instead of having to code in specific exceptions for specific formats.
Attachment #282617 - Flags: review?(bhearsum) → review+
Checking in run_tests.py;
/cvsroot/mozilla/testing/performance/talos/run_tests.py,v  <--  run_tests.py
new revision: 1.13; previous revision: 1.12
done
Status: REOPENED → RESOLVED
Closed: 13 years ago13 years ago
Resolution: --- → FIXED
Component: Testing → Release Engineering
Product: Core → mozilla.org
QA Contact: testing → release
Version: unspecified → other
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.