534819 - Create new Talos suite: tscroll

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P3)

Description

[Jeff Muizelaar [:jrmuizel]]

Attachment: add tscroll test

- I'll be the contact person for this test suite.

- This test tests scrolling performance on a set of pages. The result from each page is average scroll time in nanoseconds.

- For now this should only run on trunk on windows, linux and os x.

- Note: The patch contains binaries so it's in git format.

Comment 1

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

Moving to Future, until we sort priority of this new suite along with the other requests for new talos suites.

Comment 2

[Lukas Blakk [:lsblakk] use ?needinfo]

too busy with getting try-as-branch in config to touch this in a while, so putting back in the pool.

Comment 3

[Chris Cooper [:coop] (he/him)]

Mass move of bugs from Release Engineering:Future -> Release Engineering. See
http://coop.deadsquid.com/2010/02/kiss-the-future-goodbye/ for more details.

Comment 4

[Jeff Muizelaar [:jrmuizel]]

It's been nearly 3 months since I filed this. When can I expect it to be deployed?

Comment 5

[alice nodelman [:alice] [:anode]]

Has this been tested with standlone talos - does it actually run in the talos framework?

Comment 6

[Jeff Muizelaar [:jrmuizel]]

Attachment: zipped version

Comment 7

[alice nodelman [:alice] [:anode]]

We are currently upgrading talos staging, so this will have to wait till we have machines on which to test.

Comment 8

[Damon Sicore (:damons)]

jrmuizel, can you answer the question in Comment 5 above?

We've the Layers project in motion right now which is specifically trying to improve scrolling.  OOPP may also affect scrolling in ways we can't predict.  These two things alone are good reasons to bump this up in priority.  

Alis, how long do we expect the talos staging upgrade to take?  Are there other dependencies?  Anything we can do to help?

Comment 9

[Jeff Muizelaar [:jrmuizel]]

(In reply to comment #5)
> Has this been tested with standlone talos - does it actually run in the talos
> framework?

Yes.

Comment 10

[Damon Sicore (:damons)]

(In reply to comment #8)
 
> Alice, how long do we expect the talos staging upgrade to take?  Are there other
> dependencies?  Anything we can do to help?

Corrected spelling of Alice, and hoping to get more details.  Any news?  :)

Comment 11

[alice nodelman [:alice] [:anode]]

This is waiting on:

- successful run through talos staging (has been recently blocked as we've been rebuilding talos staging)
- graph server patches to add new test name to staging/production graph server

I will handle the staging, but this is not currently my top priority.  Will update bug as progress is made.

Comment 12

[alice nodelman [:alice] [:anode]]

This is currently blocked on availability of staging, which is in use for getting unit tests running on talos boxes.  I'd say that we are 2 weeks out from being able to do a staging run, unless I can find some availability without disrupting the current tests.

Comment 13

[alice nodelman [:alice] [:anode]]

I'm setting up a secondary talos staging instance with borrowed production talos slaves as covered in meetings with shaver + joduinn.

I should have initial staging of tscroll overnight.

Comment 14

[alice nodelman [:alice] [:anode]]

Attachment: [checked in]add scroll test to graph server

Comment 15

[alice nodelman [:alice] [:anode]]

Attachment: add scroll test to graph server, sql script

Comment 16

[alice nodelman [:alice] [:anode]]

On what operation systems/branches do we want tscroll run?

Comment 17

[alice nodelman [:alice] [:anode]]

Attachment: [checked in]add tscroll to talos code base (updated talos configs/manifests)

Comment 18

[Jeff Muizelaar [:jrmuizel]]

(In reply to comment #16)
> On what operation systems/branches do we want tscroll run?

I can't really think of any place not to run it. However, I didn't test it on fennec, so I'm not sure how well it works there.

Comment 19

[alice nodelman [:alice] [:anode]]

Attachment: [checked in]turn on tscroll everywhere (all platforms + branches)

Comment 20

[alice nodelman [:alice] [:anode]]

Graph server updates applied to both stage and production dbs.

Comment 21

[alice nodelman [:alice] [:anode]]

Comment on attachment 434425 [details] [diff] [review]
[checked in]add scroll test to graph server

changeset:   286:4b4ca01da757

Comment 22

[alice nodelman [:alice] [:anode]]

Comment on attachment 434674 [details] [diff] [review]
[checked in]turn on tscroll everywhere (all platforms + branches)

changeset:   2213:043064228efb

Comment 23

[alice nodelman [:alice] [:anode]]

Comment on attachment 434429 [details] [diff] [review]
[checked in]add tscroll to talos code base (updated talos configs/manifests)

RCS file: /cvsroot/mozilla/testing/performance/talos/page_load_test/scroll/drac.htm,v
done
Checking in page_load_test/scroll/drac.htm;
/cvsroot/mozilla/testing/performance/talos/page_load_test/scroll/drac.htm,v  <--  drac.htm
initial revision: 1.1
done
RCS file: /cvsroot/mozilla/testing/performance/talos/page_load_test/scroll/iframe.svg,v
done
Checking in page_load_test/scroll/iframe.svg;
/cvsroot/mozilla/testing/performance/talos/page_load_test/scroll/iframe.svg,v  <--  iframe.svg
initial revision: 1.1
done
RCS file: /cvsroot/mozilla/testing/performance/talos/page_load_test/scroll/reader-scroll.js,v
done
Checking in page_load_test/scroll/reader-scroll.js;
/cvsroot/mozilla/testing/performance/talos/page_load_test/scroll/reader-scroll.js,v  <--  reader-scroll.js
initial revision: 1.1
done
RCS file: /cvsroot/mozilla/testing/performance/talos/page_load_test/scroll/reader.css,v
done
Checking in page_load_test/scroll/reader.css;
/cvsroot/mozilla/testing/performance/talos/page_load_test/scroll/reader.css,v  <--  reader.css
initial revision: 1.1
done
RCS file: /cvsroot/mozilla/testing/performance/talos/page_load_test/scroll/reader.htm,v
done
Checking in page_load_test/scroll/reader.htm;
/cvsroot/mozilla/testing/performance/talos/page_load_test/scroll/reader.htm,v  <--  reader.htm
initial revision: 1.1
done
RCS file: /cvsroot/mozilla/testing/performance/talos/page_load_test/scroll/scroll-frame.js,v
done
Checking in page_load_test/scroll/scroll-frame.js;
/cvsroot/mozilla/testing/performance/talos/page_load_test/scroll/scroll-frame.js,v  <--  scroll-frame.js
initial revision: 1.1
done
RCS file: /cvsroot/mozilla/testing/performance/talos/page_load_test/scroll/scroll.js,v
done
Checking in page_load_test/scroll/scroll.js;
/cvsroot/mozilla/testing/performance/talos/page_load_test/scroll/scroll.js,v  <--  scroll.js
initial revision: 1.1
done
RCS file: /cvsroot/mozilla/testing/performance/talos/page_load_test/scroll/scroll.manifest,v
done
Checking in page_load_test/scroll/scroll.manifest;
/cvsroot/mozilla/testing/performance/talos/page_load_test/scroll/scroll.manifest,v  <--  scroll.manifest
initial revision: 1.1
done
RCS file: /cvsroot/mozilla/testing/performance/talos/page_load_test/scroll/strips-single.png,v
done
Checking in page_load_test/scroll/strips-single.png;
/cvsroot/mozilla/testing/performance/talos/page_load_test/scroll/strips-single.png,v  <--  strips-single.png
initial revision: 1.1
done
RCS file: /cvsroot/mozilla/testing/performance/talos/page_load_test/scroll/strips.png,v
done
Checking in page_load_test/scroll/strips.png;
/cvsroot/mozilla/testing/performance/talos/page_load_test/scroll/strips.png,v  <--  strips.png
initial revision: 1.1
done
RCS file: /cvsroot/mozilla/testing/performance/talos/page_load_test/scroll/tiled-downscale.html,v
done
Checking in page_load_test/scroll/tiled-downscale.html;
/cvsroot/mozilla/testing/performance/talos/page_load_test/scroll/tiled-downscale.html,v  <--  tiled-downscale.html
initial revision: 1.1
done
RCS file: /cvsroot/mozilla/testing/performance/talos/page_load_test/scroll/tiled-fixed-downscale.html,v
done
Checking in page_load_test/scroll/tiled-fixed-downscale.html;
/cvsroot/mozilla/testing/performance/talos/page_load_test/scroll/tiled-fixed-downscale.html,v  <--  tiled-fixed-downscale.html
initial revision: 1.1
done
RCS file: /cvsroot/mozilla/testing/performance/talos/page_load_test/scroll/tiled-fixed.html,v
done
Checking in page_load_test/scroll/tiled-fixed.html;
/cvsroot/mozilla/testing/performance/talos/page_load_test/scroll/tiled-fixed.html,v  <--  tiled-fixed.html
initial revision: 1.1
done
RCS file: /cvsroot/mozilla/testing/performance/talos/page_load_test/scroll/tiled.html,v
done
Checking in page_load_test/scroll/tiled.html;
/cvsroot/mozilla/testing/performance/talos/page_load_test/scroll/tiled.html,v  <--  tiled.html
initial revision: 1.1
Checking in sample.config;
/cvsroot/mozilla/testing/performance/talos/sample.config,v  <--  sample.config
new revision: 1.41; previous revision: 1.40
done

Comment 24

[alice nodelman [:alice] [:anode]]

Up and running.  All done here.

Comment 25

[(no longer active)]

Can this be marked as RESOLVED FIXED?

Comment 26

[Robert O'Callahan (:roc) (email my personal email if necessary)]

This needs to be backed out. The test is permanently orange/red on Linux. See bug 555352.

Comment 27

[alice nodelman [:alice] [:anode]]

If a stack is generated and read on linux32 shouldn't that be filed as a crash and then fixed?  I'm not convinced that the problem isn't in the browser - and I wouldn't want to disable the test if it is showing a 'real' crash.

Comment 28

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Nick hid the failing Talos boxes, which is probably better than backing this out, yes.

Comment 29

[alice nodelman [:alice] [:anode]]

We are going to track failures of this test here bug 555352.

But the rollout of the test was successful, and the work for this bug is done.

Comment 30

[Jeff Muizelaar [:jrmuizel]]

I don't know why we are crashing. However, it seems like tiled-fixed-downscale.html could be taking a really long time. Can we adjust the timeouts?

Comment 31

[alice nodelman [:alice] [:anode]]

Increase timeout patch in bug 555352 - not adding to this bug as it is closed.