767224 - Remove obsolete Talos tests of JS performance

Status: RESOLVED FIXED

(Testing :: Talos, defect)

Description

[David Mandelin [:dmandelin]]

Please remove these tests from Talos:

  tsspider
  Dromaeo JSLIB
  Dromaeo SunSpider
  Dromaeo Basics

(But keep Dromaeo CSS & DOM.) These are only rough measures of JS performance and we have better ways of tracking it. (Getting good JS tests in Talos would be great, and I'll file some separate bugs on that. No need to wait in deleting the old ones, though.)

Comment 1

[David Mandelin [:dmandelin]]

Please add

  Dromaeo V8

to the delete list as well. I didn't see it int the doc list, but if it's considered a category of tests on its own, it's definitely measuring the V8 benchmark wrong.

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

do you want me to turn these off, or remove them from the talos source as well?

Comment 3

[Jeff Hammel]

In either case, they should be turned off (in buildbot) before being deleted, or else buildbot will just err and dispatch jobs needlessly.

My inclination is to keep the tests in source, at least for the time being.

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

We need a method in talos to show which test we don't run (i.e. deprecated tests).  I know of gfx, and now this list.

If we don't see any value in these tests, I would like to remove them to avoid confusion.  It is really confusing to have tests there and work on talos code that handles these tests and their specific cases when we have no intention of running the tests anymore.

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: removed outdated js talos suites from running (1.0)

This removes the dromaeo tests and the tsspider test.  Note that we run tsspider on mobile and this turns it off by default.  

Armen, can we leave the tdhtml by itself as we will be turning on a new tsspider in the future and it can be bundled up with that chrome/nochrome suite again.

js team: if there is any reason to leave tsspider on for mobile, speak up!

Comment 6

[Armen [:armenzg]]

Comment on attachment 636312 [details] [diff] [review]
removed outdated js talos suites from running (1.0)

Review of attachment 636312 [details] [diff] [review]:
-----------------------------------------------------------------

Sounds good to me.
Please land when you see fit.

Comment 7

[David Mandelin [:dmandelin]]

Turning off only is fine with us. Keeping the source would make it easier to reverse the decision if we decide to in the near future.

We have AWFY coverage for ARMv7, so it's OK to turn off for mobile as well.

Comment 8

[Joel Maher ( :jmaher ) (UTC -8)]

landed on buildbot-configs branch:
http://hg.mozilla.org/build/buildbot-configs/rev/d3f8d66b406f

next step it to get this live in a buildbot-reconfig.  This will only be for mozilla-central/inbound, but will roll through the trains.

Comment 9

[Armen [:armenzg]]

Comment on attachment 636312 [details] [diff] [review]
removed outdated js talos suites from running (1.0)

This is live in production.

Comment 10

[David Mandelin [:dmandelin]]

(In reply to Armen Zambrano G. [:armenzg] - Release Engineer from comment #9)
> Comment on attachment 636312 [details] [diff] [review]
> removed outdated js talos suites from running (1.0)
> 
> This is live in production.

\o/

Thanks!