Closed Bug 619536 Opened 9 years ago Closed 9 years ago

turn off ts_cold tests for all platforms

Categories

(Release Engineering :: General, defect, P2)

x86
Linux
defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: dietrich, Assigned: coop)

Details

Attachments

(1 file)

the ts_cold tests as currently implemented are not providing useful data, and are therefore:

1. wasting resources

2. confusing the reporting of performance results of check-ins

Mac and Linux ts_cold were created in bug 510587, and bug 522807 is for windows.

i'll file a new bug for implementing these tests in a way that makes sense.
Grabbing, because aiui, this looks like it should be a RelEng bug to disable this test suite in production. If I've misunderstood, let me know.

We'll need to announce this before turning off, but great to know we can. Thanks Dietrich.
Component: General → Release Engineering
Product: Testing → mozilla.org
QA Contact: general → release
Version: unspecified → other
(In reply to comment #0)
> the ts_cold tests as currently implemented are not providing useful data

I believe it. What's the rationale, though?
(In reply to comment #2)
> (In reply to comment #0)
> > the ts_cold tests as currently implemented are not providing useful data
> 
> I believe it. What's the rationale, though?

It's counterproductive as deployed. It measures the wrong overhead on osx(ie libraries don't get flushed), and a lot of linux overhead is in loading system libraries which are normally in the cache. These issues combined with relatively non-deterministic seek delays on harddrives makes this test noisy and confusing.

It is possible to make this test less broken, but it will be much more useful to switch from the ts_cold measurement to measuring amount of disk io, ala bug 609111.
1) No objections to disabling in today's platform meeting.

2) I've send a post to dev.planning and dev.tree-management about this.
Assignee: nobody → coop
Status: NEW → ASSIGNED
Priority: -- → P2
Attachment #499368 - Flags: review? → review?(armenzg)
Comment on attachment 499368 [details] [diff] [review]
Turn off ts_cold tests

stamp
Attachment #499368 - Flags: review?(armenzg) → review+
Flags: needs-reconfig+
Status: ASSIGNED → RESOLVED
Closed: 9 years ago
Flags: needs-reconfig+
Resolution: --- → FIXED
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.