Closed
Bug 785236
Opened 12 years ago
Closed 12 years ago
Add tinderbox threshold for enablePrivilege
Categories
(Testing :: General, defect)
Testing
General
Tracking
(Not tracked)
RESOLVED
WONTFIX
People
(Reporter: bholley, Unassigned)
Details
Now that bug 757046, enablePrivilege is officially on life support within our test suite. I think it would make sense to add something to automation to count the number of enablePrivilege calls, and turn the tree orange if it passes a certain threshold. This would prevent new uses from creeping back in, as they did in bug 760802. Joel, is this something you know how to do? I can write the platform code to spit out the numbers on shutdown if you tell me how to format it.
Comment 1•12 years ago
|
||
I think that this could be riddled with trouble. The main reason for saying that is that we run different tests on each platform, so this threshold would be different for each type of test we run. This means that M1 will be different than M5 and that will be different between opt and debug and pgo builds, and finally all platforms will have different tests. To make matters more confusing if you have a defined threshold value for M1, that set of tests will change in a short amount of time when people add new tests. The danger is that if we have 30 instances in M1 and some tests get bumped into M2, we might have 28 instances in M1 and 2 new ones in M2. The easiest way to get a job to turn orange is to add: TEST-UNEXPECTED-FAIL in the logfile somewhere.
Reporter | ||
Comment 2•12 years ago
|
||
Thanks for the analysis, Joel. I don't think this is worth the trouble.
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → WONTFIX
You need to log in
before you can comment on or make changes to this bug.
Description
•