Closed Bug 1394785 Opened 7 years ago Closed 7 years ago

TestStandardURL NormalizePerfFails triggers only false alarms on OSX

Categories

(Testing :: General, defect)

57 Branch
Unspecified
macOS
defect
Not set
normal

Tracking

(firefox57 fixed)

RESOLVED FIXED
mozilla57
Tracking Status
firefox57 --- fixed

People

(Reporter: igoldan, Assigned: igoldan)

Details

Attachments

(1 file)

I looked over this microbenchmark test. It behaves well on all platforms, except osx-10-10. Since August 1st till now, more than 20 alerts of its sort have been invalid.

Regularly, the test alerts alternatively an improvement and then a regression.
Unless this can be stabilized somehow, I would cancel this test on osx.

To be very precise, I'm talking about TestStandardURL NormalizePerfFails on osx-10-10.
I see 2 options:
1) fix the test to make it more reliable
2) disable the test (or at least reporting of data) for osx
Valentin, can you think of a reason why this happens?
Flags: needinfo?(valentin.gosu)
Hi Ionut,

It's not obvious to me why that would happen. NormalizeIPv4 is fairly self contained, and shouldn't have much variation.
I think disabling the test on osx is OK.
Flags: needinfo?(valentin.gosu)
Assignee: nobody → ionut.goldan
Comment on attachment 8904899 [details]
Bug 1394785 - ignore unstable osx gtest

https://reviewboard.mozilla.org/r/176684/#review181722

that is simple
Attachment #8904899 - Flags: review?(jmaher) → review+
Pushed by jmaher@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/e00a0a13c7fa
ignore unstable osx gtest r=jmaher
https://hg.mozilla.org/mozilla-central/rev/e00a0a13c7fa
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla57
You need to log in before you can comment on or make changes to this bug.