1235089 - Intermittent test_ocsp_stapling.js | Test timed out

Status: RESOLVED FIXED

(Core :: Security: PSM, defect, P1)

Description

[Kartikaya Gupta (email:[e-mail])]

+++ This bug was initially created as a clone of Bug #1129771 +++

This intermittent failure from bug 1129771 is still happening, but mostly on B2G now. Cloning the bug rather than reopening because it's been closed for a long time and reopening it is probably going to get confusing.

Comment 1

[Edgar Chen [:edgar]]

This also happens on emulator-x86-kk very often, like https://treeherder.mozilla.org/#/jobs?repo=try&revision=3779955620ed.

Comment 2

[Intermittent Failures Robot]

12 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-inbound: 8
* b2g-inbound: 3
* fx-team: 1

Platform breakdown:
* b2g-emu-ics: 12

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1235089&startday=2015-12-28&endday=2016-01-03&tree=all

Comment 3

[Intermittent Failures Robot]

29 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* fx-team: 11
* mozilla-inbound: 8
* b2g-inbound: 6
* mozilla-central: 3
* try: 1

Platform breakdown:
* b2g-emu-ics: 25
* osx-10-10: 4

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1235089&startday=2016-01-04&endday=2016-01-10&tree=all

Comment 4

[Intermittent Failures Robot]

15 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-inbound: 9
* b2g-inbound: 3
* mozilla-central: 2
* fx-team: 1

Platform breakdown:
* b2g-emu-ics: 13
* osx-10-6: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1235089&startday=2016-01-11&endday=2016-01-17&tree=all

Comment 5

[Ethan Tseng [:ethan]]

Hi David,
Could you help to provide some advice on this bug?

Comment 6

[Dana Keeler (she/her) (use needinfo) [:keeler]]

Well, test_ocsp_stapling.js is a long test and the emulators are slow. Either splitting the test up or upping the timeout limit should help address this.

Comment 7

[:Cykesiopka]

Attachment: MozReview Request: Bug 1235089 - Split out OCSP Must Staple tests from test_ocsp_stapling.js to avoid intermittent time outs.

test_ocsp_stapling.js can take ~290s to run on e.g. b2g-emu-x86-kk, which is very close to the default 300s limit.
Splitting out some tests should reduce the intermittent time outs.

Review commit: https://reviewboard.mozilla.org/r/31913/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/31913/

Comment 8

[:Cykesiopka]

https://reviewboard.mozilla.org/r/31913/#review28661

::: security/manager/ssl/tests/unit/test_ocsp_must_staple.js:9
(Diff revision 1)
> -// things happen and bad things don't.
> +// extension is present in intermediate and end-entitie certificates.

Oops: s/end-entitie/end-entity/.

Comment 9

[:Cykesiopka]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=48a9b432f981
https://treeherder.mozilla.org/#/jobs?repo=try&revision=1d6eee26d1f3

b2g-emu-x86-kk goes from ~290s to ~240s, which seems good enough for now.
Presumably this will also help b2g-emu-ics, but I've long given up on that platform.

Comment 10

[Dana Keeler (she/her) (use needinfo) [:keeler]]

Comment on attachment 8710957 [details]
MozReview Request: Bug 1235089 - Split out OCSP Must Staple tests from test_ocsp_stapling.js to avoid intermittent time outs.

https://reviewboard.mozilla.org/r/31913/#review28709

Good stuff. Hopefully this is enough separated out, but we can always split it again.

::: security/manager/ssl/tests/unit/test_ocsp_must_staple.js:26
(Diff revision 1)
>      clearSessionCache();

Are any testcases run before this point? I don't think we need to call clearSessionCache here. (Similarly, I think we can set security.ssl.enable_ocsp_must_staple directly (i.e. not in a add_test block) this first time.)

::: security/manager/ssl/tests/unit/test_ocsp_must_staple.js:40
(Diff revision 1)
>    // We disable OCSP must-staple in the next two tests so we can perform checks

I think this comment was incorrect - it should be "We disable OCSP stapling in the next two tests...", right?

Comment 11

[Intermittent Failures Robot]

60 automation job failures were associated with this bug yesterday.

Repository breakdown:
* mozilla-inbound: 45
* fx-team: 10
* b2g-inbound: 3
* mozilla-central: 2

Platform breakdown:
* b2g-emu-x86-kk: 56
* b2g-emu-ics: 3
* linux64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1235089&startday=2016-01-22&endday=2016-01-22&tree=all

Comment 12

[:Cykesiopka]

https://reviewboard.mozilla.org/r/31913/#review28709

Thanks for the review.

> Are any testcases run before this point? I don't think we need to call clearSessionCache here. (Similarly, I think we can set security.ssl.enable_ocsp_must_staple directly (i.e. not in a add_test block) this first time.)

Done.

> I think this comment was incorrect - it should be "We disable OCSP stapling in the next two tests...", right?

Yes, you're correct. I'll change the comment.

Comment 13

[:Cykesiopka]

Attachment: bug1235089_split-out-ocsp-must-staple-tests-avoid-timeouts_v2.patch

Comment 14

[:Cykesiopka]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=e3eaf15643f4

(The X2 failures look unrelated.)

Comment 15

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/bbfbd1e73bf1

Comment 16

[Phil Ringnalda (:philor)]

https://hg.mozilla.org/mozilla-central/rev/bbfbd1e73bf1

Comment 17

[Intermittent Failures Robot]

108 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-inbound: 72
* fx-team: 20
* b2g-inbound: 9
* mozilla-central: 7

Platform breakdown:
* b2g-emu-x86-kk: 91
* b2g-emu-ics: 16
* linux64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1235089&startday=2016-01-18&endday=2016-01-24&tree=all