1581693 - Intermittent [ FAILED ] DatagramFragment13/TlsFragmentationAndRecoveryTest.DropFirstHalf/0, where GetParam() = true (1202 ms)

Status: REOPENED

(NSS :: Libraries, defect, P3)

Description

[Treeherder Bug Filer]

Filed by: jjones [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=266821836&repo=nss
Full log: https://queue.taskcluster.net/v1/task/DQDiNp6ARAWrCd4KJQ4--Q/runs/0/artifacts/public/logs/live_backing.log

---

Perma-failure starting from Bug 1579290

Comment 1

[J.C. Jones [:jcj] (he/they)]

Marcus, can you poke at why this test is now failing?

Comment 2

[Martin Thomson [:mt:]]

I took a brief look and couldn't reproduce it. One place to start would be to add -v to the invocation of ssl_gtest so that we get more diagnostic output from these test runs.

Comment 3

[Marcus Burghardt]

I couldn't reproduce too, but generated some debug information to analyse and understand what happened. I will try to update here soon.

Comment 4

[Marcus Burghardt]

I tried a lot of ways to reproduce this problem:

• Change timeouts;
• Increase the fragments;
• Manipulate the handshaking;
• Increase the load of my local machine;

The only and closest way to reproduce this problem was manually braking the handshake in a way that one agent never send the last ACK.
At this point I was pretty sure that was a infrastructure problem.
Analysing these failed tasks and many other successful tasks with the same code, I could realize that the MAC instances looked very loaded during the failures, with high times to finish the gtests.

So, I crossed the performance of the tests in mac instances in a interval time before and after the Bug 1579290.
I couldn't find any relationship with the patch.

My conclusion was that these failures were caused by some infrastructure problem, related to high load of the mac instances.
Now, looks that this problem is not appearing anymore. We can leave this BUG open for some days or weeks to confirm that was a temporary problem. I will continue following...

Thanks

Comment 5

[Intermittent Failures Robot]

1 failures in 4350 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• nss: 1

Platform breakdown:

• mac: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1581693&startday=2019-09-16&endday=2019-09-22&tree=all

Comment 6

[Marcus Burghardt]

Looks stable until now. Wait until next Tuesday to conclude.

Comment 7

[Intermittent Failures Robot]

1 failures in 4530 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• nss: 1

Platform breakdown:

• mac: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1581693&startday=2019-09-23&endday=2019-09-29&tree=all

Comment 8

[Marcus Burghardt]

One more report 10 days after the first.
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?startday=2019-09-01&endday=2019-10-01&tree=all&bug=1581693

The second looks pretty similar the first. I continue believing that are external problems.
To be sure of that, some analysis in that specific instances are necessary.
I will continue following that.

Comment 9

[Marcus Burghardt]

Almost 20 days without the problem.
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?startday=2019-09-14&endday=2019-10-14&tree=all&bug=1581693

I am closing this BUG with WORKSFORME status as it couldn't be reproduced and no evidence was found suggesting something related to code BUGs.
Looks to be some temporary infrastructure problem.

Comment 10

[Intermittent Failures Robot]

2 failures in 5483 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• nss: 2

Platform breakdown:

• mac: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1581693&startday=2020-03-02&endday=2020-03-08&tree=all

Comment 11

[Intermittent Failures Robot]

1 failures in 5336 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• nss: 1

Platform breakdown:

• mac: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1581693&startday=2020-03-09&endday=2020-03-15&tree=all

Comment 12

[Intermittent Failures Robot]

1 failures in 4773 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• nss: 1

Platform breakdown:

• mac: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1581693&startday=2020-03-23&endday=2020-03-29&tree=all

Comment 13

[Intermittent Failures Robot]

1 failures in 4949 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• nss: 1

Platform breakdown:

• mac: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1581693&startday=2020-06-01&endday=2020-06-07&tree=all

Comment 14

[BugBot [:suhaib / :marco/ :calixte]]

In the process of migrating remaining bugs to the new severity system, the severity for this bug cannot be automatically determined. Please retriage this bug using the new severity system.