Intermittent OS X debug "Assertion failure: IsValidNetAddr(addr) == PR_TRUE, at ../../../../../nsprpub/pr/src/pthreads/ptio.c:1711" (followed by many failures) caused by DNS/network issues

RESOLVED WORKSFORME

Status

RESOLVED WORKSFORME
6 years ago
4 years ago

People

(Reporter: dholbert, Unassigned)

Tracking

({intermittent-failure})

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

6 years ago
At some point tonight, we started getting Mac OS X 10.6 debug oranges starting with...
{
Assertion failure: IsValidNetAddr(addr) == PR_TRUE, at ../../../../../nsprpub/pr/src/pthreads/ptio.c:1711
}
...and followed by tons of failures, like:
{
TEST-UNEXPECTED-FAIL | chrome://mochitests/content/browser/browser/components/thumbnails/test/browser_thumbnails_privacy.js | Test timed out
TEST-UNEXPECTED-FAIL | chrome://mochitests/content/browser/browser/devtools/styleeditor/test/browser_styleeditor_enabled.js | Test timed out
TEST-UNEXPECTED-FAIL | chrome://mochitests/content/browser/browser/devtools/webconsole/test/browser_webconsole_bug_737873_mixedcontent.js | Timed out while waiting for: mixed content warning displayed successfully
TEST-UNEXPECTED-FAIL | chrome://mochitests/content/browser/browser/devtools/webconsole/test/browser_webconsole_bug_770099_bad_policyuri.js | Test timed out
TEST-UNEXPECTED-FAIL | chrome://mochitests/content/browser/browser/devtools/webconsole/test/browser_webconsole_bug_770099_violation.js | Test timed out
TEST-UNEXPECTED-FAIL | chrome://mochitests/content/browser/netwerk/test/browser/browser_NetUtil.js | request succeeded
TEST-UNEXPECTED-FAIL | chrome://mochitests/content/browser/netwerk/test/browser/browser_NetUtil.js | Test timed out
}
https://tbpl.mozilla.org/php/getParsedLog.php?id=17368005&tree=Mozilla-Inbound
(I only pasted the first few from that log)

Some sample logs:
bc:
https://tbpl.mozilla.org/php/getParsedLog.php?id=17368005&tree=Mozilla-Inbound
https://tbpl.mozilla.org/php/getParsedLog.php?id=17367744&tree=Mozilla-Inbound
https://tbpl.mozilla.org/php/getParsedLog.php?id=17367976&tree=Mozilla-Inbound
https://tbpl.mozilla.org/php/getParsedLog.php?id=17367168&tree=Mozilla-Inbound
m-oth:
https://tbpl.mozilla.org/php/getParsedLog.php?id=17368059&tree=Mozilla-Inbound
m-3:
https://tbpl.mozilla.org/php/getParsedLog.php?id=17367486&tree=Mozilla-Inbound

I suspect this might be a time-bomb or a machine configuration issue, since it started in different tests at different times and the first "guilty" changeset that I found (w/ this orange on "bc") was this push, which just tweaked a mochitest & couldn't have affected bc testruns:
   https://tbpl.mozilla.org/?tree=Mozilla-Inbound&rev=62bb8cc3277b
(Reporter)

Updated

6 years ago
Summary: Frequent Mac OS X 10.6 debug mac oranges w/ "Assertion failure: IsValidNetAddr(addr) == PR_TRUE, at nsprpub/pr/src/pthreads/ptio.c:1711" → Frequent (perma-orange?) Mac OS X 10.6 debug mac oranges w/ "Assertion failure: IsValidNetAddr(addr) == PR_TRUE, at nsprpub/pr/src/pthreads/ptio.c:1711"
Smells like zombocom. Do the failing test(s) rely on external resources on the network ?
(In reply to Daniel Holbert [:dholbert] from comment #0)
> https://tbpl.mozilla.org/php/getParsedLog.php?id=17368005&tree=Mozilla-
> Inbound

There is a screenshot in this log with a Firefox window saying 'The proxy server is refusing connections' (search for SCREENSHOT in the logs). It appears to be the same present in other logs too.

Comment 3

6 years ago
This happened recently on aurora/beta, but I can't for the life of me remember what the root cause was there.
Keywords: intermittent-failure

Comment 4

6 years ago
Inbound has been closed since 0925 UTC; awaiting retriggers.
I imagine they'll come back green, but hey.

Comment 5

6 years ago
All green. Trees reopened; we should still sort out these tests hitting the network...
According to my vague memory of the last time, there are two sorts of failures - the tests which expect to get an errorpage from loading this.is.not.a.valid.domain.ajkslfjsl time out, and, httpd.js fails to start up the https server because it hits that assertion which causes every test which tries to load a proxied https URL to fail, both apparently related to DNS being busted.

We don't want tests hitting the network, but requiring tests to cope with DNS being available but broken in some unknown way seems a bit excessive.

Updated

6 years ago
Summary: Frequent (perma-orange?) Mac OS X 10.6 debug mac oranges w/ "Assertion failure: IsValidNetAddr(addr) == PR_TRUE, at nsprpub/pr/src/pthreads/ptio.c:1711" → Intermittent OS X debug "Assertion failure: IsValidNetAddr(addr) == PR_TRUE, at ../../../../../nsprpub/pr/src/pthreads/ptio.c:1711" caused by DNS/network issues

Comment 7

6 years ago
https://tbpl.mozilla.org/php/getParsedLog.php?id=17583212&tree=Mozilla-Inbound
https://tbpl.mozilla.org/php/getParsedLog.php?id=17583454&tree=Mozilla-Inbound
https://tbpl.mozilla.org/php/getParsedLog.php?id=17583691&tree=Mozilla-Inbound
https://tbpl.mozilla.org/php/getParsedLog.php?id=17583555&tree=Mozilla-Inbound

Summary should now be starrable.
Summary: Intermittent OS X debug "Assertion failure: IsValidNetAddr(addr) == PR_TRUE, at ../../../../../nsprpub/pr/src/pthreads/ptio.c:1711" caused by DNS/network issues → Intermittent OS X debug "Assertion failure: IsValidNetAddr(addr) == PR_TRUE, at ../../../../../nsprpub/pr/src/pthreads/ptio.c:1711" (followed by many failures) caused by DNS/network issues

Comment 8

6 years ago
(In reply to Phil Ringnalda (:philor) from comment #6)
> According to my vague memory of the last time...

Thank you :-)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
(Reporter)

Comment 16

6 years ago
(In reply to Nick Thomas [:nthomas] from comment #2)
> (In reply to Daniel Holbert [:dholbert] from comment #0)
> > https://tbpl.mozilla.org/php/getParsedLog.php?id=17368005&tree=Mozilla-
> > Inbound
> 
> There is a screenshot in this log with a Firefox window saying 'The proxy
> server is refusing connections' (search for SCREENSHOT in the logs). It
> appears to be the same present in other logs too.

There's one of these in the first log of this morning's spike of these (Comment 9).  However, the screenshot in Comment 15 doesn't have this message -- it's just got a blank Nightly window.

This could just be a difference in the tests that are tripping over this in the two cases. (comment 9 is M-1, Comment 15 is M-b-c)...  Just wanted to make it known that the proxy refusing connections page isn't always visible, in instances of this assertion-failure.
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Resolving WFM keyword:intermittent-failure bugs last modified >3 months ago, whose whiteboard contains none of:
{random,disabled,marked,fuzzy,todo,fails,failing,annotated,time-bomb,leave open}

There will inevitably be some false positives; for that (and the bugspam) I apologise. Filter on orangewfm.
Status: NEW → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → WORKSFORME
(Assignee)

Updated

5 years ago
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.