[Gij] RangeError: port should be >= 0 and < 65536: 65537, RangeError: port should be >= 0 and < 65536: 65536 ... Not connected. To write data you must call connect first

RESOLVED FIXED in 2.6 S5 - 1/15

Status

Testing Graveyard
JSMarionette
RESOLVED FIXED
3 years ago
5 months ago

People

(Reporter: mikehenrty, Assigned: mikehenrty)

Tracking

({intermittent-failure})

unspecified
2.6 S5 - 1/15
intermittent-failure

Details

(Whiteboard: [systemsfe][MJS] [CI])

Attachments

(2 attachments)

This problem seems to happen for all Marionette JS tests sporadically. For example [1]. The problem seems to be that emptyPort can't find a valid open port. We should examine what is happening here.

1.) https://treeherder.mozilla.org/logviewer.html#?job_id=19192954&repo=mozilla-inbound
Created attachment 8703459 [details] [review]
[gaia] mikehenrty:bug-1236376-use-more-ports > mozilla-b2g:master
Assignee: nobody → mhenretty
Keywords: intermittent-failure
Whiteboard: [systemsfe][MJS]
Duplicate of this bug: 1236188
Hah, so by default empty-port would use a MAX_PORT of 65537 (65535 being the true valid max), so 2/5507 times it would pick a completely illegal port.  By specifying the max port now you help eliminate that foolishness.

Meta-wise, the randomly try a port solution is strictly worse than just binding to a 0 port to see what the OS allocates since the empty-port implementation could waste a lot of time randomly picking ports that are already taken.

Better yet would be to change the port-assigning dance by having the thing that ends up listening do the bind and then provide it over a named pipe or writing to a file or stdout or something.  That would avoid port allocation races, etc.
Duplicate of this bug: 1236279
Duplicate of this bug: 1236278
Duplicate of this bug: 1236269
Duplicate of this bug: 1236268
Duplicate of this bug: 1236251
Duplicate of this bug: 1236303
Duplicate of this bug: 1236196
Duplicate of this bug: 1236187
Duplicate of this bug: 1236180
Duplicate of this bug: 1228936
Duplicate of this bug: 1236252
Summary: [Gij] RangeError: port should be >= 0 and < 65536: 65537 ... Not connected. To write data you must call connect first → [Gij] RangeError: port should be >= 0 and < 65536: 65537, RangeError: port should be >= 0 and < 65536: 65536 ... Not connected. To write data you must call connect first

Comment 15

3 years ago
27 automation job failures were associated with this bug yesterday.

Repository breakdown:
* mozilla-inbound: 20
* fx-team: 4
* b2g-inbound: 3

Platform breakdown:
* b2g-linux64: 15
* mulet-linux64: 12

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1236376&startday=2016-01-04&endday=2016-01-04&tree=all
Comment on attachment 8703459 [details] [review]
[gaia] mikehenrty:bug-1236376-use-more-ports > mozilla-b2g:master

I received verbal ok from Aus and Gareth yesterday about this bug. Sending over to Guillaume to do final review.
Attachment #8703459 - Flags: review?(gmarty)
Comment on attachment 8703459 [details] [review]
[gaia] mikehenrty:bug-1236376-use-more-ports > mozilla-b2g:master

Looks good to me!
Attachment #8703459 - Flags: review?(gmarty) → review+
master: https://github.com/mozilla-b2g/gaia/commit/16f4490e8208cc69b8e522fb468e9617eee85d53
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
(In reply to Andrew Sutherland [:asuth] from comment #3)
> Hah, so by default empty-port would use a MAX_PORT of 65537 (65535 being the
> true valid max), so 2/5507 times it would pick a completely illegal port. 
> By specifying the max port now you help eliminate that foolishness.
> 
> Meta-wise, the randomly try a port solution is strictly worse than just
> binding to a 0 port to see what the OS allocates since the empty-port
> implementation could waste a lot of time randomly picking ports that are
> already taken.
> 
> Better yet would be to change the port-assigning dance by having the thing
> that ends up listening do the bind and then provide it over a named pipe or
> writing to a file or stdout or something.  That would avoid port allocation
> races, etc.

Thanks for this Andrew. I filed bug 1236855 about this.
This is causing lots of the following error:
[taskcluster:error] Task has been aborted prematurely. Reason: worker-shutdown
https://treeherder.mozilla.org/logviewer.html#?job_id=732216&repo=gaia-master

Hard to tell what's happening, but it's almost certainly caused by my patch here, so I'm going to back this out for now.
Reverted in master: https://github.com/mozilla-b2g/gaia/commit/7c74f6a655c44af47be9e328e65c47a9740ac19c
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Created attachment 8704167 [details] [review]
[gaia] mikehenrty:bug-1236376-use-more-reland > mozilla-b2g:master
feature-b2g: --- → 2.6+

Comment 23

3 years ago
21 automation job failures were associated with this bug yesterday.

Repository breakdown:
* mozilla-inbound: 15
* fx-team: 3
* b2g-inbound: 3

Platform breakdown:
* mulet-linux64: 13
* b2g-linux64: 8

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1236376&startday=2016-01-05&endday=2016-01-05&tree=all
Duplicate of this bug: 1236253
Comment on attachment 8704167 [details] [review]
[gaia] mikehenrty:bug-1236376-use-more-reland > mozilla-b2g:master

Let's just use querySelector instead in both places to be safe. Guillaume, wanna take a look?
Attachment #8704167 - Flags: review?(gmarty)
Comment on attachment 8704167 [details] [review]
[gaia] mikehenrty:bug-1236376-use-more-reland > mozilla-b2g:master

Looks perfect to me!
Attachment #8704167 - Flags: review?(gmarty) → review+
Gaia master: https://github.com/mozilla-b2g/gaia/commit/a59c28fabdff02a576d879f332a5c169fc12e6df

Ok, this test should no longer be a problem here.
(In reply to Michael Henretty [:mhenretty] from comment #27)
> Ok, this test should no longer be a problem here.

Sorry, that comment was meant for bug 1236329. I am not as confident of this being the final fix, but it should at least move us in the right direction.

Comment 29

3 years ago
20 automation job failures were associated with this bug yesterday.

Repository breakdown:
* mozilla-inbound: 11
* fx-team: 4
* mozilla-central: 3
* b2g-inbound: 2

Platform breakdown:
* mulet-linux64: 20

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1236376&startday=2016-01-06&endday=2016-01-06&tree=all
Whiteboard: [systemsfe][MJS] → [systemsfe][MJS] [CI]

Comment 30

3 years ago
76 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-inbound: 50
* fx-team: 15
* b2g-inbound: 8
* mozilla-central: 3

Platform breakdown:
* mulet-linux64: 52
* b2g-linux64: 24

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1236376&startday=2016-01-04&endday=2016-01-10&tree=all
Target Milestone: --- → 2.6 S5 - 1/15
Look at the brasstacks link in comment 30, this hasn't happened since Jan 7, which would be enough time for my changes to propagate to all trees. Marking as fixed.
Status: REOPENED → RESOLVED
Last Resolved: 3 years ago3 years ago
Resolution: --- → FIXED
Duplicate of this bug: 1236404

Updated

5 months ago
Product: Testing → Testing Graveyard
You need to log in before you can comment on or make changes to this bug.