783569 - DeviceManagerSUT does not pause between connection attempts

Status: RESOLVED FIXED

(Testing :: Mozbase, defect)

Description

[Mark Côté [:mcote]]

In DeviceManagerSUT.sendCmds(), there is a retry loop that attempts to establish a connection with the agent. However, there is no sleep in this loop anywhere. This is okay if the host is unreachable, since it would automatically wait a while before retrying. If, however, the host is reachable but the port isn't open, we spin through this retry loop in less than a second.

I noticed this when playing with the smoke test. It seems that the registration message is sent before the command port is opened, so occasionally Autophone tries to connect too soon and then quickly gives up. It would be easy to work around this, but I think it should be fixed here.

Comment 1

[Mark Côté [:mcote]]

I think this is getting me again, so time to fix it.

Comment 2

[Mark Côté [:mcote]]

Attachment: Pause between reconnection attempts

This patch checks to see if we lost our connection after getting an AgentError, and, if so, pauses before attempting to reconnect. The length of the pause is increased with each attempt, up to 3 retries.

Also looks like we weren't clearing our retries for the life of the DeviceManager. Probably not a huge deal but didn't make sense to me, if it was actually able to recover from the problem 3 times.

Comment 3

[Mark Côté [:mcote]]

Try run: https://tbpl.mozilla.org/?tree=Try&rev=4f84ff8cbb8c

Comment 4

[William Lachance (:wlach)]

Comment on attachment 660921 [details] [diff] [review]
Pause between reconnection attempts

Makes sense, though looking at the source I don't see why retries is a member variable at all? I think it would be better to just make that local to sendCmds, as it only seems to be used there. r+ with that change.

Comment 5

[Mozilla RelEng Bot]

Try run for 4f84ff8cbb8c is complete.
Detailed breakdown of the results available here:
    https://tbpl.mozilla.org/?tree=Try&rev=4f84ff8cbb8c
Results (out of 19 total builds):
    success: 18
    warnings: 1
Builds (or logs if builds failed) available at:
http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/[e-mail]-4f84ff8cbb8c

Comment 6

[Mark Côté [:mcote]]

Good suggestion. I also realized that it will still sleep even after it hits its maximum number of retries (after which it will immediately raise an AgentError), so I changed the "if not self._sock" to "if retries < self.retrylimit and not self._sock".

I'm running it through try again just because I'm paranoid. :)

Comment 7

[Mozilla RelEng Bot]

Try run for 0650fabaaa00 is complete.
Detailed breakdown of the results available here:
    https://tbpl.mozilla.org/?tree=Try&rev=0650fabaaa00
Results (out of 19 total builds):
    success: 19
Builds (or logs if builds failed) available at:
http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/[e-mail]-0650fabaaa00

Comment 8

[Mark Côté [:mcote]]

https://github.com/mozilla/mozbase/commit/e2132a1bfb83682a591a707da7c47b3f796d7087