If you think a bug might affect users in the 57 release, please set the correct tracking and status flags for Release Management.

Autophone - autophone-3 is down 2017-02-20

RESOLVED FIXED

Status

Testing
Autophone
P1
blocker
RESOLVED FIXED
7 months ago
7 months ago

People

(Reporter: bc, Assigned: van)

Tracking

(Blocks: 1 bug)

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

7 months ago
autophone-3 experienced a number of disconnections and reboots over the weekend and now is barely responding. It appears to be suffering an infinite number of

[122199.160056] hub 11-2.1.4.4:1.0: hub_ext_port_status failed (err = -22)

errors. I am unable to use systemctl to start the autophone service and am even unable to reboot the host due to a timeout failure.

root@autophone-3:/var/log# reboot
Failed to start reboot.target: Connection timed out
See system logs and 'systemctl status reboot.target' for details.

root@autophone-3:/var/log# systemctl status reboot.target
Failed to get properties: Connection timed out

I can not reboot the host nor start the autophone service.

I believe the new usb3 vantek hubs which were attached to the servers last week have resulted in the servers becoming unstable and they should be removed with the devices distributed across the 12 available ports on the host.

Port Assignment Inventory:
https://docs.google.com/spreadsheets/d/1ppMtFUS6by42QCv_b_6EPQko30TaswAxrmFLb-FxHcI/edit?ts=58a61d0d#gid=0
(Reporter)

Comment 1

7 months ago
Van took care of this. We had to move some devices around.
https://github.com/mozilla/autophone/commit/d444cd7289a99b1fa9e8f0721be7c8fef67060e6
Assignee: nobody → vle
Status: NEW → RESOLVED
Last Resolved: 7 months ago
Resolution: --- → FIXED
(Reporter)

Updated

7 months ago
Depends on: 1341715
You need to log in before you can comment on or make changes to this bug.