vcssync1 & vcssync2 will be rebooted for Amazon maintenance

RESOLVED INVALID

Status

Release Engineering
Buildduty
RESOLVED INVALID
3 years ago
3 years ago

People

(Reporter: nthomas, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

3 years ago
Email from AWS:

"One or more of your Amazon EC2 instances are scheduled to be rebooted for required host maintenance in order to deploy important updates.  These updates must be completed by 3/10 before the underlying issues we are addressing are made public as part of an upcoming Xen Security Advisory (XSA)."

The console shows we only have two machines affected:
vcssync1.srv.releng.usw2.mozilla.com
vcssync2.srv.releng.usw2.mozilla.com
Assignee: nobody → hwine
[hwine@vcssync1.srv.releng.usw2.mozilla.com ~]$ last reboot
reboot   system boot  2.6.32-220.el6.x Sun Mar  1 14:52 - 14:59  (00:06)    

[hwine@vcssync2.srv.releng.usw2.mozilla.com ~]$ last reboot
reboot   system boot  2.6.32-220.el6.x Sun Mar  1 15:03 - 15:08  (00:05)    

All done!
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
(Reporter)

Comment 2

3 years ago
Sorry, I failed to see this in the email:
 You will not be able to stop/start or re-launch instances in order to avoid this maintenance update.

The windows are:
vcssync1: March 8 0700-1000 UTC-7   (Converted from March 9, 0300-0600 UTC+13) 
vcssync2: March 7 0600-0900 UTC-8   (Concerted from March 8, 0300-0600 UTC+13)
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Summary: Reboot vcssync1 & vcssync2 for Amazon maintenance → vcssync1 & vcssync2 will be rebooted for Amazon maintenance
These boxes should come back up cleanly.

I did add the following to both machine's vcs2vcs user's crontab:
  @reboot /bin/mail -s "$HOSTNAME rebooted <eom>" $MAILTO </dev/null
so we'll be notified afterwards. MAILTO=release+vcs2vcs@mozilla.com on these crontabs already.
Assignee: hwine → nobody
(Reporter)

Comment 4

3 years ago
Amazon have sent further email. They've managed to reduce the number of instances that need rebooting from less than 10% to 0.1%, and we don't have any in that smaller group.
Status: REOPENED → RESOLVED
Last Resolved: 3 years ago3 years ago
Resolution: --- → INVALID
You need to log in before you can comment on or make changes to this bug.