bugzilla.mozilla.org will be intermittently unavailable on Saturday, March 24th, from 16:00 until 20:00 UTC.

usw2 puppetmaster marked as scheduled for retirement



Infrastructure & Operations
9 months ago
9 months ago


(Reporter: dividehex, Unassigned)





9 months ago
releng-puppet1.srv.releng.usw2 has been marked for death.  Aki restarted it which brought it back online but I believe we still need to deal with this.  On the other hand, the restart may have migrated it to new underlying hardware and this would be a non issue.  This might be the case since I don't see the warning flag on the instance in the console.

On Fri, Jun 23, 2017 at 4:57 PM, 'Amazon Web Services, Inc.' via Release <release@mozilla.com> wrote:

    Dear Amazon EC2 Customer,

    We have important news about your account (AWS Account ID: 314336048151). EC2 has detected degradation of the underlying hardware hosting your Amazon EC2 instance (instance-ID: i-42c0274d) in the us-west-2 region. Due to this degradation, your instance could already be unreachable. After 2017-07-07 23:00 UTC your instance, which has an EBS volume as the root device, will be stopped.

    You can see more information on your instances that are scheduled for retirement in the AWS Management Console (https://console.aws.amazon.com/ec2/v2/home?region=us-west-2#Events)

    * How does this affect you?
    Your instance's root device is an EBS volume and the instance will be stopped after the specified retirement date. You can start it again at any time. Note that if you have EC2 instance store volumes attached to the instance, any data on these volumes will be lost when the instance is stopped or terminated as these volumes are physically attached to the host computer

    * What do you need to do?
    You may still be able to access the instance. We recommend that you replace the instance by creating an AMI of your instance and launch a new instance from the AMI. For more information please see Amazon Machine Images (http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html) in the EC2 User Guide. In case of difficulties stopping your EBS-backed instance, please see the Instance FAQ (http://aws.amazon.com/instance-help/#ebs-stuck-stopping).

    * Why retirement?
    AWS may schedule instances for retirement in cases where there is an unrecoverable issue with the underlying hardware. For more information about scheduled retirement events please see the EC2 user guide (http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-retirement.html). To avoid single points of failure within critical applications, please refer to our architecture center for more information on implementing fault-tolerant architectures: http://aws.amazon.com/architecture

    If you have any questions or concerns, you can contact the AWS Support Team on the community forums and via AWS Premium Support at: http://aws.amazon.com/support

    Amazon Web Services

    This message was produced and distributed by Amazon Web Services, Inc., 410 Terry Avenue North, Seattle, Washington 98109-5210

    Reference: AWS_EC2_PERSISTENT_INSTANCE_RETIREMENT_SCHEDULED0d5c6eab-ec31-492f-9416-15b634684db5

Comment 1

9 months ago
I've confirmed restarting the instance moved the it to a new underlying. \o/
Last Resolved: 9 months ago
Resolution: --- → FIXED

Comment 2

9 months ago
You need to log in before you can comment on or make changes to this bug.