1380915 - aws-manager2 cron still tries to delete old amis for t-w10 and g-w10

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: CIDuty, task, P1)

Description

[Nick Thomas [:nthomas] (UTC+12)]

Here's (trimmed) output from delete_old_spot_amis.py via papertrail:

Jul 13 01:30:04 WARNING - Deleting EBS-backed AMI Image:ami-25261833 (spot-tst-linux64-2017-07-02-08-52)
Jul 13 01:30:05 WARNING - Deleting Snapshot:snap-0b92bf395946ea93e (spot-tst-linux64-2017-07-02-08-52)
Jul 13 01:30:06 WARNING - Deleting EBS-backed AMI Image:ami-355d6223 (spot-tst-linux32-2017-07-01-09-12)
Jul 13 01:30:06 WARNING - Deleting Snapshot:snap-0d9e1ca11e41e8b0f (spot-tst-linux32-2017-07-01-09-12)
Jul 13 01:30:08 WARNING - Deleting EBS-backed AMI Image:ami-6f291779 (spot-try-linux64-2017-07-02-08-37)
Jul 13 01:30:08 WARNING - Deleting Snapshot:snap-0d56d9476e2194f83 (spot-try-linux64-2017-07-02-08-37)
Jul 13 01:30:10 WARNING - Deleting EBS-backed AMI Image:ami-00281616 (spot-bld-linux64-2017-07-02-08-41)
Jul 13 01:30:10 WARNING - Deleting Snapshot:snap-018abcf8c6d6d0ec8 (spot-bld-linux64-2017-07-02-08-41)
Jul 13 01:30:11 WARNING - Deleting EBS-backed AMI Image:ami-031e2015 (spot-tst-emulator64-2017-07-02-09-32)
Jul 13 01:30:11 WARNING - Deleting Snapshot:snap-05f4e7efd5c1ec42e (spot-tst-emulator64-2017-07-02-09-32)
Jul 13 01:30:13 WARNING - Deleting EBS-backed AMI Image:ami-f66854e0 (spot-y-2008-2017-06-30-18-51)
Jul 13 01:30:13 WARNING - Deleting Snapshot:snap-0ace2e3f4963ed579 (spot-y-2008-2017-06-30-18-51)
Jul 13 01:30:14 WARNING - Deleting EBS-backed AMI Image:ami-a05a65b6 (spot-b-2008-2017-07-01-09-06)
Jul 13 01:30:15 WARNING - Deleting Snapshot:snap-046b99247b96d6a33 (spot-b-2008-2017-07-01-09-06)
Jul 13 01:30:16 WARNING - Deleting EBS-backed AMI Image:ami-bb1d23ad (spot-t-w732-2017-07-02-09-24)
Jul 13 01:30:16 WARNING - Deleting Snapshot:snap-029ea8c23d9666557 (spot-t-w732-2017-07-02-09-24)
Jul 13 01:30:18 WARNING - Deleting EBS-backed AMI Image:ami-e61a24f0 (spot-g-w732-2017-07-02-09-25)
Jul 13 01:30:18 WARNING - Deleting Snapshot:snap-0c31d04b37683eedb (spot-g-w732-2017-07-02-09-25)
Jul 13 01:30:18 Traceback (most recent call last):
Jul 13 01:30:18   File "delete_old_spot_amis.py", line 43, in <module>
Jul 13 01:30:18     main()
Jul 13 01:30:18   File "delete_old_spot_amis.py", line 34, in main
Jul 13 01:30:18     open("%s/%s" % (INSTANCE_CONFIGS_DIR, cfg)))[r]
Jul 13 01:30:18 IOError: [Errno 2] No such file or directory: '/builds/aws_manager/cloud-tools/cloudtools/aws/../../configs/t-w10'

Given the WONTFIX of bug 1312044 we should remove all the references to t-w10 and g-w10 in modules/aws_manager/manifests/cron.pp, and check for any other left overs.

Comment 1

[Sebastian Pacurar [:spacurar]]

Attachment: bug_1380915_puppet.patch

This is the puppet patch for modules/aws_manager/manifests/cron.pp.

I also checked for leftovers and it seems the win10 support has been removed from buildbot, cloud-tools and from manifests/moco-nodes.pp and modules/buildmaster/templates/BuildSlaves-tests.py.erb already.

Comment 2

[Mihai Tabara [:mtabara]⌚️GMT]

(In reply to Sebastian Pacurar [:spacurar] from comment #1)
> Created attachment 8892009 [details] [diff] [review]
> bug_1380915_puppet.patch
> 
> This is the puppet patch for modules/aws_manager/manifests/cron.pp.
> 
> I also checked for leftovers and it seems the win10 support has been removed
> from buildbot, cloud-tools and from manifests/moco-nodes.pp and
> modules/buildmaster/templates/BuildSlaves-tests.py.erb already.

If everything is ready to land here, feel free to ping me and I can land this for you ;-)

Comment 3

[Alin Selagea [:aselagea]]

Comment on attachment 8892009 [details] [diff] [review]
bug_1380915_puppet.patch

https://hg.mozilla.org/build/puppet/rev/62f99f05e0dd
https://hg.mozilla.org/build/puppet/rev/f12d072ae4f2

Comment 4

[Mihai Tabara [:mtabara]⌚️GMT]

Is there anything left to be done here or can we close this?

Comment 5

[Sebastian Pacurar [:spacurar]]

I just checked the logs and didn't see the error occur anymore. I guess it's done. As far as I remember this was affecting only windows 10 machines.

Comment 6

[Mihai Tabara [:mtabara]⌚️GMT]

Okay. Let's declare victory for now and reopen if things fall apart again.