The default bug view has changed. See this FAQ.

c3.xlarge instance types are expensive, let's test running those tests on a range of instance types that are cheaper

RESOLVED FIXED

Status

Release Engineering
Platform Support
RESOLVED FIXED
3 years ago
3 years ago

People

(Reporter: kmoir, Assigned: kmoir)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments, 1 obsolete attachment)

(Assignee)

Description

3 years ago
taras	kmoir: can you also set your new stuff
taras	to use a range of c3.xl, r3.xl and m3.xl?
	-->|	marcia (chatzilla@moz-E88C0D73.lightspeed.sntcca.sbcglobal.net) has joined #releng
	taras	just like we do for compiles?
	taras	or just set them to r3.xl for now
	taras	those are slightly cheape	
	catlee	yeah, I think http://hg.mozilla.org/build/cloud-tools/file/c10ba9d1f8b0/configs/watch_pending.cfg#l44 just needs adjusting
	catlee	kmoir: ^^ if you want to give that a shot
	kmoir	Right but have someone tested that the they all run on those instance types too?
	kmoir	without failure?
	taras	kmoir: i think that'll be you
	taras	r3.xl is nearly identical to c3.xl	
	kmoir	k. Well I'll open a bug and look at when I'm waiting for tests to complete on the other bug I'm working on for single locale builds for Android submitting to balrog
	catlee	it's a pretty easy config tweak
	kmoir	I know
	catlee	maybe try it on monday and see what happens
	kmoir	it's just the testing that takes time
	kmoir	would test on a branch first
	catlee	https://cheezburger.com/6615104768
(Assignee)

Updated

3 years ago
Assignee: nobody → kmoir
(Assignee)

Comment 1

3 years ago
Created attachment 8470091 [details] [diff] [review]
bug1047467.patch

Not sure how to test this on a branch without and not deploy to all.  I guess since it works for the builders it should work for the tests?
Attachment #8470091 - Flags: review?(catlee)
Comment on attachment 8470091 [details] [diff] [review]
bug1047467.patch

Review of attachment 8470091 [details] [diff] [review]:
-----------------------------------------------------------------

I think it's worth it to try and just keep a close eye on things. We can revert if necessary, and kill off the nodes if they're bad.

Please clean up the trailing whitespace.
Attachment #8470091 - Flags: review?(catlee) → review+
(Assignee)

Updated

3 years ago
Attachment #8470091 - Flags: checked-in+
(Assignee)

Comment 3

3 years ago
Comment on attachment 8470091 [details] [diff] [review]
bug1047467.patch

http://hg.mozilla.org/build/cloud-tools/rev/9dcb80cffe6c
(Assignee)

Comment 4

3 years ago
I haven't seen any issues with this on Tbpl since this was enable three hours ago and I can see in the aws console that the different instance types are being activated.
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
(Assignee)

Comment 5

3 years ago
coop|buildduty	should i be worried about the “400 Bad Request” errors in the aws_watch_pending.py output?
	nthomas	is there a lot of them ?
	nthomas	1 or a few is not unusual
	nthomas	haven't seen |Virtualization type 'hvm' is required for instances of type 'r3.xlarge'| before
	coop|buildduty	that’s the one
	coop|buildduty	repeated many, many times
	nthomas	http://hg.mozilla.org/build/cloud-tools/rev/3607eb84d963 seems unrelated
	nthomas	we can page rail or figure out what instances types to disable
	nthomas	I'm assuming AWS made a change which makes the API more restrictive
	nthomas	first message is Date: Mon, 11 Aug 2014 06:46:56 -0700
	nthomas	wonders why this hasn't been a big issue for launching build instances
	coop|buildduty	http://hg.mozilla.org/build/cloud-tools/rev/9dcb80cffe6c ?
	nthomas	bingo
	kmoir	yeah I landed that yesterday but it looked okay at the time
	kmoir	we do the same thing for the builders
	=-=	mtabara|lunch is now known as mtabara
	nthomas	there's probably something different about the ami though
	nthomas	I recall rail had a bunch of fun making one that supported both virtualisation types
	kmoir	so perhaps I should just remove r3.xlarge
	kmoir	yeah, I didn;t change the ami
	nthomas	https://aws.amazon.com/amazon-linux-ami/instance-type-matrix/ says m3.xlarge supports pv
	nthomas	so just removing r3.xlarge should be fine
	kmoir	okay I'll get a patch ready for review
(Assignee)

Updated

3 years ago
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
(Assignee)

Comment 6

3 years ago
Created attachment 8471897 [details] [diff] [review]
bug1047467-2.patch
(Assignee)

Comment 7

3 years ago
Created attachment 8471898 [details] [diff] [review]
bug1047467-2.patch
Attachment #8471897 - Attachment is obsolete: true
Attachment #8471898 - Flags: review?(nthomas)

Updated

3 years ago
Attachment #8471898 - Flags: review?(nthomas) → review+
(Assignee)

Updated

3 years ago
Attachment #8471898 - Flags: checked-in+
(Assignee)

Comment 8

3 years ago
not a messsage in the log anymore
(Assignee)

Comment 9

3 years ago
:rail how can I change the ami so it can support both virtualisation types?  r3.xlarge only supports HVM.  Looking at the amis in the aws console it seems you can only specify one type.
Flags: needinfo?(rail)
AMIs cannot support both virtualization types unfortunately... :(

We could add support for PV and HVM (using different AMIs), what would require:

* creating and testing HVM AMI for Ubuntu
* generate golden AMIs for both types
* add support for both AMI types in aws_watch_pending.py
Flags: needinfo?(rail)
(Assignee)

Comment 11

3 years ago
Okay thanks rail, I'll investigate in bug 1053800
Status: REOPENED → RESOLVED
Last Resolved: 3 years ago3 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.