1309812 - Windows 7 VM reftest jobs are not running by default if 'Windows 7' is not explicitly specified

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Alin Selagea [:aselagea]]

The following Windows 7 tests are running on both HW and VM at the moment:
    - reftest-no-accel
    - reftest
    - reftest-e10s

However, they are not running on VMs if 'Windows 7' is not explicitly specified in the try syntax (as they do on hardware). We should fix that before we stop running these tests on the hardware machines.

Comment 1

[Alin Selagea [:aselagea]]

I've spent some time investigating this, but had little luck figuring out what's wrong.
@Chris: when you have some time, can you please take a look and give us some suggestions/hints?

Thanks!

Comment 2

[Chris AtLee [:catlee]]

They're not running by default because 'try_by_default' is set to False in mozilla-tests/config.py for 'win7_vm_gfx'.

The last time we tried flipping that setting (bug 1302256) we ran into problems getting enough instances to run tests (bug 1302530). I'm not sure if that was coincidence or not.

Comment 4

[Sebastian Pacurar [:spacurar]]

I had a look and noticed that in https://bugzilla.mozilla.org/show_bug.cgi?id=1303006#c12 Joel mentioned that we should have reftest jobs running by default on try, without specifying "Windows 7" in the try syntax.

That being said, should we set "try_by_default" to True in this case?

Comment 5

[Chris AtLee [:catlee]]

We can give it a try, but we should keep a close eye on capacity and problems getting new instances from Amazon.

Comment 6

[Sebastian Pacurar [:spacurar]]

Attachment: bug_1309812.patch

I switched the value to True. After the patch is landed I still need to investigate if there will be any problems getting new instances from Amazon.

Comment 7

[Alin Selagea [:aselagea]]

Comment on attachment 8850475 [details] [diff] [review]
bug_1309812.patch

https://hg.mozilla.org/build/buildbot-configs/rev/3eada6feed10
https://hg.mozilla.org/build/buildbot-configs/rev/f704e74c48df

Comment 8

[Sebastian Pacurar [:spacurar]]

As far as I noticed on AWS console for the past days ,it seems there were no problems getting new instances from Amazon. 
I also noticed on treeherder that reftest-no-accel, reftest and reftest-e10s are now run on Windows VM without specifying "windows 7" in the try syntax.