1178644 - update compare.py to have accurate osx platforms/tests

Status: RESOLVED FIXED

(Testing :: Talos, defect)

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: compare.patch

remove osx 10.6, 10.8 and add 10.10

Comment 2

[Julien Pagès (:parkouss)]

Comment on attachment 8627529 [details] [diff] [review]
compare.patch

Looks good to me, just a couple of questions:

> diff --git a/talos/compare.py b/talos/compare.py
> --- a/talos/compare.py
> +++ b/talos/compare.py

> -platform_map['OSX64'] = 21  # 10.6
> -platform_map['OSX64 (e10s)'] = 51
>  platform_map['OSX10.8'] = 24  # TODO: 10.8 will be gone in the near future

We don't remove OSX10.8 here

> -platforms = ['Linux', 'Linux64', 'Win7', 'WinXP', 'Win8', 'OSX64', 'OSX10.8',
> -             'Android']
> +platforms = ['Linux', 'Linux64', 'Win7', 'WinXP', 'Win8', 'OSX10.10', 'Android']
>  platforms_e10s = ['Linux (e10s)', 'Linux64 (e10s)', 'Win7 (e10s)',
> -                  'WinXP (e10s)', 'Win8 (e10s)', 'OSX64 (e10s)',
> -                  'OSX10.8 (e10s)']
> +                  'WinXP (e10s)', 'Win8 (e10s)', 'OSX10.10 (e10s)']

But we do that here

> -        if osversion.startswith('OS X 10.6'):
> -            platform = 'OSX64'
> -        elif osversion.startswith('OS X 10.8'):
> +        if osversion.startswith('OS X 10.8'):
>              platform = 'OSX10.8'
> +        elif osversion.startswith('OS X 10.10'):
> +            platform = 'OSX10.10'
>      return platform

But again here OS X 10.8 is handled

> disabled_tests['OSX10.8'] = ['media_tests', 'tp5n']
> disabled_tests['OSX10.8 (e10s)'] = ['media_tests', 'tp5n']

And those are not removed.

This looks strange to me (the name of your attachment seems to imply that osx 10.8 should be removed).
At least it seems not coherent to me to remove OSX10.8 from the platforms variables only (it may be normal, though!)

Also I'm not sure why we remove remote-trobopan and remote-troboprovider from android tests within this patch.


Well it is mainly questions, I can not really do a thorough review for that patch since I'm pretty new to Talos yet. :)
The good thing is that I will learn more during this process!

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

https://hg.mozilla.org/build/talos/rev/104a8a803b1a

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

fixed the 10.8 stuff;  removed the android stuff since we don't run those tests anymore- thought I would save a review cycle.