1366355 - Expand talos quantum-pageload test (mitmproxy) to other platforms

Status: RESOLVED FIXED

(Testing :: Talos, defect)

Description

[Robert Wood [:rwood]]

Right now the priority is Windows 7/10 (Bug 1357382). Once that is up and running and stable, expand the talos quantum-pageload test to OSX and Linux. This will at minimum include:

- Creating a Python 3.6 package for those platforms and making it available for download via tooltool;
- ensuring a python3 virtualenv can be created and mitmproxy can be installed correctly on those platforms;
- adjusting all of the mitmproxy/mitmdump paths/command lines as necessary, etc.

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

I believe we want to do osx in the short term; I think using a shortcut of reusing the same recorded pageset for windows is ok assuming it works.

Comment 2

[Robert Wood [:rwood]]

Tp6 is running in production on Win & OSX, last up is linux x64, I'll use this bug for that patch. Here's a try run:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=9287c95aae9b80978d4699f284b7acc2305ffe6f

Comment 3

[Robert Wood [:rwood]]

Attachment: Bug 1366355 - Expand talos tp6 suite to linux x64;

Review commit: https://reviewboard.mozilla.org/r/187282/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/187282/

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8915994 [details]
Bug 1366355 - Expand talos tp6 suite to linux x64;

https://reviewboard.mozilla.org/r/187282/#review192294

a few nits, really close to r+

::: taskcluster/ci/test/test-sets.yml:148
(Diff revision 1)
>      - talos-other-stylo-disabled
>      - talos-svgr-stylo-disabled
>      - talos-tp5o-stylo-disabled
>      - talos-perf-reftest-stylo-disabled
>      - talos-perf-reftest-singletons-stylo-disabled
> +    - talos-tp6-stylo-disabled

what about stylo-threads

::: testing/mozharness/mozharness/mozilla/testing/talos.py:422
(Diff revision 1)
>              self.info("Skipping: mitmproxy is not required")
>              return
>  
>          # tp6 is supported in production only on win and macosx
>          os_name = self.platform_name()
> -        if 'win' not in os_name and os_name != 'macosx':
> +        if 'win' not in os_name and os_name != 'linux64' and os_name != 'macosx':

is there any other platform?  Maybe we can remove this?

::: testing/mozharness/mozharness/mozilla/testing/talos.py:467
(Diff revision 1)
>              self.mitmdump = os.path.join(mitmproxy_path, 'mitmdump')
>              if not os.path.exists(self.mitmdump):
>                  # download the mitmproxy release binary; will be overridden by the --no-download
>                  if '--no-download' not in self.config['talos_extra_options']:
> -                    self.query_mitmproxy_rel_bin('osx')
> +                    if 'osx' in self.platform_name():
> +                        _pform = 'osx'

can we use _platform instead of _pform?

Comment 5

[Robert Wood [:rwood]]

Comment on attachment 8915994 [details]
Bug 1366355 - Expand talos tp6 suite to linux x64;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/187282/diff/1-2/

Comment 6

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8915994 [details]
Bug 1366355 - Expand talos tp6 suite to linux x64;

https://reviewboard.mozilla.org/r/187282/#review193220

one small nit

::: testing/mozharness/mozharness/mozilla/testing/talos.py:492
(Diff revisions 1 - 2)
>          self.info("Downloading the mitmproxy release binary using tooltool")
>          dest = os.path.join(self.talos_path, 'talos', 'mitmproxy')
>          _manifest = "mitmproxy-rel-bin-%s.manifest" % platform
>          manifest_file = os.path.join(self.talos_path, 'talos', 'mitmproxy', _manifest)
>  
> -        if platform == 'osx':
> +        if platform == 'osx' or platform == 'linux64:':

'linux64' has a : in it.  Could you rewrite:
if platform in ['osx', 'linux64']:

Comment 7

[Robert Wood [:rwood]]

Comment on attachment 8915994 [details]
Bug 1366355 - Expand talos tp6 suite to linux x64;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/187282/diff/2-3/

Comment 8

[Robert Wood [:rwood]]

Comment on attachment 8915994 [details]
Bug 1366355 - Expand talos tp6 suite to linux x64;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/187282/diff/3-4/

Comment 9

[Robert Wood [:rwood]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=4c7e760ff1e73f68b8caad4c656d4f5478f4f541

Comment 10

[Robert Wood [:rwood]]

Update, tp6 shouldn't be added to the 'date' repo (and xperf shouldn't be there anymore either). Going to remote xperf from 'date' in this patch also while I'm at it.

From IRC:

<rwood> aki: hey, I notice on the 'date' repo, talos tp6 and xperf are running (but nothing else). Should they be removed?? Or should all other talos tests be added? Just wondering if it should be made consistent
2:04 PM 
<rwood> http://searchfox.org/mozilla-central/rev/b53e29293c9e9a2905f4849f4e3c415e2013f0cb/taskcluster/ci/test/tests.yml#1760
2:05 PM 
<aki> rwood: we're retiring the `date` repo, so we should consistently reduce the tasks running to 0 :)
2:05 PM 
<rwood> aki: ok perfect haha I'll remove those talos ones, thanks

Comment 11

[Robert Wood [:rwood]]

Comment on attachment 8915994 [details]
Bug 1366355 - Expand talos tp6 suite to linux x64;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/187282/diff/4-5/

Comment 12

[Robert Wood [:rwood]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=556f93ec9c1793d2ad464dc8d900243bd7fd3ae0

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/0ac0d5e4f556
Expand talos tp6 suite to linux x64; r=jmaher

Comment 14

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/0ac0d5e4f556