1657359 - Add unity-webgl benchmark to browsertime

Status: RESOLVED FIXED

(Testing :: Raptor, task, P1)

Description

[Alexandru Ionescu (needinfo me) [:alexandrui]]

This bug is for adding the rest of the benchmark tests from to browsertime on mobile. youtube-playback and speedometer tests were already added, and the remaining ones is unity-webgl.

Comment 1

[Dave Hunt [:davehunt] [he/him] ⌚BST]

See bug 1506865, which talks about converting this test into a pageload test. We should see if this still makes sense, and perhaps we can take care of both bugs in one go?

Comment 2

[Alexandru Ionescu (needinfo me) [:alexandrui]]

Dave, this changes the estimates, as webgl needs further investigation than to migrate a functional test from raptor to browsertime. Because this is the last benchmark, I will do the investigation on behalf of this migration task. If I will be able, I will also convert into a pageload test and migrate to browsetime, otherwise I will assign the estimate in order to do the task in future sprints.

Comment 3

[Dave Hunt [:davehunt] [he/him] ⌚BST]

(In reply to Alexandru Ionescu (needinfo me) [:alexandrui] from comment #2)

Dave, this changes the estimates, as webgl needs further investigation than to migrate a functional test from raptor to browsertime. Because this is the last benchmark, I will do the investigation on behalf of this migration task. If I will be able, I will also convert into a pageload test and migrate to browsetime, otherwise I will assign the estimate in order to do the task in future sprints.

Thanks Alex, this makes sense to me.

Comment 4

[Alexandru Ionescu (needinfo me) [:alexandrui]]

Attachment: Bug 1657359 Add unity-webgl benchmark to browsertime mobile r?#perftest,sparky

Comment 5

[Greg Mierzwinski [:sparky]]

:davehunt, the existing unity-webgl test is only on geckoview and refbrow. Should we expand this to include Fenix?

Comment 6

[Dave Hunt [:davehunt] [he/him] ⌚BST]

(In reply to Greg Mierzwinski [:sparky] from comment #5)

:davehunt, the existing unity-webgl test is only on geckoview and refbrow. Should we expand this to include Fenix?

Yes

Comment 7

[Alexandru Ionescu (needinfo me) [:alexandrui]]

This might extend my task to september sprint. I assume we need some changes to the fenix repo too, is that right? Sparky, should we do this into a different task?

Comment 8

[Greg Mierzwinski [:sparky]]

No need to add it to the Fenix branch for now, we can do that later. As long as we have it running in mc daily we'll be good.

Comment 9

[Alexandru Ionescu (needinfo me) [:alexandrui]]

Sparky, the issue in this comment is most likely taskcluster related, and might take time for me to debug it, which will become a blocker for this task. As not keeping the page-cycles in the default section is not wrong, just not recomended, I would land the patch as it is and fix the taskcluster issue in a separate bug.

Comment 10

[Greg Mierzwinski [:sparky]]

That's not a taskcluster related failure, it's a raptor related failure and it's failing during manifest parsing.

[task 2020-09-02T09:56:19.854Z] 09:56:04     INFO -  raptor-manifest Info: /builds/task_159904045834244/workspace/build/tests/raptor/raptor/raptor.ini
[task 2020-09-02T09:56:19.854Z] 09:56:04    ERROR -  Traceback (most recent call last):
[task 2020-09-02T09:56:19.854Z] 09:56:04     INFO -    File "/builds/task_159904045834244/workspace/build/tests/raptor/raptor/raptor.py", line 202, in <module>
[task 2020-09-02T09:56:19.854Z] 09:56:04     INFO -      main()
[task 2020-09-02T09:56:19.854Z] 09:56:04     INFO -    File "/builds/task_159904045834244/workspace/build/tests/raptor/raptor/raptor.py", line 72, in main
[task 2020-09-02T09:56:19.854Z] 09:56:04     INFO -      raptor_test_list = get_raptor_test_list(args, mozinfo.os)
[task 2020-09-02T09:56:19.854Z] 09:56:04     INFO -    File "/builds/task_159904045834244/workspace/build/tests/raptor/raptor/manifest.py", line 327, in get_raptor_test_list
[task 2020-09-02T09:56:19.854Z] 09:56:04     INFO -      available_tests = get_browser_test_list(args.app, args.run_local)
[task 2020-09-02T09:56:19.854Z] 09:56:04     INFO -    File "/builds/task_159904045834244/workspace/build/tests/raptor/raptor/manifest.py", line 92, in get_browser_test_list
[task 2020-09-02T09:56:19.854Z] 09:56:04     INFO -      test_manifest = TestManifest([raptor_ini], strict=False)
[task 2020-09-02T09:56:19.854Z] 09:56:04     INFO -    File "/builds/task_159904045834244/workspace/build/venv/lib/python2.7/site-packages/manifestparser/manifestparser.py", line 770, in __init__
[task 2020-09-02T09:56:19.854Z] 09:56:04     INFO -      ManifestParser.__init__(self, *args, **kwargs)
[task 2020-09-02T09:56:19.854Z] 09:56:04     INFO -    File "/builds/task_159904045834244/workspace/build/venv/lib/python2.7/site-packages/manifestparser/manifestparser.py", line 88, in __init__
[task 2020-09-02T09:56:19.854Z] 09:56:04     INFO -      self.read(*manifests)
[task 2020-09-02T09:56:19.854Z] 09:56:04     INFO -    File "/builds/task_159904045834244/workspace/build/venv/lib/python2.7/site-packages/manifestparser/manifestparser.py", line 270, in read
[task 2020-09-02T09:56:19.854Z] 09:56:04     INFO -      self._read(here, filename, defaults)
[task 2020-09-02T09:56:19.854Z] 09:56:04     INFO -    File "/builds/task_159904045834244/workspace/build/venv/lib/python2.7/site-packages/manifestparser/manifestparser.py", line 197, in _read
[task 2020-09-02T09:56:19.854Z] 09:56:04     INFO -      self._read(root, include_file, include_defaults, parentmanifest=filename_rel)
[task 2020-09-02T09:56:19.854Z] 09:56:04     INFO -    File "/builds/task_159904045834244/workspace/build/venv/lib/python2.7/site-packages/manifestparser/manifestparser.py", line 172, in _read
[task 2020-09-02T09:56:19.854Z] 09:56:04     INFO -      handle_defaults=self._handle_defaults)
[task 2020-09-02T09:56:19.854Z] 09:56:04     INFO -    File "/builds/task_159904045834244/workspace/build/venv/lib/python2.7/site-packages/manifestparser/ini.py", line 129, in read_ini
[task 2020-09-02T09:56:19.854Z] 09:56:04     INFO -      assert key not in current_section
[task 2020-09-02T09:56:19.854Z] 09:56:04     INFO -  AssertionError
[task 2020-09-02T09:56:19.854Z] 09:56:04    ERROR - Return code: 1

It's not that difficult to fix. But if you can't, then file a follow-up bug for it and we can land this one.

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a451ebba378e
Add unity-webgl benchmark to browsertime mobile r=sparky,perftest-reviewers

Comment 12

[Alexandru Michis [:malexandru]]

https://hg.mozilla.org/mozilla-central/rev/a451ebba378e

Comment 13

[Alexandru Ionescu (needinfo me) [:alexandrui]]

Attachment: Bug 1657359 Fix page_cycles duplication r?#perftest,sparky

Depends on D87861

Comment 14

[Alexandru Ionescu (needinfo me) [:alexandrui]]

Attachment: Bug 1657359 Fix page_cycles duplication r?#perftest,sparky

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4fe124e5111a
Fix page_cycles duplication r=perftest-reviewers,Bebe

Comment 16

[Alexandru Michis [:malexandru]]

https://hg.mozilla.org/mozilla-central/rev/4fe124e5111a