Closed Bug 1695871 Opened 3 years ago Closed 3 years ago

ytw perftest failing frequently or permanently

Categories

(Testing :: mozperftest, defect, P1)

Product:
Testing
Component:
mozperftest
Version:
Firefox 88
Type:
defect

Tracking

(firefox88 fixed)

Status:
RESOLVED FIXED
Milestone:
88 Branch
Tracking Flags:
Tracking Status
firefox88 --- fixed

People

(Reporter: aryx, Assigned: sparky)

Details

Attachments

(1 file)

Bug 1695871 - Disable youtube-watch perftests. r?#perftest
48 bytes, text/x-phabricator-request
Details | Review

The last run of the ytw perftests failed all of them. macOS and Windows changed their identifier but there also already failing frequently before. Some tasks have failed their last 5 runs.

Log: https://treeherder.mozilla.org/logviewer?job_id=331696622&repo=mozilla-central

[task 2021-03-02T06:20:5[task 2021-03-02T06:20:51.765Z] [2021-03-02 06:20:51] INFO: Navigating to url https://www.youtube.com/watch?v=COU5T-Wafa4 iteration 5
[task 2021-03-02T06:20:58.291Z] [2021-03-02 06:20:58] ERROR: Couldn't execute script named CUSTOM error:JavascriptError: TypeError: can't access property "click", document.getElementById(...) is null
[task 2021-03-02T06:20:58.292Z] [2021-03-02 06:20:58] ERROR: Could not find element by id toggleButton
[task 2021-03-02T06:20:58.292Z] [2021-03-02 06:20:58] ERROR: Error: Could not find element by id toggleButton
[task 2021-03-02T06:20:58.292Z] at Click.byIdAndWait (/home/cltbld/.mozbuild/browsertime/node_modules/browsertime/lib/core/engine/command/click.js:219:13)
[task 2021-03-02T06:20:58.292Z] at process._tickCallback (internal/process/next_tick.js:68:7)

...

1.765Z] [2021-03-02 06:20:51] INFO: Navigating to url https://www.youtube.com/watch?v=COU5T-Wafa4 iteration 5
[task 2021-03-02T06:20:58.291Z] [2021-03-02 06:20:58] ERROR: Couldn't execute script named CUSTOM error:JavascriptError: TypeError: can't access property "click", document.getElementById(...) is null
[task 2021-03-02T06:20:58.292Z] [2021-03-02 06:20:58] ERROR: Could not find element by id toggleButton
[task 2021-03-02T06:20:58.292Z] [2021-03-02 06:20:58] ERROR: Error: Could not find element by id toggleButton
[task 2021-03-02T06:20:58.292Z] at Click.byIdAndWait (/home/cltbld/.mozbuild/browsertime/node_modules/browsertime/lib/core/engine/command/click.js:219:13)
[task 2021-03-02T06:20:58.292Z] at process._tickCallback (internal/process/next_tick.js:68:7)
[task 2021-03-02T06:20:58.672Z] [2021-03-02 06:20:58] ERROR: No data to collect

...

[task 2021-03-02T06:25:55.475Z] [2021-03-02 06:25:55] INFO: playing...
[task 2021-03-02T06:25:55.502Z] [2021-03-02 06:25:55] INFO: Stop firefox window recorder.
[task 2021-03-02T06:26:23.944Z] [2021-03-02 06:26:23] INFO: https://www.youtube.com/watch?v=COU5T-Wafa4 TTFB: 62ms DOMContentLoaded: 711ms FCP: 316ms Load: 712ms
[task 2021-03-02T06:26:23.948Z] [2021-03-02 06:26:23] INFO: https://www.youtube.com/watch?v=COU5T-Wafa4 TTFB: 70ms (±2.35ms), FCP: 333ms (±4.79ms), DOMContentLoaded: 789ms (±26.34ms), Load: 799ms (±30.81ms) (7 runs)
[task 2021-03-02T06:26:23.950Z] [2021-03-02 06:26:23] INFO: Wrote data to ../artifacts/browsertime-results-1
[task 2021-03-02T06:26:23.957Z] Traceback (most recent call last):
[task 2021-03-02T06:26:23.957Z] File "python/mozperftest/mozperftest/runner.py", line 220, in <module>
[task 2021-03-02T06:26:23.957Z] sys.exit(main())
[task 2021-03-02T06:26:23.957Z] File "python/mozperftest/mozperftest/runner.py", line 216, in main
[task 2021-03-02T06:26:23.957Z] run_tests(mach_cmd, args, user_args)
[task 2021-03-02T06:26:23.957Z] File "python/mozperftest/mozperftest/runner.py", line 163, in run_tests
[task 2021-03-02T06:26:23.957Z] e.run(metadata)
[task 2021-03-02T06:26:23.957Z] File "/home/cltbld/tasks/task_1614665853/fetches/python/mozperftest/mozperftest/environment.py", line 93, in run
[task 2021-03-02T06:26:23.957Z] metadata = testlayer(syslayer(metadata))
[task 2021-03-02T06:26:23.957Z] File "/home/cltbld/tasks/task_1614665853/fetches/python/mozperftest/mozperftest/layers.py", line 156, in call
[task 2021-03-02T06:26:23.957Z] metadata = layer(metadata)
[task 2021-03-02T06:26:23.957Z] File "/home/cltbld/tasks/task_1614665853/fetches/python/mozperftest/mozperftest/layers.py", line 68, in call
[task 2021-03-02T06:26:23.957Z] metadata = self.run(metadata)
[task 2021-03-02T06:26:23.957Z] File "/home/cltbld/tasks/task_1614665853/fetches/python/mozperftest/mozperftest/test/browsertime/runner.py", line 342, in run
[task 2021-03-02T06:26:23.957Z] metadata = self._one_cycle(metadata, result_dir)
[task 2021-03-02T06:26:23.957Z] File "/home/cltbld/tasks/task_1614665853/fetches/python/mozperftest/mozperftest/test/browsertime/runner.py", line 413, in _one_cycle
[task 2021-03-02T06:26:23.957Z] raise NodeException(exit_code)
[task 2021-03-02T06:26:23.957Z] mozperftest.test.browsertime.runner.NodeException: 1

Severity: -- → S2
Priority: -- → P1
Assignee: nobody → gmierz2
Status: NEW → ASSIGNED
Pushed by gmierz2@outlook.com:
https://hg.mozilla.org/integration/autoland/rev/6ebde97dbd87
Disable youtube-watch perftests. r=perftest-reviewers,Bebe

Yes, that was wrongfully backed out. Sorry about that. Relanding

Flags: needinfo?(atila.butkovits)
Pushed by abutkovits@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/9d4cb88c5ddb
Disable youtube-watch perftests. r=perftest-reviewers,Bebe. CLOSED TREE

https://hg.mozilla.org/mozilla-central/rev/9d4cb88c5ddb

Status: ASSIGNED → RESOLVED
Closed: 3 years ago
status-firefox88: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → 88 Branch
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: