1830299 - Permanent grandprix tier-3 raptor-browsertime Critical: Couldn't execute script named CUSTOM error:JavascriptError: TypeError: can't access property "click", document.evaluate(...).singleNodeValue is null

Status: RESOLVED FIXED

(Testing :: Raptor, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: archaeopteryx [at] coole-files.de
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=413953625&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/X5s5DS6gQX20wwNNYxrNPQ/runs/0/artifacts/public/logs/live_backing.log

---

This is a new permanent failure for the "grandprix" performance test. The task is not shown by default because it is set as tier 3. To see it, click the "3" in the top right in Treeherder.

The failure is unrelated to check-ins on our side because it also fails on older pushes. The first failed task started around 17:10UTC on March 24, the last unaffected run was at 14:30.

task 2023-04-27T11:36:10.526Z] 11:36:10     INFO -  raptor-browsertime Info: Using url=https://grandprixbench.netlify.app/?suites=Vanilla-HTML-TodoMVC,Vanilla-TodoMVC,Preact-TodoMVC,Preact-TodoMVC-Modern,React-TodoMVC,Monaco-Editor,Monaco-Syntax-Highlight,React-Stockcharts,React-Stockcharts-SVG,Leaflet-Fractal,Proxx-Tables,Proxx-Tables-Lit,Proxx-Tables-Canvas,&iterations=25
[task 2023-04-27T11:36:10.527Z] 11:36:10     INFO -  raptor-browsertime Info: Testing url https://grandprixbench.netlify.app/?suites=Vanilla-HTML-TodoMVC,Vanilla-TodoMVC,Preact-TodoMVC,Preact-TodoMVC-Modern,React-TodoMVC,Monaco-Editor,Monaco-Syntax-Highlight,React-Stockcharts,React-Stockcharts-SVG,Leaflet-Fractal,Proxx-Tables,Proxx-Tables-Lit,Proxx-Tables-Canvas,&iterations=25 iteration 1
[task 2023-04-27T11:36:16.656Z] 11:36:16     INFO -  raptor-browsertime Info: CPU / Power usage: 669
[task 2023-04-27T11:36:19.878Z] 11:36:19    ERROR -  raptor-browsertime Error: Browsertime failed to run
[task 2023-04-27T11:36:19.899Z] 11:36:19    ERROR -  raptor-browsertime Critical: Couldn't execute script named CUSTOM error:JavascriptError: TypeError: can't access property "click", document.evaluate(...).singleNodeValue is null
[task 2023-04-27T11:36:19.899Z] 11:36:19     INFO -  raptor-perftest Info: Removing temporary directory: /tmp/tmp9yw8do90
[task 2023-04-27T11:36:19.908Z] 11:36:19    ERROR -  Traceback (most recent call last):
[task 2023-04-27T11:36:19.908Z] 11:36:19     INFO -    File "/home/cltbld/tasks/task_168259523303384/build/tests/raptor/raptor/raptor.py", line 204, in <module>
[task 2023-04-27T11:36:19.908Z] 11:36:19     INFO -      main()
[task 2023-04-27T11:36:19.909Z] 11:36:19     INFO -    File "/home/cltbld/tasks/task_168259523303384/build/tests/raptor/raptor/raptor.py", line 150, in main
[task 2023-04-27T11:36:19.909Z] 11:36:19     INFO -      success = raptor.run_tests(raptor_test_list, raptor_test_names)
[task 2023-04-27T11:36:19.909Z] 11:36:19     INFO -    File "/home/cltbld/tasks/task_168259523303384/build/tests/raptor/raptor/perftest.py", line 472, in run_tests
[task 2023-04-27T11:36:19.909Z] 11:36:19     INFO -      self.run_test(test, timeout=int(test.get("page_timeout")))
[task 2023-04-27T11:36:19.910Z] 11:36:19     INFO -    File "/home/cltbld/tasks/task_168259523303384/build/tests/raptor/raptor/browsertime/base.py", line 819, in run_test
[task 2023-04-27T11:36:19.910Z] 11:36:19     INFO -      raise Exception(self.browsertime_failure)
[task 2023-04-27T11:36:19.910Z] 11:36:19    ERROR -  Exception: Couldn't execute script named CUSTOM error:JavascriptError: TypeError: can't access property "click", document.evaluate(...).singleNodeValue is null
[task 2023-04-27T11:36:19.991Z] 11:36:19     INFO - Return code: 1

Comment 1

[Greg Mierzwinski [:sparky]]

Thank you for notifying us :aryx!

:denispal, do you know if something might have changed in the grandprix test that could have caused this? Alternatively, would you know if we could disable this test now?

Comment 2

[Denis Palmeiro [:denispal]]

I think we can safely disable this test now. I think all of the tests we care about are now in Speedometer 3, so there's no reason to monitor the grandprix tests anymore. Thanks!

Comment 3

[Greg Mierzwinski [:sparky]]

Attachment: Bug 1830299 - Remove grandprix test from CI. r?#perftest

Comment 4

[Intermittent Failures Robot]

1 failures in 3177 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• linux1804-64-shippable-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1830299&startday=2023-04-24&endday=2023-04-30&tree=all

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7bdfb97d402e
Remove grandprix test from CI. r=perftest-reviewers,kshampur

Comment 6

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/7bdfb97d402e