1799655 - Permanent g-search mozperftest.test.browsertime.runner.NodeException: 1

Status: RESOLVED FIXED

(Testing :: Raptor, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: archaeopteryx [at] coole-files.de
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=395884913&repo=mozilla-central
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/JnHkkUFJRAyqnS7-dX7bWA/runs/0/artifacts/public/logs/live_backing.log

---

These tests run as tier 3 and are not shown in Treeherder by default.

[task 2022-11-08T05:45:46.627Z] 14:48.37 https://www.google.com/search?q=Python&source=hp&ei=welpY5zDNvOE0PEPo4W50AQ&iflsig=AJiK0e8AAAAAY2n30Yin7Vaxiz5wO-X9Mc6vuYPwjlQG&ved=0ahUKEwjcp-vY7Z37AhVzAjQIHaNCDkoQ4dUDCAo&oq=Python&gs_lcp=Cgdnd3Mtd2l6EAwyBQgAEIAEMggIABCABBCxAzILCAAQgAQQsQMQgwEyCwgAEIAEELEDEIMBMggILhCABBDUAjIICAAQgAQQsQMyCwgAEIAEELEDEIMBMgsIABCABBCxAxCDATIICAAQgAQQsQMyCAgAEIAEELEDOgsILhCABBCxAxDUAlAAWBNg2S1oAHAAeACAAV6IAYYCkgEBNJgBAKABAQ&sclient=gws-wiz TTFB: 93ms (σ136.00ms), firstPaint: 148ms (σ141.00ms), FCP: 148ms (σ140.00ms), DOMContentLoaded: 624ms (σ81.00ms), Load: 1.50s (σ471.00ms) (18 runs)
[task 2022-11-08T05:45:46.634Z] 14:48.38 Wrote data to ../artifacts/browsertime-results-1
[task 2022-11-08T05:45:46.656Z] Site not up-to-date reason: "/home/cltbld/tasks/task_166788112213839/fetches/obj-x86_64-pc-linux-gnu/_virtualenvs/common" does not exist
[task 2022-11-08T05:45:46.658Z] Traceback (most recent call last):
[task 2022-11-08T05:45:46.658Z]   File "python/mozperftest/mozperftest/runner.py", line 280, in <module>
[task 2022-11-08T05:45:46.658Z]     sys.exit(main())
[task 2022-11-08T05:45:46.658Z]   File "python/mozperftest/mozperftest/runner.py", line 276, in main
[task 2022-11-08T05:45:46.658Z]     run_tests(mach_cmd, args, user_args)
[task 2022-11-08T05:45:46.658Z]   File "python/mozperftest/mozperftest/runner.py", line 168, in run_tests
[task 2022-11-08T05:45:46.658Z]     e.run(metadata)
[task 2022-11-08T05:45:46.658Z]   File "/home/cltbld/tasks/task_166788112213839/fetches/python/mozperftest/mozperftest/environment.py", line 93, in run
[task 2022-11-08T05:45:46.658Z]     metadata = testlayer(syslayer(metadata))
[task 2022-11-08T05:45:46.658Z]   File "/home/cltbld/tasks/task_166788112213839/fetches/python/mozperftest/mozperftest/layers.py", line 156, in __call__
[task 2022-11-08T05:45:46.658Z]     metadata = layer(metadata)
[task 2022-11-08T05:45:46.658Z]   File "/home/cltbld/tasks/task_166788112213839/fetches/python/mozperftest/mozperftest/layers.py", line 68, in __call__
[task 2022-11-08T05:45:46.658Z]     metadata = self.run(metadata)
[task 2022-11-08T05:45:46.658Z]   File "/home/cltbld/tasks/task_166788112213839/fetches/python/mozperftest/mozperftest/test/browsertime/runner.py", line 378, in run
[task 2022-11-08T05:45:46.658Z]     metadata = self._one_cycle(metadata, result_dir)
[task 2022-11-08T05:45:46.658Z]   File "/home/cltbld/tasks/task_166788112213839/fetches/python/mozperftest/mozperftest/test/browsertime/runner.py", line 467, in _one_cycle
[task 2022-11-08T05:45:46.658Z]     raise NodeException(exit_code)
[task 2022-11-08T05:45:46.658Z] mozperftest.test.browsertime.runner.NodeException: 1

Comment 1

[Intermittent Failures Robot]

1 failures in 3445 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform and build breakdown:

• linux1804-64-shippable: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1799655&startday=2022-11-07&endday=2022-11-13&tree=all

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit auto_nag documentation.

Comment 3

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

This has become a permanent failure, the last 6 times these tasks ran, all of them failed. Most recent run.

Comment 4

[Greg Mierzwinski [:sparky]]

I'm looking into it, based on the logs it looks like the paths have changed.

Comment 5

[Greg Mierzwinski [:sparky]]

Ok so the video is not very pretty but it shows that we're being prevented to continue because of a captcha test: https://firefoxci.taskcluster-artifacts.net/LvHNdvjTS2SRFAJvETFbkA/0/public/build/browsertime-results-1/pages/www_google_com/pageload/data/video/9.mp4

The detection is very sporadic, sometimes the tests work other times they don't. I'm going to add a try/catch around the second pageload to try to ignore this failure if it happens. That said, I'm not sure if this will fix it for good, or if it'll just come back again.

Comment 6

[Andrew Creskey [:acreskey]]

Is this a live site test?

Comment 7

[Greg Mierzwinski [:sparky]]

:acreskey, yes it is. I'm not sure if there's a way we could get around this issue outside of changing this to a recorded site test.

Note to everyone that I'll be disabling these tests for now.

Comment 8

[Greg Mierzwinski [:sparky]]

Attachment: Bug 1799655 - Disable g-search perftests for permas. r?#perftest

Comment 9

[Andrew Creskey [:acreskey]]

I believe we had a similar problem with Amazon, although it was sporadic.

Comment 10

[Greg Mierzwinski [:sparky]]

That's possible, this one is also quite sporadic but it always happens at least a handful of times during the test.

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/56fa0454d1c6
Disable g-search perftests for permas. r=perftest-reviewers,kshampur

Comment 12

[Norisz Fay [:noriszfay]]

https://hg.mozilla.org/mozilla-central/rev/56fa0454d1c6