1775109 - Intermittent raptor-browsertime Critical: No page that could be associated with the error:Cannot read property 'login_url' of undefined

Status: RESOLVED FIXED

(Testing :: Raptor, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: archaeopteryx [at] coole-files.de
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=381673150&repo=mozilla-central
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/U-oRJoAiRdKmg53HgQH7eA/runs/0/artifacts/public/logs/live_backing.log

---

Several browsertime live tests (e.g. gmail) fail permanently since bug 1754830 landed.

[task 2022-06-17T13:12:21.425Z] 13:12:21     INFO -  raptor-browsertime Info: Starting a browsertime pageload
[task 2022-06-17T13:12:21.426Z] 13:12:21     INFO -  raptor-browsertime Info: Waiting for 1000 ms (post_startup_delay)
[task 2022-06-17T13:12:22.429Z] 13:12:22     INFO -  raptor-browsertime Info: Obtaining secrets for login...
[task 2022-06-17T13:12:22.522Z] 13:12:22     INFO -  raptor-browsertime Info: Secret status code: 200
[task 2022-06-17T13:12:22.523Z] 13:12:22     INFO -  raptor-browsertime Info: Navigating to url about:blank iteration 1
[task 2022-06-17T13:12:27.572Z] 13:12:27     INFO -  raptor-browsertime Info: Unable to acquire login information
[task 2022-06-17T13:12:27.576Z] 13:12:27    ERROR -  raptor-browsertime Error: Browsertime failed to run
[task 2022-06-17T13:12:27.596Z] 13:12:27 CRITICAL -  raptor-browsertime Critical: No page that could be associated with the error:Cannot read property 'login_url' of undefined
[task 2022-06-17T13:12:27.596Z] 13:12:27     INFO -  raptor-perftest Info: Removing temporary directory: /tmp/tmpguw8q4_b
[task 2022-06-17T13:12:27.603Z] 13:12:27    ERROR -  Traceback (most recent call last):
[task 2022-06-17T13:12:27.604Z] 13:12:27     INFO -    File "/home/cltbld/tasks/task_165547144800847/build/tests/raptor/raptor/raptor.py", line 203, in <module>
[task 2022-06-17T13:12:27.604Z] 13:12:27     INFO -      main()
[task 2022-06-17T13:12:27.605Z] 13:12:27     INFO -    File "/home/cltbld/tasks/task_165547144800847/build/tests/raptor/raptor/raptor.py", line 149, in main
[task 2022-06-17T13:12:27.605Z] 13:12:27     INFO -      success = raptor.run_tests(raptor_test_list, raptor_test_names)
[task 2022-06-17T13:12:27.605Z] 13:12:27     INFO -    File "/home/cltbld/tasks/task_165547144800847/build/tests/raptor/raptor/perftest.py", line 460, in run_tests
[task 2022-06-17T13:12:27.606Z] 13:12:27     INFO -      self.run_test(test, timeout=int(test.get("page_timeout")))
[task 2022-06-17T13:12:27.606Z] 13:12:27     INFO -    File "/home/cltbld/tasks/task_165547144800847/build/tests/raptor/raptor/browsertime/base.py", line 707, in run_test
[task 2022-06-17T13:12:27.607Z] 13:12:27     INFO -      raise Exception(self.browsertime_failure)
[task 2022-06-17T13:12:27.607Z] 13:12:27     INFO -  Exception: No page that could be associated with the error:Cannot read property 'login_url' of undefined
[task 2022-06-17T13:12:27.668Z] 13:12:27    ERROR - Return code: 1

Comment 1

[Kash Shampur [:kshampur] ⌚EST]

So looks like it is failing on all platforms and all live logins associated with SCM 3

hi :sparky is it possible the task cluster secrets in scm level 3 don't have the login_url field? (and how could I check what the data structure for secrets are on scm 3?) I thankfully see that fb and netflix are passing and since those are also available in scm 1 i can confirm it had the login_url field.

Comment 2

[Greg Mierzwinski [:sparky]]

:kshampur, I see that the level 3 login secret has been added. Let me know if there's anything else you need here.

Comment 3

[Kash Shampur [:kshampur] ⌚EST]

This should be OK now as level 3 secrets are available as :sparky mentioned

Comment 4

[Kash Shampur [:kshampur] ⌚EST]

Nevermind, failure still happening e.g. https://treeherder.mozilla.org/jobs?repo=mozilla-central&tier=1%2C2%2C3&searchStr=browsertime%2Clive&revision=a66dcaea419641c5483a43aa7f577b70908d147f

Looking into it

Comment 5

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1754830

Comment 6

[Kash Shampur [:kshampur] ⌚EST]

Attachment: Bug 1775109 - Prevent Raptor-Browsertime login-logic from being used on live sites without Taskcluster secrets. r=#perftest

This patch addresses live login failures occuring in CI. It uses a similar implementation already in perftest_record.js to restrict login-logic to only happen on sites with Taskcluster secrets available. In the future we should improve this if/when more secrets are added and/or we look into being able to easily share reusable code between Raptor and Mozperftest.

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4a5fe0bbb0ab
Prevent Raptor-Browsertime login-logic from being used on live sites without Taskcluster secrets. r=perftest-reviewers,sparky

Comment 8

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/4a5fe0bbb0ab

Comment 9

[BugBot [:suhaib / :marco/ :calixte]]

The patch landed in nightly and beta is affected.
:kshampur, is this bug important enough to require an uplift?

• If yes, please nominate the patch for beta approval.
• If no, please set status-firefox103 to wontfix.

For more information, please visit auto_nag documentation.

Comment 10

[Kash Shampur [:kshampur] ⌚EST]

Attachment: Bug 1775109 - Prevent Raptor-Browsertime login-logic from being used on live sites without Taskcluster secrets. r=#perftest

Approval Request Comment
[Feature/Bug causing the regression]: failing tests on all tier 3 live-browsertime tests with logins
[User impact if declined]: None
[Is this code covered by automated tests?]: yes
[Has the fix been verified in Nightly?]: yes
[Needs manual test from QE? If yes, steps to reproduce]: no
[List of other uplifts needed for the feature/fix]: n/a
[Is the change risky?]: no
[Why is the change risky/not risky?]: does not affect user behavior
[String changes made/needed]: none

Comment 11

[Donal Meehan [:dmeehan]]

Comment on attachment 9283828 [details]
Bug 1775109 - Prevent Raptor-Browsertime login-logic from being used on live sites without Taskcluster secrets. r=#perftest

Approved for 103.0b4, thanks.

Comment 12

[Donal Meehan [:dmeehan]]

https://hg.mozilla.org/releases/mozilla-beta/rev/bd6aa4f1b1fa