Closed Bug 1901897 Opened 5 months ago Closed 5 months ago

Perma [tier 2] opt testing/firefox-ui/tests/functional/safebrowsing/test_initial_download.py TestSafeBrowsingInitialDownload.test_safe_browsing_initial_download | AssertionError: '<something>' not found in ['google4']

Categories

(Toolkit :: Safe Browsing, defect, P5)

Firefox 129
defect

Tracking

()

RESOLVED FIXED
129 Branch
Tracking Status
firefox-esr115 --- unaffected
firefox127 --- unaffected
firefox128 --- unaffected
firefox129 --- fixed

People

(Reporter: intermittent-bug-filer, Assigned: hsohaney)

References

(Regression)

Details

(Keywords: intermittent-failure, regression, Whiteboard: [stockwell disable-recommended])

Attachments

(1 file)

Filed by: acseh [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=461925340&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Qm3LvrasQqWZqHOafHT9Pg/runs/0/artifacts/public/logs/live_backing.log


[task 2024-06-11T23:00:58.005Z] 23:00:58     INFO -  1718146858005	Marionette	DEBUG	2 -> [0,922,"Marionette:SetContext",{"value":"content"}]
[task 2024-06-11T23:00:58.005Z] 23:00:58     INFO -  1718146858005	Marionette	DEBUG	2 <- [1,922,null,{"value":null}]
[task 2024-06-11T23:00:58.046Z] 23:00:58     INFO - TEST-UNEXPECTED-FAIL | testing/firefox-ui/tests/functional/safebrowsing/test_initial_download.py TestSafeBrowsingInitialDownload.test_safe_browsing_initial_download | AssertionError: 'social-track-digest256.sbstore' not found in ['google4']
[task 2024-06-11T23:00:58.046Z] 23:00:58     INFO - Traceback (most recent call last):
[task 2024-06-11T23:00:58.046Z] 23:00:58     INFO -   File "D:\task_171814384832338\build\venv\Lib\site-packages\marionette_harness\marionette_test\testcases.py", line 198, in run
[task 2024-06-11T23:00:58.046Z] 23:00:58     INFO -     testMethod()
[task 2024-06-11T23:00:58.047Z] 23:00:58     INFO -   File "D:\task_171814384832338\build\tests\firefox-ui\tests\testing\firefox-ui\tests\functional\safebrowsing\test_initial_download.py", line 128, in test_safe_browsing_initial_download
[task 2024-06-11T23:00:58.047Z] 23:00:58     INFO -     self.assertIn(f, files_on_disk_toplevel)
[task 2024-06-11T23:00:58.047Z] 23:00:58     INFO - TEST-INFO took 14870ms
[task 2024-06-11T23:00:58.047Z] 23:00:58     INFO -  1718146858040	Marionette	DEBUG	2 -> [0,923,"Marionette:GetContext",{}]
[task 2024-06-11T23:00:58.047Z] 23:00:58     INFO -  1718146858041	Marionette	DEBUG	2 <- [1,923,null,{"value":"content"}]
[task 2024-06-11T23:00:58.048Z] 23:00:58     INFO -  1718146858041	Marionette	DEBUG	2 -> [0,924,"WebDriver:DeleteSession",{}]
[task 2024-06-11T23:00:58.048Z] 23:00:58     INFO -  1718146858042	Marionette	TRACE	[1] MarionetteCommands actor destroyed for window id 2
[task 2024-06-11T23:00:58.048Z] 23:00:58     INFO -  1718146858042	Marionette	TRACE	[9] MarionetteCommands actor destroyed for window id 4294967297
[task 2024-06-11T23:00:58.048Z] 23:00:58     INFO -  1718146858044	Marionette	DEBUG	2 <- [1,924,null,{"value":null}]
[task 2024-06-11T23:00:58.122Z] 23:00:58     INFO - Application command: D:\task_171814384832338\build\application\firefox\firefox.exe -no-remote -marionette --wait-for-browser -profile D:\task_171814384832338\build\tmpgs_3tq2b.mozrunner
[task 2024-06-11T23:00:58.591Z] 23:00:58     INFO -  1718146858592	Marionette	INFO	Marionette enabled
[task 2024-06-11T23:00:58.595Z] 23:00:58     INFO -  1718146858594	Marionette	TRACE	Received observer notification final-ui-startup
[task 2024-06-11T23:00:58.685Z] 23:00:58     INFO -  1718146858686	Marionette	INFO	Listening on port 2828
[task 2024-06-11T23:00:58.689Z] 23:00:58     INFO -  1718146858689	Marionette	DEBUG	Marionette is listening
[task 2024-06-11T23:00:58.701Z] 23:00:58     INFO -  1718146858709	Marionette	DEBUG	Accepted connection 0 from 127.0.0.1:49848
[task 2024-06-11T23:00:58.719Z] 23:00:58     INFO -  1718146858719	Marionette	DEBUG	Closed connection 0
[task 2024-06-11T23:00:58.764Z] 23:00:58     INFO -  1718146858774	Marionette	DEBUG	Accepted connection 1 from 127.0.0.1:49851
[task 2024-06-11T23:00:58.775Z] 23:00:58     INFO -  console.error: (new Error("Unexpected content-type \"text/plain;charset=US-ASCII\"", "resource://services-settings/Utils.sys.mjs", 406))
[task 2024-06-11T23:00:58.777Z] 23:00:58     INFO -  console.error: (new Error("Unexpected content-type \"text/plain;charset=US-ASCII\"", "resource://services-settings/Utils.sys.mjs", 406))
[task 2024-06-11T23:00:58.777Z] 23:00:58     INFO -  console.error: (new Error("Unexpected content-type \"text/plain;charset=US-ASCII\"", "resource://services-settings/Utils.sys.mjs", 406))
[task 2024-06-11T23:00:58.936Z] 23:00:58     INFO -  1718146858936	Marionette	DEBUG	Closed connection 1
[task 2024-06-11T23:00:58.937Z] 23:00:58     INFO -  1718146858936	Marionette	DEBUG	Accepted connection 2 from 127.0.0.1:49852
[task 2024-06-11T23:00:58.984Z] 23:00:58     INFO -  1718146858989	Marionette	DEBUG	2 -> [0,1,"WebDriver:NewSession",{"strictFileInteractability":true}]
[task 2024-06-11T23:00:58.994Z] 23:00:58     INFO -  1718146858993	Marionette	DEBUG	Waiting for initial application window
[task 2024-06-11T23:01:00.474Z] 23:01:00     INFO -  JavaScript error: resource://gre/modules/CaptiveDetect.sys.mjs, line 297: NS_ERROR_FAILURE: No canonical URL set up.
[task 2024-06-11T23:01:00.601Z] 23:01:00     INFO -  console.error: (new Error("Unexpected content-type \"text/plain;charset=US-ASCII\"", "resource://services-settings/Utils.sys.mjs", 406))
[task 2024-06-11T23:01:00.607Z] 23:01:00     INFO -  console.error: (new Error("Unexpected content-type \"text/plain;charset=US-ASCII\"", "resource://services-settings/Utils.sys.mjs", 406))
[task 2024-06-11T23:01:00.616Z] 23:01:00     INFO -  console.error: (new Error("Unexpected content-type \"text/plain;charset=US-ASCII\"", "resource://services-settings/Utils.sys.mjs", 406))
[task 2024-06-11T23:01:00.628Z] 23:01:00     INFO -  console.error: (new Error("Unexpected content-type \"text/plain;charset=US-ASCII\"", "resource://services-settings/Utils.sys.mjs", 406))
[task 2024-06-11T23:01:00.710Z] 23:01:00     INFO -  console.error: (new Error("Unexpected content-type \"text/plain;charset=US-ASCII\"", "resource://services-settings/Utils.sys.mjs", 406))
[task 2024-06-11T23:01:01.040Z] 23:01:01     INFO -  console.error: (new Error("Unexpected content-type \"text/plain;charset=US-ASCII\"", "resource://services-settings/Utils.sys.mjs", 406))
[task 2024-06-11T23:01:01.241Z] 23:01:01     INFO -  console.error: (new Error("Unexpected content-type \"text/plain;charset=US-ASCII\"", "resource://services-settings/Utils.sys.mjs", 406))
[task 2024-06-11T23:01:01.258Z] 23:01:01     INFO -  console.error: (new Error("Unexpected content-type \"text/plain;charset=US-ASCII\"", "resource://services-settings/Utils.sys.mjs", 406))
[task 2024-06-11T23:01:01.259Z] 23:01:01     INFO -  console.error: (new Error("Unexpected content-type \"text/plain;charset=US-ASCII\"", "resource://services-settings/Utils.sys.mjs", 406))
[task 2024-06-11T23:01:01.260Z] 23:01:01     INFO -  console.error: (new Error("Unexpected content-type \"text/plain;charset=US-ASCII\"", "resource://services-settings/Utils.sys.mjs", 406))
[task 2024-06-11T23:01:01.351Z] 23:01:01     INFO -  console.error: (new Error("Unexpected content-type \"text/plain;charset=US-ASCII\"", "resource://services-settings/Utils.sys.mjs", 406))
[task 2024-06-11T23:01:01.400Z] 23:01:01     INFO -  1718146861400	Marionette	TRACE	Received observer notification browser-idle-startup-tasks-finished
[task 2024-06-11T23:01:01.407Z] 23:01:01     INFO -  1718146861406	RemoteAgent	TRACE	[9] ProgressListener Start: expectNavigation=false resolveWhenStarted=false unloadTimeout=5000 waitForExplicitStart=false
[task 2024-06-11T23:01:01.407Z] 23:01:01     INFO -  1718146861407	RemoteAgent	TRACE	[9] ProgressListener Setting unload timer (5000ms)
[task 2024-06-11T23:01:01.408Z] 23:01:01     INFO -  1718146861407	RemoteAgent	TRACE	[9] Wait for initial navigation: isInitial=false, isLoadingDocument=false
[task 2024-06-11T23:01:01.408Z] 23:01:01     INFO -  1718146861407	RemoteAgent	TRACE	[9] Document already finished loading: about:blank
[task 2024-06-11T23:01:01.408Z] 23:01:01     INFO -  1718146861407	RemoteAgent	TRACE	[9] ProgressListener Stop: has error=false url=about:blank
[task 2024-06-11T23:01:01.425Z] 23:01:01     INFO -  1718146861424	Marionette	DEBUG	2 <- [1,1,null,{"sessionId":"7274643c-ad26-4215-9ee8-be49339d9d36","capabilities":{"acceptInsecureCerts":false,"browserName":"firefox","browserVersion":"129.0a1","platformName":"windows","unhandledPromptBehavior":"dismiss and notify","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:129.0) Gecko/20100101 Firefox/129.0","moz:buildID":"20240611223415","moz:headless":false,"moz:platformVersion":"10.0","moz:processID":7860,"moz:profile":"D:\\task_171814384832338\\build\\tmpgs_3tq2b.mozrunner","moz:shutdownTimeout":60000,"pageLoadStrategy":"normal","timeouts":{"implicit":0,"pageLoad":300000,"script":30000},"setWindowRect":true,"strictFileInteractability":true,"moz:accessibilityChecks":false,"moz:webdriverClick":true,"moz:windowless":false,"proxy":{}}}]
[task 2024-06-11T23:01:01.433Z] 23:01:01     INFO -  1718146861432	Marionette	DEBUG	2 -> [0,2,"Marionette:SetContext",{"value":"content"}]
[task 2024-06-11T23:01:01.435Z] 23:01:01     INFO -  1718146861432	Marionette	DEBUG	2 <- [1,2,null,{"value":null}]
[task 2024-06-11T23:01:01.435Z] 23:01:01     INFO -  1718146861434	Marionette	DEBUG	2 -> [0,3,"WebDriver:DeleteSession",{}]
[task 2024-06-11T23:01:01.437Z] 23:01:01     INFO -  1718146861436	Marionette	DEBUG	2 <- [1,3,null,{"value":null}]
[task 2024-06-11T23:01:01.443Z] 23:01:01     INFO -  1718146861441	Marionette	DEBUG	Closed connection 2
[task 2024-06-11T23:01:01.446Z] 23:01:01     INFO - TEST-START | testing/firefox-ui/tests/functional/safebrowsing/test_notification.py TestSafeBrowsingNotificationBar.test_notification_bar
Summary: Perma testing/firefox-ui/tests/functional/safebrowsing/test_initial_download.py TestSafeBrowsingInitialDownload.test_safe_browsing_initial_download | AssertionError: 'social-track-digest256.sbstore' not found in ['google4'] → Perma [tier 2] testing/firefox-ui/tests/functional/safebrowsing/test_initial_download.py TestSafeBrowsingInitialDownload.test_safe_browsing_initial_download | AssertionError: 'social-track-digest256.sbstore' not found in ['google4']

:hsohaney, since you are the author of the regressor, bug 1899359, could you take a look?

For more information, please visit BugBot documentation.

Flags: needinfo?(hsohaney)
Summary: Perma [tier 2] testing/firefox-ui/tests/functional/safebrowsing/test_initial_download.py TestSafeBrowsingInitialDownload.test_safe_browsing_initial_download | AssertionError: 'social-track-digest256.sbstore' not found in ['google4'] → Perma [tier 2] testing/firefox-ui/tests/functional/safebrowsing/test_initial_download.py TestSafeBrowsingInitialDownload.test_safe_browsing_initial_download | AssertionError: '<something>' not found in ['google4']

Set release status flags based on info from the regressing bug 1899359

Component: Firefox UI Tests → Safe Browsing
Product: Testing → Toolkit
QA Contact: hskupin

Tim, can we find someone to get this regression fixed in case of Harshit not being around?

Flags: needinfo?(tihuang)

This is a perma failure basically on opt builds across all platforms.

Dimi could you maybe help at least with the investigation?

Flags: needinfo?(dlee)
OS: Unspecified → All
Hardware: Unspecified → All
Summary: Perma [tier 2] testing/firefox-ui/tests/functional/safebrowsing/test_initial_download.py TestSafeBrowsingInitialDownload.test_safe_browsing_initial_download | AssertionError: '<something>' not found in ['google4'] → Perma [tier 2] opt testing/firefox-ui/tests/functional/safebrowsing/test_initial_download.py TestSafeBrowsingInitialDownload.test_safe_browsing_initial_download | AssertionError: '<something>' not found in ['google4']
Version: unspecified → Firefox 129

weird, from the log, we did successfully download the safeborwsing data. I'll take a look.

(In reply to Dimi Lee [:dimi] from comment #11)

weird, from the log, we did successfully download the safeborwsing data. I'll take a look.

Discussed with :Tim, the issue happened because the services.settings.server preference doesn't point to the real server, which is why the download fails. :Tim, could someone in your team help fix this issue? thanks!

Flags: needinfo?(dlee)

Oh, so it's this line then? https://hg.mozilla.org/mozilla-central/rev/859498ffdfa5#l1.14

Dimi, which impact does that have for our Nightly users? Does safebrowsing not work correctly for them at the moment?

Assignee: nobody → hsohaney
Status: NEW → ASSIGNED
Attachment #9408213 - Attachment description: WIP: Bug 1901897 - Fix test failure caused by switching to remote settings from shavar. WIP → Bug 1901897 - Fix test failure caused by switching to remote settings from shavar. r?timhuang

(In reply to Henrik Skupin [:whimboo][⌚️UTC+1] from comment #13)

Oh, so it's this line then? https://hg.mozilla.org/mozilla-central/rev/859498ffdfa5#l1.14

Dimi, which impact does that have for our Nightly users? Does safebrowsing not work correctly for them at the moment?

There is no impact on our users. This is a configuration issue for the testing environment.

Flags: needinfo?(tihuang)

The fix is being worked on!

Flags: needinfo?(hsohaney)
Pushed by hsohaney@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/2cb6b880de5c Fix test failure caused by switching to remote settings from shavar. r=webdriver-reviewers,timhuang,whimboo
Status: ASSIGNED → RESOLVED
Closed: 5 months ago
Resolution: --- → FIXED
Target Milestone: --- → 129 Branch
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: