1956510 - Perma WD mozilla/tests/webdriver/classic/addon_uninstall/addon_uninstall.py | test_uninstall_addon[firefox/signed.xpi-True] - AssertionError: assert '1FC7D53C-0B0A-49E7-A8C0-47E77496A919@web-platform-tests.org' not in ['formautofill@mozilla.org', 'newtab@

Reporter

Description

•

1 year ago

treeherder

Filed by: abutkovits [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer?job_id=501031538&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/OSuLPRSPSJmPT3ntfXc_sg/runs/0/artifacts/public/logs/live_backing.log

[task 2025-03-26T12:10:10.151Z] 12:10:10     INFO - TEST-OK | /_mozilla/webdriver/classic/addon_uninstall/addon_uninstall.py | took 6298ms
[task 2025-03-26T12:10:10.151Z] 12:10:10     INFO - Restarting browser for new test group
[task 2025-03-26T12:10:10.222Z] 12:10:10     INFO - STDOUT: filename = 'firefox/signed.xpi', temporary = True
[task 2025-03-26T12:10:10.223Z] 12:10:10     INFO - STDOUT:     @pytest.mark.allow_system_access
[task 2025-03-26T12:10:10.223Z] 12:10:10     INFO - STDOUT:     @pytest.mark.parametrize(
[task 2025-03-26T12:10:10.223Z] 12:10:10     INFO - STDOUT:         "filename, temporary",
[task 2025-03-26T12:10:10.223Z] 12:10:10     INFO - STDOUT:         [("firefox/signed.xpi", True), ("firefox/unsigned.xpi", False)],
[task 2025-03-26T12:10:10.224Z] 12:10:10     INFO - STDOUT:     )
[task 2025-03-26T12:10:10.224Z] 12:10:10     INFO - STDOUT:     def test_uninstall_addon(session, filename, temporary):
[task 2025-03-26T12:10:10.224Z] 12:10:10     INFO - STDOUT:         response = install_addon(
[task 2025-03-26T12:10:10.225Z] 12:10:10     INFO - STDOUT:             session, "addon", get_base64_for_extension_file(filename), temporary
[task 2025-03-26T12:10:10.225Z] 12:10:10     INFO - Closing logging queue
[task 2025-03-26T12:10:10.226Z] 12:10:10     INFO - STDOUT:         )
[task 2025-03-26T12:10:10.226Z] 12:10:10     INFO - STDOUT:         addon_id = assert_success(response)
[task 2025-03-26T12:10:10.226Z] 12:10:10     INFO - STDOUT:     
[task 2025-03-26T12:10:10.227Z] 12:10:10     INFO - STDOUT:         installed_addon_ids = get_ids_for_installed_addons(session)
[task 2025-03-26T12:10:10.227Z] 12:10:10     INFO - STDOUT:     
[task 2025-03-26T12:10:10.227Z] 12:10:10     INFO - STDOUT:         assert addon_id in installed_addon_ids
[task 2025-03-26T12:10:10.231Z] 12:10:10     INFO - STDOUT:     
[task 2025-03-26T12:10:10.231Z] 12:10:10     INFO - STDOUT:         response = uninstall_addon(session, addon_id)
[task 2025-03-26T12:10:10.232Z] 12:10:10     INFO - STDOUT:         assert_success(response)
[task 2025-03-26T12:10:10.232Z] 12:10:10     INFO - STDOUT:     
[task 2025-03-26T12:10:10.232Z] 12:10:10     INFO - STDOUT:         installed_addon_ids = get_ids_for_installed_addons(session)
[task 2025-03-26T12:10:10.232Z] 12:10:10     INFO - STDOUT:     
[task 2025-03-26T12:10:10.232Z] 12:10:10     INFO - STDOUT: >       assert addon_id not in installed_addon_ids
[task 2025-03-26T12:10:10.232Z] 12:10:10     INFO - STDOUT: E       AssertionError: assert '1FC7D53C-0B0A-49E7-A8C0-47E77496A919@web-platform-tests.org' not in ['formautofill@mozilla.org', 'newtab@mozilla.org', 'pictureinpicture@mozilla.org', 'addons-search-detection@mozilla.com', 'webcompat@mozilla.org', 'default-theme@mozilla.org', ...]
[task 2025-03-26T12:10:10.232Z] 12:10:10     INFO - STDOUT: addon_id   = '1FC7D53C-0B0A-49E7-A8C0-47E77496A919@web-platform-tests.org'
[task 2025-03-26T12:10:10.232Z] 12:10:10     INFO - STDOUT: filename   = 'firefox/signed.xpi'
[task 2025-03-26T12:10:10.232Z] 12:10:10     INFO - STDOUT: installed_addon_ids = ['formautofill@mozilla.org',
[task 2025-03-26T12:10:10.233Z] 12:10:10     INFO -  'newtab@mozilla.org',
[task 2025-03-26T12:10:10.233Z] 12:10:10     INFO -  'pictureinpicture@mozilla.org',
[task 2025-03-26T12:10:10.233Z] 12:10:10     INFO -  'addons-search-detection@mozilla.com',
[task 2025-03-26T12:10:10.233Z] 12:10:10     INFO -  'webcompat@mozilla.org',
[task 2025-03-26T12:10:10.233Z] 12:10:10     INFO -  'default-theme@mozilla.org',
[task 2025-03-26T12:10:10.233Z] 12:10:10     INFO -  'firefox-compact-light@mozilla.org',
[task 2025-03-26T12:10:10.233Z] 12:10:10     INFO -  'firefox-compact-dark@mozilla.org',
[task 2025-03-26T12:10:10.233Z] 12:10:10     INFO -  'firefox-alpenglow@mozilla.org',
[task 2025-03-26T12:10:10.233Z] 12:10:10     INFO -  '1FC7D53C-0B0A-49E7-A8C0-47E77496A919@web-platform-tests.org',
[task 2025-03-26T12:10:10.233Z] 12:10:10     INFO -  'gmp-gmpopenh264',
[task 2025-03-26T12:10:10.234Z] 12:10:10     INFO -  'gmp-widevinecdm']
[task 2025-03-26T12:10:10.234Z] 12:10:10     INFO - STDOUT: response   = <Response: status=200 body={"value": null}>
[task 2025-03-26T12:10:10.234Z] 12:10:10     INFO - STDOUT: session    = <Session 53af61c8-f1e3-4d24-a2e1-75c0f14f7614>
[task 2025-03-26T12:10:10.234Z] 12:10:10     INFO - STDOUT: temporary  = True
[task 2025-03-26T12:10:10.234Z] 12:10:10     INFO - STDOUT: tests/web-platform/mozilla/tests/webdriver/classic/addon_uninstall/addon_uninstall.py
[task 2025-03-26T12:10:10.234Z] 12:10:10     INFO - STDOUT: :34: AssertionError
[task 2025-03-26T12:10:10.234Z] 12:10:10     INFO - STDOUT: =============================== warnings summary ===============================
[task 2025-03-26T12:10:10.234Z] 12:10:10     INFO - STDOUT: venv/lib/python3.8/site-packages/mozrunner/utils.py:20
[task 2025-03-26T12:10:10.234Z] 12:10:10     INFO - STDOUT:   /builds/worker/workspace/build/venv/lib/python3.8/site-packages/mozrunner/utils.py:20: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
[task 2025-03-26T12:10:10.234Z] 12:10:10     INFO -     import pkg_resources
[task 2025-03-26T12:10:10.235Z] 12:10:10     INFO - STDOUT: -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
[task 2025-03-26T12:10:10.235Z] 12:10:10     INFO - STDOUT: =========================== short test summary info ============================
[task 2025-03-26T12:10:10.235Z] 12:10:10     INFO - STDOUT: FAILED tests/web-platform/mozilla/tests/webdriver/classic/addon_uninstall/addon_uninstall.py::test_uninstall_addon[firefox/signed.xpi-True] - AssertionError: assert '1FC7D53C-0B0A-49E7-A8C0-47E77496A919@web-platform-tests.org' not in ['formautofill@mozilla.org', 'newtab@mozilla.org', 'pictureinpicture@mozilla.org', 'addons-search-detection@mozilla.com', 'webcompat@mozilla.org', 'default-theme@mozilla.org', ...]
[task 2025-03-26T12:10:10.235Z] 12:10:10     INFO - STDOUT: ==================== 1 failed, 2 passed, 1 warning in 6.01s ====================
[task 2025-03-26T12:10:10.235Z] 12:10:10     INFO - queue closed
[task 2025-03-26T12:10:10.236Z] 12:10:10     INFO - Starting WebDriver: /builds/worker/fetches/geckodriver --host 127.0.0.1 --port 0 -vv --binary /builds/worker/workspace/build/application/firefox/firefox
[task 2025-03-26T12:10:10.339Z] 12:10:10     INFO - PID 2713 | 1742991010237	geckodriver	INFO	Listening on 127.0.0.1:45864
[task 2025-03-26T12:10:10.339Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /status
[task 2025-03-26T12:10:10.339Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session
[task 2025-03-26T12:10:10.340Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}
[task 2025-03-26T12:10:10.340Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}/url
[task 2025-03-26T12:10:10.341Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}/url
[task 2025-03-26T12:10:10.343Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}/back
[task 2025-03-26T12:10:10.345Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}/forward
[task 2025-03-26T12:10:10.347Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}/refresh
[task 2025-03-26T12:10:10.350Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}/title
[task 2025-03-26T12:10:10.352Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}/source
[task 2025-03-26T12:10:10.354Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}/window
[task 2025-03-26T12:10:10.356Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}/window/handles
[task 2025-03-26T12:10:10.358Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}/window/new
[task 2025-03-26T12:10:10.360Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}/window
[task 2025-03-26T12:10:10.360Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}/window/size
[task 2025-03-26T12:10:10.360Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}/window/size
[task 2025-03-26T12:10:10.360Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}/window/position
[task 2025-03-26T12:10:10.360Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}/window/position
[task 2025-03-26T12:10:10.361Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}/window/rect
[task 2025-03-26T12:10:10.361Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}/window/rect
[task 2025-03-26T12:10:10.361Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}/window/minimize
[task 2025-03-26T12:10:10.361Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}/window/maximize
[task 2025-03-26T12:10:10.361Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}/window/fullscreen
[task 2025-03-26T12:10:10.361Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}/window
[task 2025-03-26T12:10:10.362Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}/frame
[task 2025-03-26T12:10:10.362Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}/frame/parent
[task 2025-03-26T12:10:10.362Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}/element
[task 2025-03-26T12:10:10.362Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}/element/{elementId}/element
[task 2025-03-26T12:10:10.362Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}/elements
[task 2025-03-26T12:10:10.362Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}/element/{elementId}/elements
[task 2025-03-26T12:10:10.363Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}/shadow/{shadowId}/element
[task 2025-03-26T12:10:10.363Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}/shadow/{shadowId}/elements
[task 2025-03-26T12:10:10.363Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}/element/active
[task 2025-03-26T12:10:10.363Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}/element/{elementId}/shadow
[task 2025-03-26T12:10:10.363Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}/element/{elementId}/displayed
[task 2025-03-26T12:10:10.363Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}/element/{elementId}/selected
[task 2025-03-26T12:10:10.364Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}/element/{elementId}/attribute/{name}
[task 2025-03-26T12:10:10.364Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}/element/{elementId}/property/{name}
[task 2025-03-26T12:10:10.364Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}/element/{elementId}/css/{propertyName}
[task 2025-03-26T12:10:10.364Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}/element/{elementId}/text
[task 2025-03-26T12:10:10.364Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}/element/{elementId}/computedlabel
[task 2025-03-26T12:10:10.364Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}/element/{elementId}/computedrole
[task 2025-03-26T12:10:10.365Z] 12:10:10     INFO - PID 2713 | 1742991010237	webdriver::server	TRACE	Build standard route for /session/{sessionId}/element/{elementId}/name
[task 2025-03-26T12:10:10.365Z] 12:10:10     INFO - PID 2713 | 1742991010238	webdriver::server	TRACE	Build standard route for /session/{sessionId}/element/{elementId}/rect
[task 2025-03-26T12:10:10.365Z] 12:10:10     INFO - PID 2713 | 1742991010238	webdriver::server	TRACE	Build standard route for /session/{sessionId}/element/{elementId}/enabled
[task 2025-03-26T12:10:10.365Z] 12:10:10     INFO - PID 2713 | 1742991010238	webdriver::server	TRACE	Build standard route for /session/{sessionId}/execute/sync
[task 2025-03-26T12:10:10.365Z] 12:10:10     INFO - PID 2713 | 1742991010238	webdriver::server	TRACE	Build standard route for /session/{sessionId}/execute/async
[task 2025-03-26T12:10:10.366Z] 12:10:10     INFO - PID 2713 | 1742991010238	webdriver::server	TRACE	Build standard route for /session/{sessionId}/cookie
[task 2025-03-26T12:10:10.367Z] 12:10:10     INFO - PID 2713 | 1742991010238	webdriver::server	TRACE	Build standard route for /session/{sessionId}/cookie/{name}
[task 2025-03-26T12:10:10.368Z] 12:10:10     INFO - PID 2713 | 1742991010238	webdriver::server	TRACE	Build standard route for /session/{sessionId}/cookie
[task 2025-03-26T12:10:10.369Z] 12:10:10     INFO - PID 2713 | 1742991010238	webdriver::server	TRACE	Build standard route for /session/{sessionId}/cookie
[task 2025-03-26T12:10:10.370Z] 12:10:10     INFO - PID 2713 | 1742991010238	webdriver::server	TRACE	Build standard route for /session/{sessionId}/cookie/{name}
[task 2025-03-26T12:10:10.372Z] 12:10:10     INFO - PID 2713 | 1742991010238	webdriver::server	TRACE	Build standard route for /session/{sessionId}/timeouts
[task 2025-03-26T12:10:10.373Z] 12:10:10     INFO - PID 2713 | 1742991010238	webdriver::server	TRACE	Build standard route for /session/{sessionId}/timeouts
[task 2025-03-26T12:10:10.373Z] 12:10:10     INFO - PID 2713 | 1742991010238	webdriver::server	TRACE	Build standard route for /session/{sessionId}/element/{elementId}/click
[task 2025-03-26T12:10:10.374Z] 12:10:10     INFO - PID 2713 | 1742991010238	webdriver::server	TRACE	Build standard route for /session/{sessionId}/element/{elementId}/clear
[task 2025-03-26T12:10:10.375Z] 12:10:10     INFO - PID 2713 | 1742991010238	webdriver::server	TRACE	Build standard route for /session/{sessionId}/element/{elementId}/value
[task 2025-03-26T12:10:10.375Z] 12:10:10     INFO - PID 2713 | 1742991010238	webdriver::server	TRACE	Build standard route for /session/{sessionId}/alert/dismiss
[task 2025-03-26T12:10:10.376Z] 12:10:10     INFO - PID 2713 | 1742991010238	webdriver::server	TRACE	Build standard route for /session/{sessionId}/alert/accept
[task 2025-03-26T12:10:10.376Z] 12:10:10     INFO - PID 2713 | 1742991010238	webdriver::server	TRACE	Build standard route for /session/{sessionId}/alert/text
[task 2025-03-26T12:10:10.377Z] 12:10:10     INFO - PID 2713 | 1742991010238	webdriver::server	TRACE	Build standard route for /session/{sessionId}/alert/text
[task 2025-03-26T12:10:10.377Z] 12:10:10     INFO - PID 2713 | 1742991010238	webdriver::server	TRACE	Build standard route for /session/{sessionId}/screenshot
[task 2025-03-26T12:10:10.377Z] 12:10:10     INFO - PID 2713 | 1742991010238	webdriver::server	TRACE	Build standard route for /session/{sessionId}/element/{elementId}/screenshot
[task 2025-03-26T12:10:10.378Z] 12:10:10     INFO - PID 2713 | 1742991010238	webdriver::server	TRACE	Build standard route for /session/{sessionId}/actions
[task 2025-03-26T12:10:10.378Z] 12:10:10     INFO - PID 2713 | 1742991010238	webdriver::server	TRACE	Build standard route for /session/{sessionId}/actions
[task 2025-03-26T12:10:10.379Z] 12:10:10     INFO - PID 2713 | 1742991010238	webdriver::server	TRACE	Build standard route for /session/{sessionId}/permissions
[task 2025-03-26T12:10:10.379Z] 12:10:10     INFO - PID 2713 | 1742991010238	webdriver::server	TRACE	Build standard route for /session/{sessionId}/print
[task 2025-03-26T12:10:10.380Z] 12:10:10     INFO - PID 2713 | 1742991010238	webdriver::server	TRACE	Build standard route for /session/{sessionId}/webauthn/authenticator
[task 2025-03-26T12:10:10.380Z] 12:10:10     INFO - PID 2713 | 1742991010238	webdriver::server	TRACE	Build standard route for /session/{sessionId}/webauthn/authenticator/{authenticatorId}
[task 2025-03-26T12:10:10.380Z] 12:10:10     INFO - PID 2713 | 1742991010238	webdriver::server	TRACE	Build standard route for /session/{sessionId}/webauthn/authenticator/{authenticatorId}/credential
[task 2025-03-26T12:10:10.381Z] 12:10:10     INFO - PID 2713 | 1742991010238	webdriver::server	TRACE	Build standard route for /session/{sessionId}/webauthn/authenticator/{authenticatorId}/credentials
[task 2025-03-26T12:10:10.381Z] 12:10:10     INFO - PID 2713 | 1742991010238	webdriver::server	TRACE	Build standard route for /session/{sessionId}/webauthn/authenticator/{authenticatorId}/credentials/{credentialId}
[task 2025-03-26T12:10:10.382Z] 12:10:10     INFO - PID 2713 | 1742991010238	webdriver::server	TRACE	Build standard route for /session/{sessionId}/webauthn/authenticator/{authenticatorId}/credentials
[task 2025-03-26T12:10:10.382Z] 12:10:10     INFO - PID 2713 | 1742991010238	webdriver::server	TRACE	Build standard route for /session/{sessionId}/webauthn/authenticator/{authenticatorId}/uv
[task 2025-03-26T12:10:10.383Z] 12:10:10     INFO - PID 2713 | 1742991010238	webdriver::server	TRACE	Build vendor route for /session/{sessionId}/moz/context
[task 2025-03-26T12:10:10.384Z] 12:10:10     INFO - PID 2713 | 1742991010238	webdriver::server	TRACE	Build vendor route for /session/{sessionId}/moz/context
[task 2025-03-26T12:10:10.384Z] 12:10:10     INFO - PID 2713 | 1742991010238	webdriver::server	TRACE	Build vendor route for /session/{sessionId}/moz/addon/install
[task 2025-03-26T12:10:10.385Z] 12:10:10     INFO - PID 2713 | 1742991010238	webdriver::server	TRACE	Build vendor route for /session/{sessionId}/moz/addon/uninstall
[task 2025-03-26T12:10:10.385Z] 12:10:10     INFO - PID 2713 | 1742991010238	webdriver::server	TRACE	Build vendor route for /session/{sessionId}/moz/screenshot/full
[task 2025-03-26T12:10:10.385Z] 12:10:10     INFO - Webdriver started successfully.
[task 2025-03-26T12:10:10.386Z] 12:10:10     INFO - Starting runner
[task 2025-03-26T12:10:10.632Z] 12:10:10     INFO - TEST-START | /_mozilla/webdriver/classic/navigate_to/https_upgrade.py

Atila Butkovits

Comment 1

•

1 year ago

Hello Aryx, could this be a manifest issue?

Flags: needinfo?(aryx.bugmail)

Henrik Skupin [:whimboo][⌚️UTC+2]

Assignee

Comment 2

•

1 year ago

On autoland we do not run all the tests but dynamically shuffle tests within chunks. It might be that some combination of the test ordering could cause this. It would be good to compare several jobs that were failing on autoland if there is a common path. We could have a look.

Henrik Skupin [:whimboo][⌚️UTC+2]

Assignee

Comment 3

•

1 year ago

So it looks like that the addition of a new test via the latest wpt-sync changed the chunking of webdriver tests and put this affected test into wd1 jobs. Given that backfills will keep the same order of tests, there will be no visible change.

status-firefox136: --- → unaffected

status-firefox137: --- → unaffected

status-firefox138: --- → affected

status-firefox-esr115: --- → unaffected

status-firefox-esr128: --- → unaffected

Flags: needinfo?(aryx.bugmail)

Keywords: regression

Regressed by: 1956180

Henrik Skupin [:whimboo][⌚️UTC+2]

Assignee

Comment 4

•

1 year ago

The uninstall command for the temporarily installed signed web extension was executed and didn't cause a failure, but it doesn't seem to have actually uninstalled the web extension given that it is still listed afterward:

https://treeherder.mozilla.org/logviewer?job_id=501031538&repo=autoland&lineNumber=228307-228326

INFO - PID 2091 | 1742991009496	webdriver::server	DEBUG	-> POST /session/53af61c8-f1e3-4d24-a2e1-75c0f14f7614/moz/addon/uninstall {"id": "1FC7D53C-0B0A-49E7-A8C0-47E77496A919@web-platform-tests.org"}
INFO - PID 2091 | 1742991009499	Marionette	DEBUG	0 -> [0,12,"Addon:Uninstall",{"id":"1FC7D53C-0B0A-49E7-A8C0-47E77496A919@web-platform-tests.org"}]
INFO - PID 2091 | 1742991009519	Marionette	DEBUG	0 <- [1,12,null,{"value":null}]
INFO - PID 2091 | 1742991009523	webdriver::server	DEBUG	<- 200 OK {"value":null}
INFO - PID 2091 | 1742991009526	webdriver::server	DEBUG	-> GET /session/53af61c8-f1e3-4d24-a2e1-75c0f14f7614/moz/context
INFO - PID 2091 | 1742991009530	Marionette	DEBUG	0 -> [0,13,"Marionette:GetContext",{}]
INFO - PID 2091 | 1742991009530	Marionette	DEBUG	0 <- [1,13,null,{"value":"content"}]
INFO - PID 2091 | 1742991009532	webdriver::server	DEBUG	<- 200 OK {"value":"content"}
INFO - PID 2091 | 1742991009534	webdriver::server	DEBUG	-> POST /session/53af61c8-f1e3-4d24-a2e1-75c0f14f7614/moz/context {"context": "chrome"}
INFO - PID 2091 | 1742991009534	Marionette	DEBUG	0 -> [0,14,"Marionette:SetContext",{"value":"chrome"}]
INFO - PID 2091 | 1742991009534	Marionette	DEBUG	0 <- [1,14,null,{"value":null}]
INFO - PID 2091 | 1742991009535	webdriver::server	DEBUG	<- 200 OK {"value":null}
INFO - PID 2091 | 1742991009537	webdriver::server	DEBUG	-> POST /session/53af61c8-f1e3-4d24-a2e1-75c0f14f7614/execute/async {"script": "\n              const [resolve] = arguments;\ ... .map(x => x.id);\n                resolve(ids);\n              }\n\n              getAllAddons();\n            ", "args": []}
INFO - PID 2091 | 1742991009542	Marionette	DEBUG	0 -> [0,15,"WebDriver:ExecuteAsyncScript",{"args":[],"script":"\n              const [resolve] = arguments;\n              const { AddonManager } = ChromeUtils.importESModule(\n               ...  const ids = addons.map(x => x.id);\n                resolve(ids);\n              }\n\n              getAllAddons();\n            "}]
INFO - PID 2091 | 1742991009545	Marionette	DEBUG	0 <- [1,15,null,{"value":["formautofill@mozilla.org","newtab@mozilla.org","pictureinpicture@mozilla.org","addons-search-detection@mozilla.com","webcompat@mozilla.org","default-theme@mozilla.org","firefox-compact-light@mozilla.org","firefox-compact-dark@mozilla.org","firefox-alpenglow@mozilla.org","1FC7D53C-0B0A-49E7-A8C0-47E77496A919@web-platform-tests.org","gmp-gmpopenh264","gmp-widevinecdm"]}]
INFO - PID 2091 | 1742991009545	webdriver::server	DEBUG	<- 200 OK {"value":["formautofill@mozilla.org","newtab@mozilla.org","pictureinpicture@mozilla.org","addons-search-detection@m ... ox-alpenglow@mozilla.org","1FC7D53C-0B0A-49E7-A8C0-47E77496A919@web-platform-tests.org","gmp-gmpopenh264","gmp-widevinecdm"]}
INFO - PID 2091 | 1742991009547	webdriver::server	DEBUG	-> POST /session/53af61c8-f1e3-4d24-a2e1-75c0f14f7614/moz/context {"context": "content"}
INFO - PID 2091 | 1742991009548	Marionette	DEBUG	0 -> [0,16,"Marionette:SetContext",{"value":"content"}]
INFO - PID 2091 | 1742991009548	Marionette	DEBUG	0 <- [1,16,null,{"value":null}]
INFO - PID 2091 | 1742991009549	webdriver::server	DEBUG	<- 200 OK {"value":null}
INFO - STDOUT: FAILED

Henrik Skupin [:whimboo][⌚️UTC+2]

Assignee

Comment 5

•

1 year ago

Sasha gave a good pointer that we actually run a WebDriver BiDi test before which already installs this extension but doesn't uninstall it given that it actually expects the install to fail:

https://treeherder.mozilla.org/logviewer?job_id=501040264&repo=autoland&lineNumber=83723-83742

Because of bug 1947683 it is marked as expected fail, and as such we do not see the failure.

Running just these two tests after each other allows me to reproduce the failure in the WebDriver classic test. I used the following command locally:

mach wpt --no-install --setpref="remote.log.truncate=false" --webdriver-arg=-vv --webdriver-binary=target/debug/geckodriver testing/web-platform/tests/webdriver/tests/bidi/web_extension/install/invalid.py testing/web-platform/mozilla/tests/webdriver/classic/addon_uninstall/addon_uninstall.py

Henrik Skupin [:whimboo][⌚️UTC+2]

Assignee

Comment 6

•

1 year ago

Attached file Bug 1956510 - [wdspec] Temporarily disable the "test_uninstall_addon" WebDriver classic test due to perma failures. — Details

Phabricator Automation

Updated

•

1 year ago

Assignee: nobody → hskupin

Status: NEW → ASSIGNED

Henrik Skupin [:whimboo][⌚️UTC+2]

Assignee

Comment 7

•

1 year ago

We are going to land a metadata update for this particular failing test so that we do not run it until we figured out the underlying reason.

Keywords: leave-open

Pulsebot

Comment 8

•

1 year ago

Pushed by hskupin@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/f4d143b19597 [wdspec] Temporarily disable the "test_uninstall_addon" WebDriver classic test due to perma failures. r=webdriver-reviewers,Sasha

Pulsebot

Comment 9

•

1 year ago

Backout by csabou@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/304c460fde8d Backed out changeset f4d143b19597 for causing lint failures on addon_uninstall.py.

Cosmin Sabou [:CosminS]

Comment 10

•

1 year ago

Backed out changeset f4d143b19597 for causing lint failures on addon_uninstall.py.

Push with failures

Failure log

Backout link

Flags: needinfo?(hskupin)

Henrik Skupin [:whimboo][⌚️UTC+2]

Assignee

Comment 11

•

1 year ago

Sorry, for the linter failures. I completely missed to check and fix the imports of these test files. I'll update the patch so it can be relanded.

Flags: needinfo?(hskupin)

Pulsebot

Comment 12

•

1 year ago

Pushed by csabou@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/923c32eea3df [wdspec] Temporarily disable the "test_uninstall_addon" WebDriver classic test due to perma failures. r=Sasha,webdriver-reviewers

Norisz Fay [:noriszfay]

Comment 13

•

1 year ago

bugherder

https://hg.mozilla.org/mozilla-central/rev/923c32eea3df

Comment hidden (Intermittent Failures Robot)

Henrik Skupin [:whimboo][⌚️UTC+2]

Assignee

Comment 15

•

1 year ago

A simple Marionette test looks like the following:

    def test(self):
        def get_ids_for_installed_addons():
            with self.marionette.using_context("chrome"):
                return self.marionette.execute_async_script(
                    """
                    const [resolve] = arguments;
                    const { AddonManager } = ChromeUtils.importESModule(
                        "resource://gre/modules/AddonManager.sys.mjs"
                    );

                    async function getAllAddons() {
                        const addons = await AddonManager.getAllAddons();
                        const ids = addons.map(x => x.id);
                        resolve(ids);
                    }

                    getAllAddons();
                    """,
                )

        addons = Addons(self.marionette)
        addons.install(
            "/Users/henrik/code/gecko/testing/web-platform/tests/webdriver/tests/support/webextensions/firefox/signed.xpi", temp=False)

        id = addons.install(
            "/Users/henrik/code/gecko/testing/web-platform/tests/webdriver/tests/support/webextensions/firefox/signed.xpi", temp=True)
        addons.uninstall(id)

        self.assertNotIn(id, get_ids_for_installed_addons())

Hereby it's required to install the web extension as not temporary first. Then with the second temporary installation we may get a duplicate installed. At least given by the logs both installations end-up with the same extension id:

1743058470812	Marionette	DEBUG	3 -> [0,9,"Addon:Install",{"path":"/Users/henrik/code/gecko/testing/web-platform/tests/webdriver/tests/support/webextensions/firefox/signed.xpi","temporary":false}]
1743058470821	Marionette	DEBUG	3 <- [1,9,null,{"value":"1FC7D53C-0B0A-49E7-A8C0-47E77496A919@web-platform-tests.org"}]
1743058470823	Marionette	DEBUG	3 -> [0,10,"Addon:Install",{"path":"/Users/henrik/code/gecko/testing/web-platform/tests/webdriver/tests/support/webextensions/firefox/signed.xpi","temporary":true}]
1743058470828	Marionette	DEBUG	3 <- [1,10,null,{"value":"1FC7D53C-0B0A-49E7-A8C0-47E77496A919@web-platform-tests.org"}]

It looks like that this might be a bug somewhere in the AddonManager APIs, but I need to look more closely. about:addons at least doesn't show a duplicated item.

Luca Greco [:rpl] [:luca] [:lgreco]

Comment 16

•

1 year ago

When a temporarily installed addon is installed on top of a non-temporarily installed with the same addon id it is hiding the non-temporarily installed one not replacing completely and so when the temporarily installed addon is uninstalled then it is expected to unhide the non-temporarily installed one.

Uninstalling again would ensure the addon is completely gone, e.g. this tweaked version of the test from comment 15 does show that and passes:

class TempInstallOnTop(
    MarionetteTestCase
):
    def test(self):
        def get_ids_for_installed_addons():
            with self.marionette.using_context("chrome"):
                return self.marionette.execute_async_script(
                    """
                    const [resolve] = arguments;
                    const { AddonManager } = ChromeUtils.importESModule(
                        "resource://gre/modules/AddonManager.sys.mjs"
                    );

                    async function getAllAddons() {
                        const addons = await AddonManager.getAllAddons();
                        const ids = addons.map(x => x.id);
                        resolve(ids);
                    }

                    getAllAddons();
                    """,
                )

        ADDON_XPI_PATH = "/home/rpl/my/mozlab/mc-reviews/treestyletab@piro.sakura.ne.jp.xpi"
        addons = Addons(self.marionette)
        # install non-temporarily addon
        addons.install(ADDON_XPI_PATH, temp=False)
        # install temporarily installed addon (same addon id) on top
        id = addons.install(ADDON_XPI_PATH, temp=True)
        # uninstall the temporarily installed addon.
        addons.uninstall(id)
        # uninstall the non-temporarily installed addon (unhidden when the temporarily installed is removed).
        addons.uninstall(id)
        
        self.assertNotIn(id, get_ids_for_installed_addons())

Henrik Skupin [:whimboo][⌚️UTC+2]

Assignee

Comment 17

•

1 year ago

Thank you Luca! This is helpful information which actually leads us to putting focus on installing addons temporary by default as Chrome is already doing. That would avoid situations like these between tests.

We should put focus on bug 1947678 soon. Until it's fixed and we can re-enable the test lets leave this bug open.

Assignee: hskupin → nobody

Status: ASSIGNED → NEW

status-firefox138: affected → disabled

Depends on: 1947678

Keywords: test-disabled

Henrik Skupin [:whimboo][⌚️UTC+2]

Assignee

Updated

•

1 year ago

Depends on: 1956885

Henrik Skupin [:whimboo][⌚️UTC+2]

Assignee

Comment 18

•

1 year ago

Attached file Bug 1956510 - [wdspec] Revert temporarily disable the "test_uninstall_addon" WebDriver classic test due to perma failures. — Details

This reverts commit 1aa6e6005666b9a01339f0e01dbc8f11507e4cf1.

Comment hidden (Intermittent Failures Robot)

BugBot [:suhaib / :marco/ :calixte]

Comment 20

•

1 year ago

Set release status flags based on info from the regressing bug 1956180

status-firefox139: --- → affected

Henrik Skupin [:whimboo][⌚️UTC+2]

Assignee

Updated

•

1 year ago

status-firefox139: affected → disabled

Pulsebot

Comment 21

•

1 year ago

Pushed by hskupin@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/5be4df497481 [wdspec] Revert temporarily disable the "test_uninstall_addon" WebDriver classic test due to perma failures. r=webdriver-reviewers,jgraham

Norisz Fay [:noriszfay]

Comment 22

•

1 year ago

bugherder

https://hg.mozilla.org/mozilla-central/rev/5be4df497481

Henrik Skupin [:whimboo][⌚️UTC+2]

Assignee

Comment 23

•

1 year ago

Fixed by bug 1947678. We might want to think about an uplift given that it is early in 138 beta and the webextension module was shipped in 137, which just got released.

Assignee: nobody → hskupin

Status: NEW → RESOLVED

Closed: 1 year ago

status-firefox139: disabled → fixed

Resolution: --- → FIXED

Target Milestone: --- → 139 Branch

BugBot (nomail) [:suhaib / :marco/ :calixte]

Updated

•

1 year ago

Keywords: leave-open

Henrik Skupin [:whimboo][⌚️UTC+2]

Assignee

Comment 24

•

1 year ago

Basically fixed by bug 1947678. So I don't think we have to account points for this bug.

Keywords: test-disabled

Whiteboard: [webdriver:m16][webdriver:external]

Bug 1956510 - [wdspec] Temporarily disable the "test_uninstall_addon" WebDriver classic test due to perma failures. 1 year ago Henrik Skupin [:whimboo][⌚️UTC+2] 48 bytes, text/x-phabricator-request		Details \| Review
Bug 1956510 - [wdspec] Revert temporarily disable the "test_uninstall_addon" WebDriver classic test due to perma failures. 1 year ago Henrik Skupin [:whimboo][⌚️UTC+2] 48 bytes, text/x-phabricator-request		Details \| Review

	cf	headless	no_variant	nofis
android-em-7-0-x86_64-qr--/opt				1
linux1804-64-asan-qr--/opt		4	4	1
linux1804-64-qr--/opt		2	3	1
linux1804-64-tsan-qr--/opt		4	4
linux2204-64-wayland--/opt	1	3	56
macosx1470-64--/opt		4	4
windows11-64-24h2--/opt		3	4

	cf	headless	no_variant	nofis
android-em-7-0-x86_64-qr--/opt				1
linux1804-64-asan-qr--/opt		4	4	1
linux1804-64-qr--/opt		2	3	1
linux1804-64-tsan-qr--/opt		4	4
linux2204-64-wayland--/opt	1	3	72
macosx1470-64--/opt		4	4
windows11-64-24h2--/opt		3	4

Perma WD mozilla/tests/webdriver/classic/addon_uninstall/addon_uninstall.py | test_uninstall_addon[firefox/signed.xpi-True] - AssertionError: assert '1FC7D53C-0B0A-49E7-A8C0-47E77496A919@web-platform-tests.org' not in ['formautofill@mozilla.org', 'newtab@

Repository breakdown:

Table

For more details, see:

Repository breakdown:

Table

For more details, see: