1818980 - Marionette test bleedthrough

Status: NEW

(Testing :: Marionette Client and Harness, defect, P3)

Description

[Joshua Marshall]

In CI, imports in an earlier marionette test are visible to later tests. E.g. an import in dom/workers/test/marionette/test_service_workers_disabled.py is visible to toolkit/components/extensions/test/marionette/test_extension_serviceworkers_purged_on_pref_disabled.py.

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

I cannot find this test in mozilla-central yet. So what specifically are you referring to? Something on the Marionette client side or within some JS module in Firefox?

Comment 2

[Joshua Marshall]

For example, the import at https://searchfox.org/mozilla-central/source/toolkit/components/extensions/test/marionette/test_extension_serviceworkers_purged_on_pref_disabled.py#14 sees a sys.path that was modified by previously-run tests. test_service_workers_disabled.py is incoming, but you can reproduce the behavior by doing a sys.path.append in https://searchfox.org/mozilla-central/source/dom/workers/test/marionette/test_service_workers_at_startup.py.

Comment 3

[BugBot [:suhaib / :marco/ :calixte]]

The severity field is not set for this bug.
:whimboo, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Does that happen because you added the local import folder via sys.path.append()? Also what exactly bleeds through? Do you have some more details? Does it work if you remove the path from sys.path after the test has been finished?

Comment 5

[Joshua Marshall]

Yes, when one test does a sys.path.append, the modified path persists through subsequent tests. I was expecting a clean start for each test. Removing the added directory at the end of the test would fix the issue (but right now no marionette tests clean after themselves up in this way).

Comment 6

[BugBot [:suhaib / :marco/ :calixte]]

The severity field is not set for this bug.
:whimboo, could you have a look please?

For more information, please visit auto_nag documentation.