Open Bug 2081122 Opened 1 month ago Updated 13 days ago

geckodriver leaves temporary profile directories (rust_mozprofile) behind

Categories

(Testing :: geckodriver, defect, P3)

defect

Tracking

(Not tracked)

People

(Reporter: whimboo, Unassigned)

References

(Depends on 1 open bug)

Details

(Whiteboard: [webdriver:backlog])

While writing tests for bug 2078511 I noticed that we miss to remove profiles as created by the create_custom_profile in testing/web-platform/mozilla/tests/webdriver/support/fixtures.py and the create_custom_profile helper in support/helpers.py do not reliably remove the profiles they create. There are two separate problems:

  1. The fixture never calls cleanup()

Inside _create_custom_profile the assignment to profile creates a new local variable, so the teardown always sees the outer profile = None and skips profile.cleanup(). Collecting the created profiles in a list fixes this, and also covers tests that create more than one profile:

@pytest.fixture(name="create_custom_profile")
def fixture_create_custom_profile(default_preferences, profile_folder):
    profiles = []

    def _create_custom_profile(clone=True):
        profile = create_custom_profile(
            profile_folder, default_preferences, clone=clone
        )
        profiles.append(profile)

        return profile

    yield _create_custom_profile

    for profile in profiles:
        profile.cleanup()
  1. Profiles created with clone=False are never deleted, even when cleanup() runs

The helper creates them with Profile(tempfile.mkdtemp(prefix="wdspec-")). Because a path is passed, mozprofile sets create_new = False and treats the directory as owned by the caller, so Profile.cleanup() does not remove it. Cloned profiles are not affected, because Profile.clone() sets create_new = True. Letting mozprofile create the temporary directory itself fixes it:

    else:
        profile = Profile(preferences=default_preferences)

Alternatively, keep the current call and set profile.create_new = True afterwards, if the wdspec- prefix should be kept.

Impact:

  • Cloned profiles are eventually removed by Profile.__del__ when garbage collected, so they do not leak in practice, but the cleanup is not deterministic.
  • Every test using create_custom_profile(clone=False) leaves a wdspec-* directory in the temporary folder, for example classic/new_session/accept_insecure_certs.py, bidi/session/new/accept_insecure_certs.py, bidi/browser/create_user_context/accept_insecure_certs.py and bidi/browsing_context/navigate/error.py. Callers using the helper directly (like the new harness/app_data.py from bug 2078511) only benefit from the second fix.

The severity field is not set for this bug.
:whimboo, could you have a look please?

For more information, please visit BugBot documentation.

Flags: needinfo?(hskupin)

Bug 2078511 might help here.

Depends on: 2078511
Flags: needinfo?(hskupin)
Severity: -- → S3
Priority: -- → P3
Whiteboard: [webdriver:backlog]

Something to add here specifically for macOS is that there is a difference between the system (/tmp or /private/tmp) and the user (/var/folders/.../T/) temp folder. The automatic clean-up only happens for the system folder each day, but everything from the user tmp folder is left on disk until the next reboot. The temporary location that geckodriver and wdspec are using is the user temp folder. In my case I've now about 4GB of disk space used by orphaned wdspec and mozrunner profiles.

This actually doesn't only affect wdspec tests but as well other WebDriver clients like Selenium as reported at https://github.com/mozilla/geckodriver/issues/2258.

Component: Marionette → geckodriver
Product: Remote Protocol → Testing
Summary: Mozilla wdspec tests leave temporary profile directories behind → geckodriver leaves temporary profile directories (rust_mozprofile) behind
You need to log in before you can comment on or make changes to this bug.