1434281 - test_refresh_firefox.py doesn't clean-up cache folder of created profile

Status: RESOLVED FIXED

(Firefox :: Migration, defect, P3)

Description

[Gian-Carlo Pascutto [:gcp]]

https://bugzilla.mozilla.org/show_bug.cgi?id=1386404#c104

During marionette testruns, it seems the .cache/mozilla directory is not properly cleaned up, and this causes that directory to accumulate a large number of profile-subdirectories, each (for example) containing SafeBrowsing and cache2 data.

Cleaning out this directory makes some tests pass that are orange due to timeouts otherwise.

Comment 1

[Henrik Skupin (:whimboo) [⌚️UTC+1]]

So by default the cache should be placed under the profile directory. That is what usually happens for mozprofile created profiles in use by Marionette. Is that different for the refresh firefox test, which is failing in your case?

Also with XDG_CACHE_HOME set to a different folder, mozprofile would have to know the exact path of the cache. Gian-Carlo can you please check if the folder name matches the one from the profile itself?

Comment 2

[Gian-Carlo Pascutto [:gcp]]

>So by default the cache should be placed under the profile directory.

The way this is worded is a bit suspect. There is a .cache/XDG_CACHE_HOME directory on every system. Inside this directory, we make a per-profile specific subdir for various caches, that has got nothing to do with what is regularly considered "the profile".

>Is that different for the refresh firefox test, which is failing in your case?

No. About the only specific thing that test does is have two profiles alive at once during the test.

>Gian-Carlo can you please check if the folder name matches the one from the profile itself?

I don't really understand what you're asking here.

Comment 3

[Henrik Skupin (:whimboo) [⌚️UTC+1]]

(In reply to Gian-Carlo Pascutto [:gcp] from comment #2)
> >So by default the cache should be placed under the profile directory.
> 
> The way this is worded is a bit suspect. There is a .cache/XDG_CACHE_HOME
> directory on every system. Inside this directory, we make a per-profile
> specific subdir for various caches, that has got nothing to do with what is
> regularly considered "the profile".

Maybe this is not the case because the profile gets created at a temporary location and not withing app data? For each profile which is created via mozprofile the subfolders like `OfflineCache`, `Cache2` etc are always located under the created profile in %tmp%. I don't know where the decision is made which path to use.

> >Gian-Carlo can you please check if the folder name matches the one from the profile itself?
> 
> I don't really understand what you're asking here.

Maybe the above explains a bit more, and might be the underlying reason why XDG_CACHE_HOME is not honored.

Comment 4

[Gian-Carlo Pascutto [:gcp]]

>For each profile which is created via mozprofile the subfolders like `OfflineCache`, `Cache2` etc are always located under the created profile in %tmp%

As explained above, this does not appear to be the case, see also: https://bugzilla.mozilla.org/show_bug.cgi?id=1386404#c104

Comment 5

[Henrik Skupin (:whimboo) [⌚️UTC+1]]

(In reply to Gian-Carlo Pascutto [:gcp] from comment #4)
> As explained above, this does not appear to be the case, see also:
> https://bugzilla.mozilla.org/show_bug.cgi?id=1386404#c104

This is a temporary profile which gets created by Firefox but only for this specific test. In all other cases we use mozprofile to create the user profile, which then have the cache files located in itself. 

So the test test_refresh_firefox.py should then also clean-up the cache folder during teardown.

Gian-Carlo, can you please check that no other test beside the above leaves traces behind? If that is the case we should move this bug to Firefox / Migration.

Comment 6

[Gian-Carlo Pascutto [:gcp]]

>Gian-Carlo, can you please check that no other test beside the above leaves traces behind?

I only found this case because it was causing a test to go orange on a (mostly) not related patch, and I stumbled upon this bug while inspecting a loaner. I have no idea how to do a generic check here?

Comment 7

[Henrik Skupin (:whimboo) [⌚️UTC+1]]

Ok, so this is clearly a problem with `test_refresh_firefox.py` and that it creates the new to migrate to profile on its own. Beside the profile folder it clearly should remove the cache folder too.

Gijs could you or someone else please have a look at this? Thanks

Comment 8

[:Gijs (he/him)]

Attachment: Bug 1434281 - also delete local/cache profile directory from refresh test, r?whimboo

Ensure local profile directory is also deleted in test teardown.

Comment 9

[Phabricator Automation]

Comment on attachment 8997086 [details]
Bug 1434281 - also delete local/cache profile directory from refresh test, r?whimboo

Henrik Skupin (:whimboo) has approved the revision.

https://phabricator.services.mozilla.com/D2671

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/1a0c506d69a8
also delete local/cache profile directory from refresh test, r=whimboo

Comment 11

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/1a0c506d69a8