514052 - Temporary profiles created by mozrunner are not deleted

Status: VERIFIED FIXED

(Testing :: Mozbase, defect)

Description

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Running my software update tests on Windows which base on mozmill-restart causes mozrunner to not delete the temporarily created profile folder. The following error is thrown. The same happens also for the mozmill CLI. But there are no error is thrown. 

At the moment I don't know if the error interferes with my update test. So I will make it blocking my software update test.

Traceback (most recent call last):
  File "c:\mozilla-build\python25\Scripts\mozmill-restart-script.py", line 8, in
 <module>
    load_entry_point('mozmill==1.2.1', 'console_scripts', 'mozmill-restart')()
  File "c:\mozilla\tools\mozmill\mozmill\__init__.py", line 415, in restart_cli
    RestartCLI().run()
  File "c:\mozilla\tools\mozmill\mozmill\__init__.py", line 402, in run
    super(RestartCLI, self).run(*args, **kwargs)
  File "c:\mozilla\tools\mozmill\mozmill\__init__.py", line 375, in run
    self.options.report)
  File "c:\mozilla\tools\mozmill\mozmill\__init__.py", line 310, in run_tests
    self.run_dir(d, report, sleeptime)
  File "c:\mozilla\tools\mozmill\mozmill\__init__.py", line 285, in run_dir
    profile.cleanup()
  File "build\bdist.win32\egg\mozrunner\__init__.py", line 332, in cleanup
  File "c:\mozilla-build\python25\lib\shutil.py", line 169, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "c:\mozilla-build\python25\lib\shutil.py", line 169, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "c:\mozilla-build\python25\lib\shutil.py", line 169, in rmtree
    rmtree(fullname, ignore_errors, onerror)
  File "c:\mozilla-build\python25\lib\shutil.py", line 174, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File "c:\mozilla-build\python25\lib\shutil.py", line 172, in rmtree
    os.remove(fullname)
WindowsError: [Error 13] Access is denied: 'c:\\docume~1\\mozilla\\locals~1\\tem
p\\tmp_c0r7v.mozrunner\\extensions\\[e-mail]\\.svn\\all-wcprops'

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

The problem is with cleaning up the plugins folder under the profile (line 332):

for plugin in self.plugins_installed:
=>  shutil.rmtree(plugin)

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+2]]

It's because there are read-only files in the profile. shutils.rmtree cannot handle those.

Mikeal we should go the way which has been reported as comment 3 on this blog post: http://code.activestate.com/recipes/193736/#c3

Comment 3

[Mikeal Rogers [:mikeal] (mrogers@)]

Should be fixed in 1.2.1.1

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+2]]

See also http://www.opensubscriber.com/message/[e-mail]/6304218.html for another approach.

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+2]]

The .svn case works now but we could still end up in situations when read-only files exist in the profile. We should get a general solution.

Comment 6

[cmtalbert]

I cannot think of any case where we would get a read-only item in a Mozilla profile that would *not* be a bug in the Mozilla product under test.  I think this fix will be sufficient for the blocking issue here, and if we want a generalized solution we should file on another bug for that.  I don't see why we should continue to block while we hunt for a more perfect solution.

Should we mark this FIXED?

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Ok, so if we see the problem again I will file a new bug. 

For this specific case we cannot mark it fixed. I checked again the temporary folder and the profiles are still there after a restart run. They are not deleted. Only the error cannot be seen anymore. So we still fail somewhere.

Comment 8

[Henrik Skupin [:whimboo][⌚️UTC+2]]

The problem persist on all platforms. We don't delete existing profiles. As seen in bug 508643 we call createProfile twice. But it looks like we loose any reference to the profile because both profiles still exist after a Mozmill run on OS X.

Comment 9

[Mikeal Rogers [:mikeal] (mrogers@)]

How do I remove a read-only file on Windows? It sounds like that would be breaking the permissions model?

Comment 10

[Henrik Skupin [:whimboo][⌚️UTC+2]]

That one has been fixed. We don't have any read-only files anymore in the profile. It's just about deleting the whole profile.

Comment 11

[Mikeal Rogers [:mikeal] (mrogers@)]

Oh yeah, that is another bug, it's logged and will be fixed this week.

Mozmill just needs to make sure it calls the profile.cleanup() function, which it isn't for some reason.

Comment 12

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Now with the changes to bug 508643 to use the profile manager to create a profile we end-up in a mess. Sadly I haven't recognized it before but each time you start Mozmill via the command line another profile is added to the list of available profiles in the profile manager. None of them get deleted. We really should clean it up after the run.

Mikeal, would the profile.cleanup() call manage that? Can we get a fix soon? I will have to run a couple of tests the next days and don't wanna clean up everything manually. Thanks.

Comment 13

[Mikeal Rogers [:mikeal] (mrogers@)]

Fix pushed to mikeal/mozmill/master

http://github.com/mikeal/mozmill/commits/master

http://github.com/mikeal/mozmill/commit/b523e8cf0f2b608a43ce404e31fa4bb504c0e2fe

Comment 14

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Looks good now. Thanks Mikeal. Will test more today.

Comment 15

[Henrik Skupin [:whimboo][⌚️UTC+2]]

With the software update tests run yesterday I cannot see any left-over profiles on all platforms. Marking as verified.