843561 - WARNING - Failed to remove C:\slave\test\build on try 1.

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Carsten Book [:Tomcat]]

see http://dev-master01.build.scl1.mozilla.com:8042/builders/Rev3%20WINNT%206.2%20cedar%20opt%20test%20crashtest/builds/12/steps/run_script/logs/stdio

1:17:44     INFO - Running command: rmdir /S /Q "C:\slave\test\build"
01:18:49     INFO -  C:\slave\test\build\venv\Scripts - The directory is not empty.
01:18:49     INFO - Return code: 0
01:18:49  WARNING - Failed to remove C:\slave\test\build on try 1.
01:18:49     INFO - Sleeping 2 seconds...
01:18:51     INFO - Running command: rmdir /S /Q "C:\slave\test\build"
01:18:51     INFO -  C:\slave\test\build\venv\Scripts\mozinstall.exe - Access is denied.
01:18:51     INFO - Return code: 0
01:18:51  WARNING - Failed to remove C:\slave\test\build on try 2.
01:18:51     INFO - Sleeping 4 seconds...
01:18:55     INFO - Running command: rmdir /S /Q "C:\slave\test\build"
01:18:55     INFO -  C:\slave\test\build\venv\Scripts\mozinstall.exe - Access is denied.
01:18:55     INFO - Return code: 0
01:18:55  WARNING - Failed to remove C:\slave\test\build on try 3.
01:18:55     INFO - Sleeping 6 seconds...
01:19:01     INFO - Running command: rmdir /S /Q "C:\slave\test\build"
01:19:01     INFO -  C:\slave\test\build\venv\Scripts\mozinstall.exe - Access is denied.
01:19:01     INFO - Return code: 0
01:19:01  WARNING - Failed to remove C:\slave\test\build on try 4.
01:19:01     INFO - Sleeping 8 seconds...
01:19:09     INFO - Running command: rmdir /S /Q "C:\slave\test\build"
01:19:09     INFO -  C:\slave\test\build\venv\Scripts\mozinstall.exe - Access is denied.
01:19:09     INFO - Return code: 0
01:19:09  WARNING - Failed to remove C:\slave\test\build on try 5.
01:19:09     INFO - Sleeping 10 seconds...
01:19:19     INFO - Running command: rmdir /S /Q "C:\slave\test\build"
01:19:19     INFO -  C:\slave\test\build\venv\Scripts\mozinstall.exe - Access is denied.
01:19:19     INFO - Return code: 0
01:19:19  WARNING - Failed to remove C:\slave\test\build on try 6.
01:19:19    ERROR - Unable to remove C:\slave\test\build!

seems this is a fallout of the permission problem / uac issue , don't think its critical since the test passed finally but at least good to have the bug on file

Comment 1

[Armen [:armenzg]]

This actually has caused the installation of files down the road to fail:
20:29:51     INFO -  Installing mozinstall.exe script to C:\slave\test\build\venv\Scripts
20:29:51     INFO -  error: C:\slave\test\build\venv\Scripts\mozinstall.exe:

In any case, this happened on #002 because we had a UAC prompt from a previous job that was waiting on "mozinstall -h". I clear the window. We should not hit this anymore.

Nevertheless, I believe it is important for mozharness to stop if it fails to clobber.

Comment 2

[Aki Sasaki (not active)]

To do so, all we have to do is change

http://hg.mozilla.org/build/mozharness/file/6f70eb4b3c0c/mozharness/base/script.py#l745

from

self.rmtree(dirs['abs_work_dir'])

to

self.rmtree(dirs['abs_work_dir'], error_level=FATAL)

Comment 3

[Carsten Book [:Tomcat]]

Attachment: patch

(In reply to Aki Sasaki [:aki] from comment #2)
> To do so, all we have to do is change
> 
> http://hg.mozilla.org/build/mozharness/file/6f70eb4b3c0c/mozharness/base/
> script.py#l745
> 
> from
> 
> self.rmtree(dirs['abs_work_dir'])
> 
> to
> 
> self.rmtree(dirs['abs_work_dir'], error_level=FATAL)

created the patch :)

Comment 4

[Carsten Book [:Tomcat]]

aki, armen: good to push that patch?

Comment 5

[Carsten Book [:Tomcat]]

pushed -> https://hg.mozilla.org/build/mozharness/rev/5fca529f70e7

Comment 6

[Armen [:armenzg]]

This is now live with other changes that I merged to the "production" branch:
http://hg.mozilla.org/build/mozharness/graph

FYI we don't close a bug as FIXED until the change is actually live in production rather than just landed.

For the repoos, mozharness, buildbot-configs and builbotcustom it requires merging changes from the default branch to the their respective production branches.

Other repo changes like puppet, require deploying the change toe the puppet masters and at that point we can close. Nevertheless, we tend to land & deploy almost immediately.

I hope this helps :)

Comment 7

[Carsten Book [:Tomcat]]

(In reply to Armen Zambrano G. [:armenzg] from comment #6)
> This is now live with other changes that I merged to the "production" branch:
> http://hg.mozilla.org/build/mozharness/graph
> 
> FYI we don't close a bug as FIXED until the change is actually live in
> production rather than just landed.
> 

ah ok cool, thanks armen for the pointer, since this is different to other parts was not aware of it, but will follow that way from now on!