798170 - Replace all copies of _dumbwin32proc.py

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: CIDuty, task, P2)

Description

[Chris Cooper [:coop] (he/him)]

While doing work on the new WinXP ref platform (bug 770579), I discovered that we're still replacing the copy of _dumbwin32proc.py that lives under C:\mozilla-build\python25\Lib\site-packages\twisted\internet. Another copy of that module als o gets installed under C:\mozilla-build\buildbotve\python25\Lib\site-packages\twisted\internet. I'm uncertain whether this other copy is ever called, but we should probably be replacing both version to be safe.

https://hg.mozilla.org/build/opsi-package-sources/file/e55c081cb8cf/twisted_dumbwin32proc/CLIENT_DATA/twisted_dumbwin32proc.ins

Comment 1

[Armen [:armenzg]]

Probably safe to compare the .bak file with the other one you want to replace:
C:\mozilla-build\python25\Lib\site-packages\twisted\internet\_dumbwin32proc.py.bak
C:\mozilla-build\buildbotve\python25\Lib\site-packages\twisted\internet\_dumbwin32proc.py

Great to see that you caught this!

Comment 2

[Chris Cooper [:coop] (he/him)]

Attachment: Replace the version of _dumbwin32proc.py under buildbotve also

Comment 3

[Chris Cooper [:coop] (he/him)]

Comment on attachment 688766 [details] [diff] [review]
Replace the version of _dumbwin32proc.py under buildbotve also

https://hg.mozilla.org/build/opsi-package-sources/rev/ac29b3f0987c

Comment 4

[Chris Cooper [:coop] (he/him)]

I've marked all the XP slaves to re-install this package on next boot. I'll check on them tomorrow to make sure I didn't inadvertently tank the whole pool.

Comment 5

[Armen [:armenzg]]

We need to work on this again:
C:\Documents and Settings\cltbld>dir C:\mozilla-build\buildbotve\python25\Lib\site-packages\twisted\internet\_dumbwin32proc.py*
The system cannot find the path specified.

We got the path wrong:
C:\mozilla-build\buildbotve\Lib\site-packages\twisted\internet
versus
C:\mozilla-build\buildbotve\python25\Lib\site-packages\twisted\internet

This is what I did on a w7 and an xp slave:
cd C:\mozilla-build\buildbotve\Lib\site-packages\twisted\internet
del _dumbwin32proc.pyc
move _dumbwin32proc.py _dumbwin32proc.py.bak
wget -q --no-check-certificate https://hg.mozilla.org/build/opsi-package-sources/raw-file/e55c081cb8cf/twisted_dumbwin32proc/CLIENT_DATA/_dumbwin32proc.py

I assume we need to do the same on win64 and win8 slaves.

#########################################
diff -pU 8 _dumbwin32proc.py _dumbwin32proc.py.bak
--- _dumbwin32proc.py   Wed Mar 13 10:41:13 2013
+++ _dumbwin32proc.py.bak       Thu Sep 01 08:31:04 2011
@@ -233,17 +233,18 @@ class Process(_pollingfile._PollingTimer

         self._addPollableResource(_Reaper(self))


     def signalProcess(self, signalID):
         if self.pid is None:
             raise error.ProcessExitedAlready()
         if signalID in ("INT", "TERM", "KILL"):
-            os.popen('taskkill /T /F /PID %s' % self.pid)
+            win32process.TerminateProcess(self.hProcess, 1)
+

     def _getReason(self, status):
         if status == 0:
             return error.ProcessDone(status)
         return error.ProcessTerminated(status)


     def write(self, data):

Comment 6

[Armen [:armenzg]]

Attachment: _dumbwin32proc.py

Comment 7

[Chris Cooper [:coop] (he/him)]

Comment on attachment 724494 [details] [diff] [review]
_dumbwin32proc.py

Review of attachment 724494 [details] [diff] [review]:
-----------------------------------------------------------------

Did you check to make sure there's only one copy of _dumbwin32proc.py on each slave? We should replace it everywhere it exists to be safe.

Comment 8

[Armen [:armenzg]]

(In reply to Chris Cooper [:coop] from comment #7)
> Comment on attachment 724494 [details] [diff] [review]
> _dumbwin32proc.py
> 
> Review of attachment 724494 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> Did you check to make sure there's only one copy of _dumbwin32proc.py on
> each slave? We should replace it everywhere it exists to be safe.

For both win7 and winxp we only have it in here:
C:\mozilla-build\python25\Lib\site-packages\twisted\internet
C:\mozilla-build\buildbotve\Lib\site-packages\twisted\internet

On win64:
C:\mozilla-build\python\Lib\site-packages\twisted\internet
Not under C:\mozilla-build\python27
C:\mozilla-build\buildbotve\Lib\site-packages\twisted\internet

On win8:
Not under C:\mozilla-build\python27
C:\mozilla-build\buildbotve\Lib\site-packages\twisted\internet

AFAIU, we only want to replace the _dumbwin32proc.py that is used by buildbot.

Comment 9

[Armen [:armenzg]]

Comment on attachment 724494 [details] [diff] [review]
_dumbwin32proc.py

Deployed.

I have marked machines xp-0[01-10] to receive this package.

Comment 10

[Armen [:armenzg]]

Attachment: deploy_dumbwin32proc.bat

Comment 11

[Armen [:armenzg]]

This can be deployed like this:
C:\mozilla-build\wget\wget.exe -Odeploy_dumbwin32proc.bat https://bugzilla.mozilla.org/attachment.cgi?id=725099
deploy_dumbwin32proc.bat

I have deployed this change to:
* talos-r3-w7-012
* t-w864-ix-012
* w64-ix-slave12

I will review it tomorrow.

Comment 12

[Armen [:armenzg]]

I marked all XP slaves to get this change.
I have deployed this change to all win7 slaves as well besides:
* talos-r3-w7-006
* talos-r3-w7-011

I will wait for win64 slaves on Monday and for win8 machines for when they have ssh.

Comment 13

[Armen [:armenzg]]

All remaining Windows 7 machines have already got done.

For win64 machines as Administrator (all done):
C:\mozilla-build\wget\wget.exe -q -Odeploy_dumbwin32proc.bat https://bugzilla.mozilla.org/attachment.cgi?id=725099
deploy_dumbwin32proc.bat

Win8 is left. I will see if I can ask Q to deploy it through GPO.

Comment 14

[Armen [:armenzg]]

Documented:
https://wiki.mozilla.org/ReferencePlatforms/Test/Win7#Deploy_dumbwin32proc.py
https://wiki.mozilla.org/ReferencePlatforms/Win64#Deploy_dumbwin32proc.py

WinXP is taking care by OPSI.
Win64 and Win8 are taken care by bug 853609.

Comment 15

[Armen [:armenzg]]

Both Win64 TS and Win8 GPO got done by Q.

I adjusted the "deploy new _dumbwin32proc" section:
https://wiki.mozilla.org/ReferencePlatforms/Win64#Deploy_dumbwin32proc.py

Comment 16

[Dustin J. Mitchell [:dustin] (he/him)]

For the record, I would have liked to see this done with either a monkeypatch in the Buildbot source or as part of the Buildbot install process, rather than a later patch.  But I'm glad it's fixed at any rate.

Another Windows user at the PyCon sprints was talking about this problem.  Yet nobody can come up with a general solution :(