Closed Bug 1325348 Opened 7 years ago Closed 7 years ago

mochitest-media jobs failing as we need pywin32 package available

Categories

(Taskcluster :: General, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED
mozilla53

People

(Reporter: jmaher, Assigned: jmaher)

References

Details

Attachments

(1 file)

on the gpu instances here: https://treeherder.mozilla.org/#/jobs?repo=try&revision=eda8d1f9b255ce149b03a92baf15e7992b42308e&filter-tier=1&filter-tier=2&filter-tier=3&filter-searchStr=tc%20mda&group_state=expanded&selectedJob=33311409

we fail a couple of tests which rely upon a ICE server that fails to startup due to missing spanwProcess functionality which is found in pywin32:
11:53:22     INFO -  2199 INFO TEST-START | dom/media/tests/mochitest/test_peerConnection_basicAudioNATRelay.html
11:53:23     INFO -  TEST DEVICES: No test devices found (in media.{audio,video}_loopback_dev, using fake streams.
11:53:23     INFO -  TEST-INFO | started process screenshot
11:53:23     INFO -  Traceback (most recent call last):
11:53:23     INFO -    File "Z:\build\venv\lib\site-packages\twisted\python\log.py", line 84, in callWithLogger
11:53:23     INFO -      return callWithContext({"system": lp}, func, *args, **kw)
11:53:23     INFO -    File "Z:\build\venv\lib\site-packages\twisted\python\log.py", line 69, in callWithContext
11:53:23     INFO -      return context.call({ILogContext: newCtx}, func, *args, **kw)
11:53:23     INFO -    File "Z:\build\venv\lib\site-packages\twisted\python\context.py", line 59, in callWithContext
11:53:23     INFO -      return self.currentContext().callWithContext(ctx, func, *args, **kw)
11:53:23     INFO -    File "Z:\build\venv\lib\site-packages\twisted\python\context.py", line 37, in callWithContext
11:53:23     INFO -      return func(*args,**kw)
11:53:23     INFO -  --- <exception caught here> ---
11:53:23     INFO -    File "Z:\build\venv\lib\site-packages\twisted\internet\selectreactor.py", line 146, in _doReadOrWrite
11:53:23     INFO -      why = getattr(selectable, method)()
11:53:23     INFO -    File "Z:\build\venv\lib\site-packages\twisted\internet\tcp.py", line 460, in doRead
11:53:23     INFO -      return self.protocol.dataReceived(data)
11:53:23     INFO -    File "Z:\build\venv\lib\site-packages\txws.py", line 629, in dataReceived
11:53:23     INFO -      self.parseFrames()
11:53:23     INFO -    File "Z:\build\venv\lib\site-packages\txws.py", line 469, in parseFrames
11:53:23     INFO -      ProtocolWrapper.dataReceived(self, data)
11:53:23     INFO -    File "Z:\build\venv\lib\site-packages\twisted\protocols\policies.py", line 83, in dataReceived
11:53:23     INFO -      self.wrappedProtocol.dataReceived(data)
11:53:23     INFO -    File "websocketprocessbridge\websocketprocessbridge.py", line 63, in dataReceived
11:53:23     INFO -      env=os.environ)
11:53:23     INFO -    File "Z:\build\venv\lib\site-packages\twisted\internet\posixbase.py", line 335, in spawnProcess
11:53:23     INFO -      raise NotImplementedError, "spawnProcess not available since pywin32 is not installed."
11:53:23     INFO -  exceptions.NotImplementedError: spawnProcess not available since pywin32 is not installed.
11:53:23     INFO -  Traceback (most recent call last):
11:53:23     INFO -    File "websocketprocessbridge\websocketprocessbridge.py", line 98, in <module>
11:53:23     INFO -      reactor.run()
11:53:23     INFO -    File "Z:\build\venv\lib\site-packages\twisted\internet\base.py", line 1158, in run
11:53:23     INFO -      self.mainLoop()
11:53:23     INFO -    File "Z:\build\venv\lib\site-packages\twisted\internet\base.py", line 1170, in mainLoop
11:53:23     INFO -      self.doIteration(t)
11:53:23     INFO -    File "Z:\build\venv\lib\site-packages\twisted\internet\selectreactor.py", line 140, in doSelect
11:53:23     INFO -      _logrun(selectable, _drdw, selectable, method, dict)
11:53:23     INFO -  --- <exception caught here> ---
11:53:23     INFO -    File "Z:\build\venv\lib\site-packages\twisted\python\log.py", line 84, in callWithLogger
11:53:23     INFO -      return callWithContext({"system": lp}, func, *args, **kw)
11:53:23     INFO -    File "Z:\build\venv\lib\site-packages\twisted\python\log.py", line 69, in callWithContext
11:53:23     INFO -      return context.call({ILogContext: newCtx}, func, *args, **kw)
11:53:23     INFO -    File "Z:\build\venv\lib\site-packages\twisted\python\context.py", line 59, in callWithContext
11:53:23     INFO -      return self.currentContext().callWithContext(ctx, func, *args, **kw)
11:53:23     INFO -    File "Z:\build\venv\lib\site-packages\twisted\python\context.py", line 37, in callWithContext
11:53:23     INFO -      return func(*args,**kw)
11:53:23     INFO -    File "Z:\build\venv\lib\site-packages\twisted\internet\selectreactor.py", line 156, in _doReadOrWrite
11:53:23     INFO -      self._disconnectSelectable(selectable, why, method=="doRead")
11:53:23     INFO -    File "Z:\build\venv\lib\site-packages\twisted\internet\posixbase.py", line 256, in _disconnectSelectable
11:53:23     INFO -      selectable.connectionLost(failure.Failure(why))
11:53:23     INFO -    File "Z:\build\venv\lib\site-packages\twisted\internet\tcp.py", line 519, in connectionLost
11:53:23     INFO -      protocol.connectionLost(reason)
11:53:23     INFO -    File "Z:\build\venv\lib\site-packages\twisted\protocols\policies.py", line 87, in connectionLost
11:53:23     INFO -      self.wrappedProtocol.connectionLost(reason)
11:53:23     INFO -    File "websocketprocessbridge\websocketprocessbridge.py", line 67, in connectionLost
11:53:23     INFO -      self.processSide.socketGone()
11:53:23     INFO -    File "websocketprocessbridge\websocketprocessbridge.py", line 43, in socketGone
11:53:23     INFO -      self.transport.loseConnection()
11:53:23     INFO -  exceptions.AttributeError: 'NoneType' object has no attribute 'loseConnection'
11:53:23     INFO -  TEST-INFO | screenshot: exit 0
11:53:23     INFO -  Buffered messages logged at 11:53:23
11:53:23     INFO -  2200 INFO TEST-PASS | dom/media/tests/mochitest/test_peerConnection_basicAudioNATRelay.html | A valid string reason is expected
11:53:23     INFO -  2201 INFO TEST-PASS | dom/media/tests/mochitest/test_peerConnection_basicAudioNATRelay.html | Reason cannot be empty
11:53:23     INFO -  2202 INFO websocket/process bridge open, starting ICE Server...
11:53:23     INFO -  2203 INFO ICE Server websocket closed
11:53:23     INFO -  Buffered messages finished
11:53:23    ERROR -  2204 INFO TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/test_peerConnection_basicAudioNATRelay.html | Error executing test: ICE Server gone before getting configuration
11:53:23     INFO -      runTestWhenReady/<@dom/media/tests/mochitest/head.js:372:7
11:53:23     INFO -      Async*runTestWhenReady@dom/media/tests/mochitest/head.js:370:10
11:53:23     INFO -      promise callback*runNetworkTest@dom/media/tests/mochitest/pc.js:1873:10
11:53:23     INFO -      @dom/media/tests/mochitest/test_peerConnection_basicAudioNATRelay.html:16:3
11:53:23     INFO -  MEMORY STAT | vsize 437MB | vsizeMaxContiguous 911MB | residentFast 94MB | heapAllocated 14MB
11:53:23     INFO -  2205 INFO TEST-OK | dom/media/tests/mochitest/test_peerConnection_basicAudioNATRelay.html | took 258ms


pywin32 is one of those odd python packages which requires a .exe to be installed as there are compiled components.  Possibly there is a newer method available for pip installation.
i did recently add the v 219 wheels to pypi.pub.build.mozilla.org. they are here:

http://pypi.pub.build.mozilla.org/pub/pypiwin32-219-cp27-none-win32.whl
http://pypi.pub.build.mozilla.org/pub/pypiwin32-219-cp27-none-win_amd64.whl

exes for v 216 were already there:

http://pypi.pub.build.mozilla.org/pub/pywin32-216.win32-py2.7.exe
http://pypi.pub.build.mozilla.org/pub/pywin32-216.win-amd64-py2.7.exe

note that the wheels are from the newer, renamed package "pypiwin32" rather than "pywin32".

the tests would have to be updated to include a `pip install pypiwin32 --find-links http://pypi.pub.build.mozilla.org/pub --trusted-host pypi.pub.build.mozilla.org` (or similar) command to make use of these.

maybe it's as simple as adding `"virtualenv_modules": ['pypiwin32']` to the mozharness config (similar to https://dxr.mozilla.org/mozilla-central/rev/10a5e2fc24e4e24fbe115fb190bcc28df4627ee2/testing/mozharness/configs/talos/windows_config.py#19)
ok, this worked out and we are green with that patch!
Attached patch pywin32.patchSplinter Review
Assignee: nobody → jmaher
Status: NEW → ASSIGNED
Attachment #8821159 - Flags: review?(rthijssen)
Attachment #8821159 - Flags: review?(rthijssen) → review+
Pushed by jmaher@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/8dff43d455b9
mochitest-media jobs failing as we need pywin32 package available. r=grenade
https://hg.mozilla.org/mozilla-central/rev/8dff43d455b9
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla53
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: