Closed
Bug 1007230
Opened 10 years ago
Closed 10 years ago
Timeouts during pip install ("Error running install of package, /builds/slave/test/build/venv/bin/pip install --download-cache /builds/...", "Error running install of package, /builds/slave/talos-slave/test/build/venv/bin/pip install --download-cache...")
Categories
(Release Engineering :: General, defect)
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: dholbert, Assigned: pmoore)
References
()
Details
(Keywords: intermittent-failure)
Attachments
(2 files, 4 obsolete files)
8.95 KB,
application/gzip
|
Details | |
6.70 KB,
patch
|
jlund
:
review+
pmoore
:
checked-in+
|
Details | Diff | Splinter Review |
https://tbpl.mozilla.org/php/getParsedLog.php?id=39191591&tree=Try { 02:21:47 INFO - Downloading/unpacking mock 02:22:03 INFO - Exception: 02:22:03 ERROR - Traceback (most recent call last): 02:22:03 INFO - File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/pip-0.8.2-py2.7.egg/pip/basecommand.py", line 130, in main 02:22:03 INFO - self.run(options, args) 02:22:03 INFO - File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/pip-0.8.2-py2.7.egg/pip/commands/install.py", line 223, in run 02:22:03 INFO - requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle) 02:22:03 INFO - File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/pip-0.8.2-py2.7.egg/pip/req.py", line 917, in prepare_files 02:22:03 INFO - self.unpack_url(url, location, self.is_download) 02:22:03 INFO - File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/pip-0.8.2-py2.7.egg/pip/req.py", line 1023, in unpack_url 02:22:03 INFO - return unpack_http_url(link, location, self.download_cache, only_download) 02:22:03 INFO - File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/pip-0.8.2-py2.7.egg/pip/download.py", line 442, in unpack_http_url 02:22:03 INFO - download_hash = _download_url(resp, link, temp_location) 02:22:03 INFO - File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/pip-0.8.2-py2.7.egg/pip/download.py", line 367, in _download_url 02:22:03 INFO - chunk = resp.read(4096) 02:22:03 INFO - File "/usr/lib/python2.7/socket.py", line 380, in read 02:22:03 INFO - data = self._sock.recv(left) 02:22:03 INFO - File "/usr/lib/python2.7/httplib.py", line 561, in read 02:22:03 INFO - s = self.fp.read(amt) 02:22:03 INFO - File "/usr/lib/python2.7/socket.py", line 380, in read 02:22:03 INFO - data = self._sock.recv(left) 02:22:03 INFO - timeout: timed out 02:22:03 INFO - Storing complete log in /home/cltbld/.pip/pip.log 02:22:03 ERROR - Return code: 2 02:22:03 FATAL - Error running install of package, /builds/slave/test/build/venv/bin/pip install --download-cache /builds/slave/test/build/venv/cache --no-index --find-links http://pypi.pvt.build.mozilla.org/pub --find-links http://pypi.pub.build.mozilla.org/pub mock! 02:22:03 FATAL - Running post_fatal callback... 02:22:03 FATAL - Exiting -1 }
Reporter | ||
Comment 1•10 years ago
|
||
Reporter | ||
Comment 2•10 years ago
|
||
This is the red "bc1" on this Try push: https://tbpl.mozilla.org/?tree=Try&rev=94b409a1da51 And here's the header info from the log, FWIW: Ubuntu ASAN VM 12.04 x64 try opt test mochitest-browser-chrome-1 on 2014-05-07 02:18:48 PDT for push 94b409a1da51 slave: tst-linux64-spot-004
Comment 3•10 years ago
|
||
Bug 1003729 will fix the noisy traceback & I (maybe?) give us a retry for free.
Depends on: 1003729
Comment 4•10 years ago
|
||
This is an automation bug that perhaps could be fixed by free with the dependent bug.
Component: Buildduty → General Automation
QA Contact: bugspam.Callek → catlee
Comment 5•10 years ago
|
||
Tweaking summary to make it TBPL parsable.
Summary: Test runner red due to exception (timeout?) while "Downloading/unpacking mock" → Timeouts during pip install ("Error running install of package, /builds/slave/test/build/venv/bin/pip install --download-cache /builds/slave/test/build/venv/cache --no-index --find-links http://pypi.pvt.build.mozilla.org/pub --find-links")
Updated•10 years ago
|
Keywords: intermittent-failure
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment 71•10 years ago
|
||
Error running install of package, /builds/slave/talos-slave/test/build/venv/bin/pip install --download-cache /builds/slave/talos-slave/test/build/venv/cache --no-index --find-links http://pypi.pvt.build.mozilla.org/pub --find-links http://pypi.pub.build.mozilla.org/pub mock! https://tbpl.mozilla.org/php/getParsedLog.php?id=41506670&tree=Mozilla-Inbound
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment 98•10 years ago
|
||
Rail, is this something that falls into your realm?
Comment 99•10 years ago
|
||
(In reply to Ryan VanderMeulen [:RyanVM UTC-4] from comment #98) > Rail, is this something that falls into your realm? Bug 1003729 would get us most if not all of the way there with this :-)
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment 120•10 years ago
|
||
(In reply to Ed Morley [:edmorley UTC+0] from comment #99) > Bug 1003729 would get us most if not all of the way there with this :-) Coop, this is currently #8 on OrangeFactor. Can we please find someone who can work on this soon?
Flags: needinfo?(coop)
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment 144•10 years ago
|
||
From comment 143: 20:16:03 INFO - File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/pip-0.8.2-py2.7.egg/pip/download.py", line 84, in __call__ Bug 1003729 had already landed, but hopefully it's just that we needed to clobber to regenerate the virtualenvs; which I've now done on all trunk trees.
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment 164•10 years ago
|
||
Windows: https://tbpl.mozilla.org/php/getParsedLog.php?id=42134114&tree=Mozilla-Central
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment 177•10 years ago
|
||
(In reply to TBPL Robot from comment #176) > KWierso > https://tbpl.mozilla.org/php/getParsedLog.php?id=42182039&tree=B2g-Inbound > b2g_ubuntu64_vm b2g-inbound opt test gaia-linter on 2014-06-20 16:49:21 > revision: 59fda077a2ab > slave: tst-linux64-spot-271 > > Traceback (most recent call last): > Return code: 2 > Error running install of package, /builds/slave/test/build/venv/bin/pip > install --download-cache /builds/slave/test/build/venv/cache --no-index > --find-links http://pypi.pvt.build.mozilla.org/pub --find-links > http://pypi.pub.build.mozilla.org/pub closure_linter==2.3.13! > Running post_fatal callback... > Exiting -1 > ImportError: No module named mozsystemmonitor.resourcemonitor note: this one happened with pip 1.5.5 16:58:32 ERROR - Traceback (most recent call last): 16:58:32 INFO - File "/builds/slave/test/build/venv/local/lib/python2.7/site-packages/pip-1.5.5-py2.7.egg/pip/basecommand.py", line 122, in main
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment 179•10 years ago
|
||
c#178 was also pip 1.5.5
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment 644•10 years ago
|
||
Tweaking summary now that bug 1042679 has landed & means we don't need to include as much of the failure line, so that I can include another variant to save manual starring of it.
Summary: Timeouts during pip install ("Error running install of package, /builds/slave/test/build/venv/bin/pip install --download-cache /builds/slave/test/build/venv/cache --no-index --find-links http://pypi.pvt.build.mozilla.org/pub --find-links") → Timeouts during pip install ("Error running install of package, /builds/slave/test/build/venv/bin/pip install --download-cache /builds/...", "Error running install of package, /builds/slave/talos-slave/test/build/venv/bin/pip install --download-cache...")
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Assignee | ||
Comment 801•10 years ago
|
||
Can we not simply add some retries in there for pip install if it timesout? Also, have we investigated if this is a performance problem on our pypi server? NI'ing dustin in case we have anything (e.g. logs) under http://pypi.pub.build.mozilla.org/ to help us understand why we are getting timeouts. However, I suspect retry attempts at the pip installs could considerably help...
Flags: needinfo?(dustin)
Comment 802•10 years ago
|
||
We could also try raising the timeout: http://pip.readthedocs.org/en/latest/user_guide.html#configuration eg: pip --timeout=60 install ... pip has a default of 15 seconds, which seems a little short: https://github.com/pypa/pip/blob/d420497b32c11defc3f341a8df180780809a337c/pip/cmdoptions.py#L127 timeout = OptionMaker( '--timeout', '--default-timeout', metavar='sec', dest='timeout', type='float', default=15, help='Set the socket timeout (default %default seconds).')
Assignee | ||
Comment 803•10 years ago
|
||
This patch worked fine for me testing locally. Note, I added the check if "--timeout" already exists as an argument, even though I don't think it is possible, because I envisage that at some point you might be able to pass arbitrary pip options in from the config, and if it was explicitly set already, I wouldn't want to overrule the existing value. We can also remove the check - I just thought it was a good safety net to have in place. Note this would also increase people's default timeouts when working locally - not just our infra. I guess this is ok (since maybe they also hit our pypi server) - I figured this was simpler than updating *all* configs we have to include this increased default timeout. Let me know what you think!
Attachment #8469222 -
Flags: review?(jlund)
Comment 805•10 years ago
|
||
Agreed -- retry is the best fix here. The web server is just a dead-simple serve-files-from-NFS vhost, via a Zeus load balancer. Mozilla pushes a *lot* of traffic through both the NetApp and Zeus, so if there were issues with either one we'd know about it. We could certainly ask someone in webops if the ZLB can tell us anything. I suspect, instead, that there's something in the HTTP TCP connection between the client and the load balancer. Packet loss happens, and TCP's supposed to deal with it smoothly, but sometimes dealing with it can take a while as timers expire and retransmits occur. If we wanted to figure out what's going on here, we'd need a way to replicate it reliably within a few seconds, and then gather tcpdump data from both ends of the connection. Since our slaves seem to have really low cache rates, we handle *huge* amounts of traffic for pypi, talos-remote, talos-bundles, and tooltool. So capturing traffic on the webhead for long enough (several hours) to see an instance of this would result in a tcpdump file too big to store -- and we still wouldn't have a dump of the slave side of the connection.
Flags: needinfo?(dustin)
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment 844•10 years ago
|
||
(In reply to Dustin J. Mitchell [:dustin] from comment #805) > Agreed -- retry is the best fix here. Pete: I might advocate a hybrid approach here. Grab/copy/import the retry logic from mozharness/base/script.py into mozharness/base/python.py *and* increase the pip timeout by some factor, say 30s if the current timeout is 15s.
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment 848•10 years ago
|
||
Comment on attachment 8469222 [details] [diff] [review] bug1007230_mozharness_v1.patch Review of attachment 8469222 [details] [diff] [review]: ----------------------------------------------------------------- nice, I agree it's a good call to ensure --timeout isn't already passed. You could also do a say `--timeout %s' % c.get('pip_timeout', 120)` and that way you would be that person who not only, as you said, 'envisioned being able to add this to the config' but actually made it so. Up to you, I'll roll with your preference. :) sounds like dustin and coop are in an agreement for wrapping a retry around this. I am leaning that way as well as that is how we tend to deal with a lot of the mozharness networky stuff. No import needed for retry, it will be part of self when this mixin is used. We could also avail of retry's `good_statuses`, `error_level`, and `error_message` to clean up the the current semi clunky way we handle `optional` and the resulting return code (bonus!) one more note, since retry defaults to 5 attempts and a timeout of 60s where the timeout grows by a factor of timeout*2, we may wish to play with some of the dials. Feel free to change the pip timeout, and retry's timeout/attempts as letting this go too long could result in an overall timeout for the script. does this make sense? obviously, feel free to push back or ping me for questions. I'm going to r- to clear my queue, will re-eval once a new patch is put forward or we discuss this more :)
Attachment #8469222 -
Flags: review?(jlund) → review-
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Assignee | ||
Comment 916•10 years ago
|
||
Hey Jordan, Thanks for your feedback - I've incorporated your suggestions into this new patch. Let me know what you think. Thanks, Pete
Attachment #8469222 -
Attachment is obsolete: true
Attachment #8471002 -
Flags: review?(jlund)
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment 928•10 years ago
|
||
Comment on attachment 8471002 [details] [diff] [review] bug1007230_mozharness_v2.patch Review of attachment 8471002 [details] [diff] [review]: ----------------------------------------------------------------- lgtm. see two questions below. I'll leave them to your discretion. is this new patch tested on dev-master or ash? it be nice to confirm this won't break anything. landing on default and testing on Cedar is fine if you're confident :) ::: mozharness/base/python.py @@ +210,5 @@ > if virtualenv_cache_dir: > command += ["--download-cache", virtualenv_cache_dir] > + # To avoid timeouts with our pypi server, increase default timeout: > + # https://bugzilla.mozilla.org/show_bug.cgi?id=1007230#c802 > + if '--timeout' not in command: Q1. is this condition still needed? Is it fair to say if we want to adjust the timeout, devs should use pip_timeout? Keeping this condition feels like we would be covering their mistake if they added another '--timeou'. open to your thoughts. @@ +269,5 @@ > + 'error_list': VirtualenvErrorList, > + 'success_codes': success_codes, > + 'cwd': cwd, > + # Need to throw exception for retry mechanism to work > + 'throw_exception': True, FYI - good_statuses should make this work too see: http://mxr.mozilla.org/build/source/ash-mozharness/mozharness/base/script.py#528 @@ +276,5 @@ > + attempts = 1 > + else: > + # None will cause default value to be used > + attempts = None > + if self.retry(self.run_command, args=[command,], attempts=attempts, kwargs=kwargs) != 0: q2. if this repeatedly fails, I think this will last for 25min. do we need that much time or should we scale back these dials: attempts, sleeptime, and pip_timeout?
Attachment #8471002 -
Flags: review?(jlund) → review+
Comment hidden (Legacy TBPL/Treeherder Robot) |
Assignee | ||
Comment 930•10 years ago
|
||
(In reply to Jordan Lund (:jlund) from comment #928) > Comment on attachment 8471002 [details] [diff] [review] > bug1007230_mozharness_v2.patch > > Review of attachment 8471002 [details] [diff] [review]: > ----------------------------------------------------------------- > > lgtm. see two questions below. I'll leave them to your discretion. > > is this new patch tested on dev-master or ash? it be nice to confirm this > won't break anything. landing on default and testing on Cedar is fine if > you're confident :) I've tested locally, and there is no buildbot integration etc, so I'm confident that the testing I did locally was sufficient. I tested installing non-existent packages together with existing packages, I tested the config option added (pip_timeout) and I tested installing optional and non-optional packages (optional packages don't retry). It is pretty quiet in the European mornings, so I'm confident I can land it in a quiet period, and if there are any problems, roll it back before it gets busy, which I don't expect to do as I tested thoroughly locally. I think spending more time on this issue might result in little return value for effort involved. > > ::: mozharness/base/python.py > @@ +210,5 @@ > > if virtualenv_cache_dir: > > command += ["--download-cache", virtualenv_cache_dir] > > + # To avoid timeouts with our pypi server, increase default timeout: > > + # https://bugzilla.mozilla.org/show_bug.cgi?id=1007230#c802 > > + if '--timeout' not in command: > > Q1. is this condition still needed? Is it fair to say if we want to adjust > the timeout, devs should use pip_timeout? Keeping this condition feels like > we would be covering their mistake if they added another '--timeou'. open to > your thoughts. > I think this option does no harm, and acts as a safety measure in case e.g. a timeout is specified directly in the pip command or someone does some other refactor which introduces the possibility a timeout gets added unexpectedly. I could of course pull it out, but I don't see a lot of benefit that would be added (I think the code would be just as readable). If it is ok with you, I prefer to leave it in for now to avoid more patching/testing cycles, and also we've hit almost 1000 issues in production now (the number of comments in this bug by TBPL Robot) and a further iteration costs a full day (due to us being in non-overlapping time zones) so adds to the amount of fallout we'll have for this issue not being solved (also on PTO tomorrow). We can of course raise a separate bug for this optimisation if you like, but I'd prefer not to delay the rollout of this fix unless if possible. > @@ +269,5 @@ > > + 'error_list': VirtualenvErrorList, > > + 'success_codes': success_codes, > > + 'cwd': cwd, > > + # Need to throw exception for retry mechanism to work > > + 'throw_exception': True, > > FYI - good_statuses should make this work too > > see: > http://mxr.mozilla.org/build/source/ash-mozharness/mozharness/base/script. > py#528 > Thanks. From reading the code, the effect of using 'good_statuses': [0,] or 'throw_exception': True seem to be equivalent (to cause a retry), so since this is tested and working I'll leave it as it is for now (I don't see a benefit to switching if they both do the same). > @@ +276,5 @@ > > + attempts = 1 > > + else: > > + # None will cause default value to be used > > + attempts = None > > + if self.retry(self.run_command, args=[command,], attempts=attempts, kwargs=kwargs) != 0: > > q2. if this repeatedly fails, I think this will last for 25min. do we need > that much time or should we scale back these dials: attempts, sleeptime, and > pip_timeout? I think this is what we want. 25 mins out of a compute time of 10 days (or however much it is these days) I don't see as too significant - and also, if there is a problem, and a package cannot be installed, I think it is good to hold off as long as reasonably possible until the http service is functioning again, to avoid needing to retrigger, which also is a waste of time for steps that have already taken place. The problem is, if after 25 mins the pypi server is not available, we don't save any time by killing the job. We can only run the job successfully when the pypi server is available, so we haven't "lost" time by waiting 25 mins (if you see what I mean). If pypi goes down, we should get nagios notifications anyway to say it is unavailable, so this is just a way to give us 25 mins to get that service back up and running again, before all jobs start to die. When the jobs start dying, we have more fallout than if they patiently wait for pypi to be back up. So I think I'm happy with it. I'm going to check with Ed Morley if he's happy for me to roll out, in light of my comments here - and if he gives the green light, I'll roll out this morning, and keep a close eye on things. Thanks Jordan for taking the time to review the patch in detail - very much appreciated. :) Pete
Assignee | ||
Comment 931•10 years ago
|
||
Ok Ed's given the thumbs up on irc #releng - so rolling out now...
Comment 932•10 years ago
|
||
Agree with the logic in comment 930 - and either way we can always tweak this slightly in a followup - would just be good to stop the failures in the meantime :-)
Assignee | ||
Comment 933•10 years ago
|
||
Comment on attachment 8471002 [details] [diff] [review] bug1007230_mozharness_v2.patch Checked in and merged to production: default: https://hg.mozilla.org/build/mozharness/rev/b1a11c100d17 production: https://hg.mozilla.org/build/mozharness/rev/249a646054fc
Attachment #8471002 -
Flags: checked-in+
Assignee | ||
Comment 934•10 years ago
|
||
Backed out (In reply to Pete Moore [:pete][:pmoore] from comment #930) > ... and if there are any problems, roll it back before it gets busy, which I don't > expect to do as I tested thoroughly locally. what a foolish thing to say! From https://tbpl.mozilla.org/php/getParsedLog.php?id=45734024&full=1&branch=try: 01:36:23 INFO - ##### 01:36:23 INFO - ##### Running create-virtualenv step. 01:36:23 INFO - ##### 01:36:23 INFO - Running pre-action listener: _install_mozbase 01:36:23 INFO - Running pre-action listener: _pre_create_virtualenv 01:36:23 INFO - Running pre-action listener: _resource_record_pre_action 01:36:23 INFO - Running main action method: create_virtualenv 01:36:23 INFO - Creating virtualenv /builds/slave/test/build/venv 01:36:23 INFO - Running command: ['/tools/buildbot/bin/python', '/tools/misc-python/virtualenv.py', '--no-site-packages', '--distribute', '/builds/slave/test/build/venv'] in /builds/slave/test/build 01:36:23 INFO - Copy/paste: /tools/buildbot/bin/python /tools/misc-python/virtualenv.py --no-site-packages --distribute /builds/slave/test/build/venv 01:36:23 INFO - The --no-site-packages flag is deprecated; it is now the default behavior. 01:36:23 INFO - Using real prefix '/usr' 01:36:23 INFO - New python executable in /builds/slave/test/build/venv/bin/python 01:36:24 INFO - Installing distribute.............................................................................................................................................................................................done. 01:36:28 INFO - Installing pip.................done. 01:36:28 INFO - Return code: 0 01:36:28 INFO - Installing psutil==0.7.1 into virtualenv /builds/slave/test/build/venv 01:36:28 INFO - retry: Calling <bound method DesktopUnittest.run_command of <__main__.DesktopUnittest object at 0xa4a1cec>> with args: [['/builds/slave/test/build/venv/bin/pip', 'install', '--download-cache', '/builds/slave/test/build/venv/cache', '--timeout', 120, '--no-index', '--find-links', 'http://pypi.pvt.build.mozilla.org/pub', '--find-links', 'http://pypi.pub.build.mozilla.org/pub', 'psutil==0.7.1']], kwargs: {'error_list': [{'substr': 'not found or a compiler error:', 'level': 'warning'}, {'regex': <_sre.SRE_Pattern object at 0xb72fe808>, 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0xb72d9e30>, 'level': 'warning'}, {'regex': <_sre.SRE_Pattern object at 0xa34a850>, 'level': 'debug'}, {'substr': 'command not found', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0xa346be8>, 'level': 'warning'}, {'substr': 'Traceback (most recent call last)', 'level': 'error'}, {'substr': 'SyntaxError: ', 'level': 'error'}, {'substr': 'TypeError: ', 'level': 'error'}, {'substr': 'NameError: ', 'level': 'error'}, {'substr': 'ZeroDivisionError: ', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0xa33b280>, 'level': 'critical'}, {'regex': <_sre.SRE_Pattern object at 0xa33cf60>, 'level': 'critical'}], 'cwd': '/builds/slave/test/build', 'throw_exception': True, 'success_codes': [0, 1]}, attempt #1 01:36:28 INFO - Running command: ['/builds/slave/test/build/venv/bin/pip', 'install', '--download-cache', '/builds/slave/test/build/venv/cache', '--timeout', 120, '--no-index', '--find-links', 'http://pypi.pvt.build.mozilla.org/pub', '--find-links', 'http://pypi.pub.build.mozilla.org/pub', 'psutil==0.7.1'] in /builds/slave/test/build 01:36:28 ERROR - <bound method DesktopUnittest.run_command of <__main__.DesktopUnittest object at 0xa4a1cec>> failed after 1 tries! 01:36:28 ERROR - Caught exception: argument of type 'int' is not iterable 01:36:28 INFO - Running post-action listener: _resource_record_post_action 01:36:28 INFO - Running post-action listener: _start_resource_monitoring 01:36:28 WARNING - Unable to start resource monitor: Traceback (most recent call last): 01:36:28 WARNING - File "/builds/slave/test/scripts/mozharness/base/python.py", line 465, in _start_resource_monitoring 01:36:28 WARNING - from mozsystemmonitor.resourcemonitor import SystemResourceMonitor 01:36:28 WARNING - ImportError: No module named mozsystemmonitor.resourcemonitor 01:36:28 FATAL - Uncaught exception: Traceback (most recent call last): 01:36:28 FATAL - File "/builds/slave/test/scripts/mozharness/base/script.py", line 1246, in run 01:36:28 FATAL - self.run_action(action) 01:36:28 FATAL - File "/builds/slave/test/scripts/mozharness/base/script.py", line 1188, in run_action 01:36:28 FATAL - self._possibly_run_method(method_name, error_if_missing=True) 01:36:28 FATAL - File "/builds/slave/test/scripts/mozharness/base/script.py", line 1129, in _possibly_run_method 01:36:28 FATAL - return getattr(self, method_name)() 01:36:28 FATAL - File "/builds/slave/test/scripts/mozharness/base/python.py", line 417, in create_virtualenv 01:36:28 FATAL - optional=optional, editable=editable 01:36:28 FATAL - File "/builds/slave/test/scripts/mozharness/base/python.py", line 283, in install_module 01:36:28 FATAL - ' '.join(command)) 01:36:28 FATAL - TypeError: sequence item 5: expected string, int found 01:36:28 FATAL - Running post_fatal callback... 01:36:28 FATAL - Exiting -1 01:36:28 INFO - Running post-run listener: _resource_record_post_run 01:36:28 INFO - Running post-run listener: _upload_blobber_files 01:36:28 INFO - Blob upload gear active. 01:36:28 WARNING - Blob upload directory does not exist! program finished with exit code 255 elapsedTime=14.395070 ========= Finished '/tools/buildbot/bin/python scripts/scripts/desktop_unittest.py ...' failed (results: 2, elapsed: 14 secs) (at 2014-08-12 01:36:28.779095) =========
Assignee | ||
Updated•10 years ago
|
Attachment #8471002 -
Flags: checked-in+ → checked-in-
Comment hidden (Legacy TBPL/Treeherder Robot) |
Assignee | ||
Comment 936•10 years ago
|
||
Hey Ed, Sending over to review to you, but we can test in Cedar. The only change since the previous patch is: --- a/mozharness/base/python.py +++ b/mozharness/base/python.py @@ -280,9 +280,9 @@ class VirtualenvMixin(object): if self.retry(self.run_command, args=[command,], attempts=attempts, kwargs=kwargs) != 0: if optional: self.warning("Error running install of optional package, %s." % - ' '.join(command)) + ' '.join(map(str, command))) else: - self.fatal("Error running install of package, %s!" % ' '.join(command)) + self.fatal("Error running install of package, %s!" % ' '.join(map(str, command))) def create_virtualenv(self, modules=(), requirements=()): """ The command had an integer in it ('--timeout' 120) and join expects strings. One fix would have been to make it a string ('120' instead of 120) but there is a risk people will put an integer in the configs, so I think it makes sense to convert to a string in the case it is in an integer, hence why I use map(function, iterable) to convert the list of objects to a list of strings, before "join"ing them.
Attachment #8471002 -
Attachment is obsolete: true
Attachment #8471512 -
Flags: review?(emorley)
Comment 937•10 years ago
|
||
Comment on attachment 8471512 [details] [diff] [review] bug1007230_mozharness_v3.patch I can't find any other uses of map() when handling these: http://mxr.mozilla.org/build-central/search?string=map%28&case=on&find=%2Fmozharness%2F&findi=&filter=^[^\0]*%24&hitlimit=&tree=build-central ...so I don't know if we're over-engineering & so would prefer to defer to someone who is a mozharness peer, since I'm not technically permitted to review this.
Attachment #8471512 -
Flags: review?(emorley) → review?(jlund)
Comment hidden (Legacy TBPL/Treeherder Robot) |
Assignee | ||
Comment 939•10 years ago
|
||
No worries, then we can wait for jlund. jlund: For context, this is a reproduction of the error shown in comment 934: pmoore@Elisandra:~/work/mapper master $ python Python 2.7.6rc1 (v2.7.6rc1:4913d0e9be30+, Oct 27 2013, 20:52:11) [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> >>> command = [ ... '/builds/slave/test/build/venv/bin/pip', ... 'install', ... '--download-cache', ... '/builds/slave/test/build/venv/cache', ... '--timeout', ... 120, ... '--no-index', ... '--find-links', ... 'http://pypi.pvt.build.mozilla.org/pub', ... '--find-links', ... 'http://pypi.pub.build.mozilla.org/pub', ... 'psutil==0.7.1' ... ] >>> >>> print ("Error running install of optional package, %s." % ' '.join(command)) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: sequence item 5: expected string, int found Now with the patch to use map(str, command): >>> >>> print ("Error running install of optional package, %s." % ... ' '.join(map(str, command))) Error running install of optional package, /builds/slave/test/build/venv/bin/pip install --download-cache /builds/slave/test/build/venv/cache --timeout 120 --no-index --find-links http://pypi.pvt.build.mozilla.org/pub --find-links http://pypi.pub.build.mozilla.org/pub psutil==0.7.1. map(str, command) returns the "command" list with all elements converted to strings.
Comment hidden (Legacy TBPL/Treeherder Robot) |
Assignee | ||
Comment 941•10 years ago
|
||
OK, tweaked a bit. :) Tested locally: ./mozharness/scripts/desktop_unittest.py --cfg unittests/linux_unittest.py --mochitest-suite plain-chunked --total-chunks 5 --this-chunk 4 --blob-upload-branch try --download-symbols ondemand --no-download-and-extract and: ./mozharness/scripts/vcs-sync/vcs_sync.py -c /Users/pmoore/work/mapper/l10n-pete-test.py --base-work-dir /Users/pmoore/vcs2vcs-work-dir
Attachment #8471512 -
Attachment is obsolete: true
Attachment #8471512 -
Flags: review?(jlund)
Attachment #8471569 -
Flags: review?(jlund)
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Assignee | ||
Comment 946•10 years ago
|
||
OK this one is good to go. I'll stop touching it now.
Attachment #8471569 -
Attachment is obsolete: true
Attachment #8471569 -
Flags: review?(jlund)
Attachment #8471588 -
Flags: review?(jlund)
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment 949•10 years ago
|
||
Comment on attachment 8471588 [details] [diff] [review] Last tweak, I promise... lgtm. I won't block anymore on getting this rolled out. I did not look at previous tweaks but I understand why you are doing things here.
Attachment #8471588 -
Flags: review?(jlund) → review+
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Comment hidden (Legacy TBPL/Treeherder Robot) |
Assignee | ||
Comment 958•10 years ago
|
||
Tested fine on Cypress! Will merge to production!!! :D From http://buildbot-master03.srv.releng.use1.mozilla.com:8201/builders/Ubuntu%20VM%2012.04%20cypress%20opt%20test%20mochitest-1/builds/0/steps/run_script/logs/stdio: 12:54:25 INFO - ##### 12:54:25 INFO - ##### Running create-virtualenv step. 12:54:25 INFO - ##### 12:54:25 INFO - Running pre-action listener: _install_mozbase 12:54:25 INFO - Running pre-action listener: _pre_create_virtualenv 12:54:25 INFO - Running pre-action listener: _resource_record_pre_action 12:54:25 INFO - Running main action method: create_virtualenv 12:54:25 INFO - Creating virtualenv /builds/slave/test/build/venv 12:54:25 INFO - Running command: ['/tools/buildbot/bin/python', '/tools/misc-python/virtualenv.py', '--no-site-packages', '--distribute', '/builds/slave/test/build/venv'] in /builds/slave/test/build 12:54:25 INFO - Copy/paste: /tools/buildbot/bin/python /tools/misc-python/virtualenv.py --no-site-packages --distribute /builds/slave/test/build/venv 12:54:26 INFO - The --no-site-packages flag is deprecated; it is now the default behavior. 12:54:26 INFO - Using real prefix '/usr' 12:54:26 INFO - New python executable in /builds/slave/test/build/venv/bin/python 12:54:27 INFO - Installing distribute.............................................................................................................................................................................................done. 12:54:31 INFO - Installing pip.................done. 12:54:31 INFO - Return code: 0 12:54:31 INFO - Installing psutil==0.7.1 into virtualenv /builds/slave/test/build/venv 12:54:31 INFO - retry: Calling <bound method DesktopUnittest.run_command of <__main__.DesktopUnittest object at 0x9aa3d2c>> with args: [['/builds/slave/test/build/venv/bin/pip', 'install', '--download-cache', '/builds/slave/test/build/venv/cache', '--timeout', '120', '--no-index', '--find-links', 'http://pypi.pvt.build.mozilla.org/pub', '--find-links', 'http://pypi.pub.build.mozilla.org/pub', 'psutil==0.7.1']], kwargs: {'error_list': [{'substr': 'not found or a compiler error:', 'level': 'warning'}, {'regex': <_sre.SRE_Pattern object at 0xb72af808>, 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0xb728ae30>, 'level': 'warning'}, {'regex': <_sre.SRE_Pattern object at 0x994c850>, 'level': 'debug'}, {'substr': 'command not found', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x9948be8>, 'level': 'warning'}, {'substr': 'Traceback (most recent call last)', 'level': 'error'}, {'substr': 'SyntaxError: ', 'level': 'error'}, {'substr': 'TypeError: ', 'level': 'error'}, {'substr': 'NameError: ', 'level': 'error'}, {'substr': 'ZeroDivisionError: ', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x993d280>, 'level': 'critical'}, {'regex': <_sre.SRE_Pattern object at 0x993ef60>, 'level': 'critical'}], 'cwd': '/builds/slave/test/build', 'success_codes': [0, 1]}, attempt #1 12:54:31 INFO - Running command: ['/builds/slave/test/build/venv/bin/pip', 'install', '--download-cache', '/builds/slave/test/build/venv/cache', '--timeout', '120', '--no-index', '--find-links', 'http://pypi.pvt.build.mozilla.org/pub', '--find-links', 'http://pypi.pub.build.mozilla.org/pub', 'psutil==0.7.1'] in /builds/slave/test/build 12:54:31 INFO - Copy/paste: /builds/slave/test/build/venv/bin/pip install --download-cache /builds/slave/test/build/venv/cache --timeout 120 --no-index --find-links http://pypi.pvt.build.mozilla.org/pub --find-links http://pypi.pub.build.mozilla.org/pub psutil==0.7.1 12:54:32 INFO - Ignoring indexes: https://pypi.python.org/simple/ 12:54:32 INFO - Downloading/unpacking psutil==0.7.1 12:54:32 INFO - http://pypi.pvt.build.mozilla.org/pub uses an insecure transport scheme (http). Consider using https if pypi.pvt.build.mozilla.org has it available 12:54:32 INFO - http://pypi.pub.build.mozilla.org/pub uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available 12:54:32 INFO - http://pypi.pvt.build.mozilla.org/pub uses an insecure transport scheme (http). Consider using https if pypi.pvt.build.mozilla.org has it available 12:54:32 INFO - http://pypi.pub.build.mozilla.org/pub uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available 12:54:34 INFO - Creating supposed download cache at /builds/slave/test/build/venv/cache 12:54:34 INFO - Storing download in cache at ./venv/cache/http%3A%2F%2Fpypi.pvt.build.mozilla.org%2Fpub%2Fpsutil-0.7.1.tar.gz 12:54:35 INFO - Running setup.py (path:/builds/slave/test/build/venv/build/psutil/setup.py) egg_info for package psutil 12:54:35 INFO - Installing collected packages: psutil 12:54:35 INFO - Running setup.py install for psutil 12:54:35 INFO - building '_psutil_linux' extension 12:54:35 INFO - gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -c psutil/_psutil_linux.c -o build/temp.linux-i686-2.7/psutil/_psutil_linux.o 12:54:35 INFO - gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro build/temp.linux-i686-2.7/psutil/_psutil_linux.o -o build/lib.linux-i686-2.7/_psutil_linux.so 12:54:35 INFO - building '_psutil_posix' extension 12:54:35 INFO - gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -c psutil/_psutil_posix.c -o build/temp.linux-i686-2.7/psutil/_psutil_posix.o 12:54:35 INFO - gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro build/temp.linux-i686-2.7/psutil/_psutil_posix.o -o build/lib.linux-i686-2.7/_psutil_posix.so 12:54:36 INFO - Successfully installed psutil 12:54:36 INFO - Cleaning up... 12:54:36 INFO - Return code: 0 12:54:36 INFO - Installing mozsystemmonitor==0.0.0 into virtualenv /builds/slave/test/build/venv 12:54:36 INFO - retry: Calling <bound method DesktopUnittest.run_command of <__main__.DesktopUnittest object at 0x9aa3d2c>> with args: [['/builds/slave/test/build/venv/bin/pip', 'install', '--download-cache', '/builds/slave/test/build/venv/cache', '--timeout', '120', '--no-index', '--find-links', 'http://pypi.pvt.build.mozilla.org/pub', '--find-links', 'http://pypi.pub.build.mozilla.org/pub', 'mozsystemmonitor==0.0.0']], kwargs: {'error_list': [{'substr': 'not found or a compiler error:', 'level': 'warning'}, {'regex': <_sre.SRE_Pattern object at 0xb72af808>, 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0xb728ae30>, 'level': 'warning'}, {'regex': <_sre.SRE_Pattern object at 0x994c850>, 'level': 'debug'}, {'substr': 'command not found', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x9948be8>, 'level': 'warning'}, {'substr': 'Traceback (most recent call last)', 'level': 'error'}, {'substr': 'SyntaxError: ', 'level': 'error'}, {'substr': 'TypeError: ', 'level': 'error'}, {'substr': 'NameError: ', 'level': 'error'}, {'substr': 'ZeroDivisionError: ', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x993d280>, 'level': 'critical'}, {'regex': <_sre.SRE_Pattern object at 0x993ef60>, 'level': 'critical'}], 'cwd': '/builds/slave/test/build', 'success_codes': [0, 1]}, attempt #1 12:54:36 INFO - Running command: ['/builds/slave/test/build/venv/bin/pip', 'install', '--download-cache', '/builds/slave/test/build/venv/cache', '--timeout', '120', '--no-index', '--find-links', 'http://pypi.pvt.build.mozilla.org/pub', '--find-links', 'http://pypi.pub.build.mozilla.org/pub', 'mozsystemmonitor==0.0.0'] in /builds/slave/test/build 12:54:36 INFO - Copy/paste: /builds/slave/test/build/venv/bin/pip install --download-cache /builds/slave/test/build/venv/cache --timeout 120 --no-index --find-links http://pypi.pvt.build.mozilla.org/pub --find-links http://pypi.pub.build.mozilla.org/pub mozsystemmonitor==0.0.0 12:54:36 INFO - Ignoring indexes: https://pypi.python.org/simple/ 12:54:36 INFO - Downloading/unpacking mozsystemmonitor==0.0.0 12:54:36 INFO - http://pypi.pvt.build.mozilla.org/pub uses an insecure transport scheme (http). Consider using https if pypi.pvt.build.mozilla.org has it available 12:54:36 INFO - http://pypi.pub.build.mozilla.org/pub uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available 12:54:36 INFO - http://pypi.pvt.build.mozilla.org/pub uses an insecure transport scheme (http). Consider using https if pypi.pvt.build.mozilla.org has it available 12:54:36 INFO - http://pypi.pub.build.mozilla.org/pub uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available 12:54:39 INFO - Downloading mozsystemmonitor-0.0.tar.gz 12:54:39 INFO - Storing download in cache at ./venv/cache/http%3A%2F%2Fpypi.pvt.build.mozilla.org%2Fpub%2Fmozsystemmonitor-0.0.tar.gz 12:54:39 INFO - Running setup.py (path:/builds/slave/test/build/venv/build/mozsystemmonitor/setup.py) egg_info for package mozsystemmonitor 12:54:39 INFO - Requirement already satisfied (use --upgrade to upgrade): psutil>=0.7.1 in ./venv/lib/python2.7/site-packages (from mozsystemmonitor==0.0.0) 12:54:39 INFO - Installing collected packages: mozsystemmonitor 12:54:39 INFO - Running setup.py install for mozsystemmonitor 12:54:39 INFO - Successfully installed mozsystemmonitor 12:54:39 INFO - Cleaning up... 12:54:39 INFO - Return code: 0 12:54:39 INFO - Installing blobuploader==1.2.1 into virtualenv /builds/slave/test/build/venv 12:54:39 INFO - retry: Calling <bound method DesktopUnittest.run_command of <__main__.DesktopUnittest object at 0x9aa3d2c>> with args: [['/builds/slave/test/build/venv/bin/pip', 'install', '--download-cache', '/builds/slave/test/build/venv/cache', '--timeout', '120', '--no-index', '--find-links', 'http://pypi.pvt.build.mozilla.org/pub', '--find-links', 'http://pypi.pub.build.mozilla.org/pub', 'blobuploader==1.2.1']], kwargs: {'error_list': [{'substr': 'not found or a compiler error:', 'level': 'warning'}, {'regex': <_sre.SRE_Pattern object at 0xb72af808>, 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0xb728ae30>, 'level': 'warning'}, {'regex': <_sre.SRE_Pattern object at 0x994c850>, 'level': 'debug'}, {'substr': 'command not found', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x9948be8>, 'level': 'warning'}, {'substr': 'Traceback (most recent call last)', 'level': 'error'}, {'substr': 'SyntaxError: ', 'level': 'error'}, {'substr': 'TypeError: ', 'level': 'error'}, {'substr': 'NameError: ', 'level': 'error'}, {'substr': 'ZeroDivisionError: ', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x993d280>, 'level': 'critical'}, {'regex': <_sre.SRE_Pattern object at 0x993ef60>, 'level': 'critical'}], 'cwd': '/builds/slave/test/build', 'success_codes': [0]}, attempt #1 12:54:39 INFO - Running command: ['/builds/slave/test/build/venv/bin/pip', 'install', '--download-cache', '/builds/slave/test/build/venv/cache', '--timeout', '120', '--no-index', '--find-links', 'http://pypi.pvt.build.mozilla.org/pub', '--find-links', 'http://pypi.pub.build.mozilla.org/pub', 'blobuploader==1.2.1'] in /builds/slave/test/build 12:54:39 INFO - Copy/paste: /builds/slave/test/build/venv/bin/pip install --download-cache /builds/slave/test/build/venv/cache --timeout 120 --no-index --find-links http://pypi.pvt.build.mozilla.org/pub --find-links http://pypi.pub.build.mozilla.org/pub blobuploader==1.2.1 12:54:39 INFO - Ignoring indexes: https://pypi.python.org/simple/ 12:54:39 INFO - Downloading/unpacking blobuploader==1.2.1 12:54:39 INFO - http://pypi.pvt.build.mozilla.org/pub uses an insecure transport scheme (http). Consider using https if pypi.pvt.build.mozilla.org has it available 12:54:39 INFO - http://pypi.pub.build.mozilla.org/pub uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available 12:54:39 INFO - http://pypi.pvt.build.mozilla.org/pub uses an insecure transport scheme (http). Consider using https if pypi.pvt.build.mozilla.org has it available 12:54:39 INFO - http://pypi.pub.build.mozilla.org/pub uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available 12:54:42 INFO - Downloading blobuploader-1.2.1.tar.gz 12:54:42 INFO - Storing download in cache at ./venv/cache/http%3A%2F%2Fpypi.pvt.build.mozilla.org%2Fpub%2Fblobuploader-1.2.1.tar.gz 12:54:42 INFO - Running setup.py (path:/builds/slave/test/build/venv/build/blobuploader/setup.py) egg_info for package blobuploader 12:54:42 INFO - Downloading/unpacking requests==1.2.3. (from blobuploader==1.2.1) 12:54:42 INFO - http://pypi.pvt.build.mozilla.org/pub uses an insecure transport scheme (http). Consider using https if pypi.pvt.build.mozilla.org has it available 12:54:42 INFO - http://pypi.pub.build.mozilla.org/pub uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available 12:54:42 INFO - http://pypi.pvt.build.mozilla.org/pub uses an insecure transport scheme (http). Consider using https if pypi.pvt.build.mozilla.org has it available 12:54:42 INFO - http://pypi.pub.build.mozilla.org/pub uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available 12:54:43 INFO - Storing download in cache at ./venv/cache/http%3A%2F%2Fpypi.pvt.build.mozilla.org%2Fpub%2Frequests-1.2.3.tar.gz 12:54:43 INFO - Running setup.py (path:/builds/slave/test/build/venv/build/requests/setup.py) egg_info for package requests 12:54:43 INFO - Downloading/unpacking docopt==0.6.1 (from blobuploader==1.2.1) 12:54:43 INFO - http://pypi.pvt.build.mozilla.org/pub uses an insecure transport scheme (http). Consider using https if pypi.pvt.build.mozilla.org has it available 12:54:43 INFO - http://pypi.pub.build.mozilla.org/pub uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available 12:54:43 INFO - http://pypi.pvt.build.mozilla.org/pub uses an insecure transport scheme (http). Consider using https if pypi.pvt.build.mozilla.org has it available 12:54:43 INFO - http://pypi.pub.build.mozilla.org/pub uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available 12:54:43 INFO - Downloading docopt-0.6.1.tar.gz 12:54:43 INFO - Storing download in cache at ./venv/cache/http%3A%2F%2Fpypi.pvt.build.mozilla.org%2Fpub%2Fdocopt-0.6.1.tar.gz 12:54:43 INFO - Running setup.py (path:/builds/slave/test/build/venv/build/docopt/setup.py) egg_info for package docopt 12:54:44 INFO - Installing collected packages: blobuploader, requests, docopt 12:54:44 INFO - Running setup.py install for blobuploader 12:54:44 INFO - changing mode of build/scripts-2.7/blobberc.py from 664 to 775 12:54:44 INFO - changing mode of /builds/slave/test/build/venv/bin/blobberc.py to 775 12:54:44 INFO - Running setup.py install for requests 12:54:45 INFO - Running setup.py install for docopt 12:54:45 INFO - Successfully installed blobuploader requests docopt 12:54:45 INFO - Cleaning up... 12:54:45 INFO - Return code: 0 12:54:45 INFO - Installing None into virtualenv /builds/slave/test/build/venv 12:54:45 INFO - retry: Calling <bound method DesktopUnittest.run_command of <__main__.DesktopUnittest object at 0x9aa3d2c>> with args: [['/builds/slave/test/build/venv/bin/pip', 'install', '--no-deps', '--download-cache', '/builds/slave/test/build/venv/cache', '--timeout', '120', '-r', '/builds/slave/test/build/tests/config/mozbase_requirements.txt', '--no-index', '--find-links', 'http://pypi.pvt.build.mozilla.org/pub', '--find-links', 'http://pypi.pub.build.mozilla.org/pub']], kwargs: {'error_list': [{'substr': 'not found or a compiler error:', 'level': 'warning'}, {'regex': <_sre.SRE_Pattern object at 0xb72af808>, 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0xb728ae30>, 'level': 'warning'}, {'regex': <_sre.SRE_Pattern object at 0x994c850>, 'level': 'debug'}, {'substr': 'command not found', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x9948be8>, 'level': 'warning'}, {'substr': 'Traceback (most recent call last)', 'level': 'error'}, {'substr': 'SyntaxError: ', 'level': 'error'}, {'substr': 'TypeError: ', 'level': 'error'}, {'substr': 'NameError: ', 'level': 'error'}, {'substr': 'ZeroDivisionError: ', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x993d280>, 'level': 'critical'}, {'regex': <_sre.SRE_Pattern object at 0x993ef60>, 'level': 'critical'}], 'cwd': '/builds/slave/test/build/tests/config', 'success_codes': [0]}, attempt #1 12:54:45 INFO - Running command: ['/builds/slave/test/build/venv/bin/pip', 'install', '--no-deps', '--download-cache', '/builds/slave/test/build/venv/cache', '--timeout', '120', '-r', '/builds/slave/test/build/tests/config/mozbase_requirements.txt', '--no-index', '--find-links', 'http://pypi.pvt.build.mozilla.org/pub', '--find-links', 'http://pypi.pub.build.mozilla.org/pub'] in /builds/slave/test/build/tests/config 12:54:45 INFO - Copy/paste: /builds/slave/test/build/venv/bin/pip install --no-deps --download-cache /builds/slave/test/build/venv/cache --timeout 120 -r /builds/slave/test/build/tests/config/mozbase_requirements.txt --no-index --find-links http://pypi.pvt.build.mozilla.org/pub --find-links http://pypi.pub.build.mozilla.org/pub 12:54:45 INFO - Ignoring indexes: https://pypi.python.org/simple/ 12:54:45 INFO - Unpacking /builds/slave/test/build/tests/mozbase/manifestparser 12:54:45 INFO - Running setup.py (path:/tmp/pip-oSWika-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/manifestparser 12:54:46 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozcrash 12:54:46 INFO - Running setup.py (path:/tmp/pip-aA78IH-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozcrash 12:54:46 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozdevice 12:54:46 INFO - Running setup.py (path:/tmp/pip-99mjWS-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozdevice 12:54:46 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozfile 12:54:46 INFO - Running setup.py (path:/tmp/pip-8SjADk-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozfile 12:54:46 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozhttpd 12:54:46 INFO - Running setup.py (path:/tmp/pip-oQHoVo-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozhttpd 12:54:46 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozinfo 12:54:46 INFO - Running setup.py (path:/tmp/pip-54tifg-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozinfo 12:54:46 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozinstall 12:54:46 INFO - Running setup.py (path:/tmp/pip-QnPwco-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozinstall 12:54:46 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozlog 12:54:46 INFO - Running setup.py (path:/tmp/pip-OdzZYa-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozlog 12:54:47 INFO - Unpacking /builds/slave/test/build/tests/mozbase/moznetwork 12:54:47 INFO - Running setup.py (path:/tmp/pip-rPpf4p-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/moznetwork 12:54:47 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozprocess 12:54:47 INFO - Running setup.py (path:/tmp/pip-EGaHgT-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozprocess 12:54:47 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozprofile 12:54:47 INFO - Running setup.py (path:/tmp/pip-WV6q69-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozprofile 12:54:47 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozrunner 12:54:47 INFO - Running setup.py (path:/tmp/pip-eC4OuJ-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozrunner 12:54:47 INFO - Unpacking /builds/slave/test/build/tests/mozbase/moztest 12:54:47 INFO - Running setup.py (path:/tmp/pip-CWfjg4-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/moztest 12:54:47 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozversion 12:54:47 INFO - Running setup.py (path:/tmp/pip-Mtjiz9-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozversion 12:54:47 INFO - Installing collected packages: manifestparser, mozcrash, mozdevice, mozfile, mozhttpd, mozinfo, mozInstall, mozlog, moznetwork, mozprocess, mozprofile, mozrunner, moztest, mozversion 12:54:47 INFO - Running setup.py install for manifestparser 12:54:48 INFO - Installing manifestparser script to /builds/slave/test/build/venv/bin 12:54:48 INFO - Running setup.py install for mozcrash 12:54:48 INFO - Running setup.py install for mozdevice 12:54:48 INFO - Installing sutini script to /builds/slave/test/build/venv/bin 12:54:48 INFO - Installing dm script to /builds/slave/test/build/venv/bin 12:54:48 INFO - Running setup.py install for mozfile 12:54:48 INFO - Running setup.py install for mozhttpd 12:54:48 INFO - Installing mozhttpd script to /builds/slave/test/build/venv/bin 12:54:48 INFO - Running setup.py install for mozinfo 12:54:49 INFO - Installing mozinfo script to /builds/slave/test/build/venv/bin 12:54:49 INFO - Running setup.py install for mozInstall 12:54:49 INFO - Installing moz_remove_from_system script to /builds/slave/test/build/venv/bin 12:54:49 INFO - Installing mozuninstall script to /builds/slave/test/build/venv/bin 12:54:49 INFO - Installing mozinstall script to /builds/slave/test/build/venv/bin 12:54:49 INFO - Installing moz_add_to_system script to /builds/slave/test/build/venv/bin 12:54:49 INFO - Running setup.py install for mozlog 12:54:49 INFO - Installing structlog script to /builds/slave/test/build/venv/bin 12:54:49 INFO - Running setup.py install for moznetwork 12:54:49 INFO - Running setup.py install for mozprocess 12:54:50 INFO - Running setup.py install for mozprofile 12:54:50 INFO - Installing mozprofile script to /builds/slave/test/build/venv/bin 12:54:50 INFO - Installing diff-profiles script to /builds/slave/test/build/venv/bin 12:54:50 INFO - Installing view-profile script to /builds/slave/test/build/venv/bin 12:54:50 INFO - Running setup.py install for mozrunner 12:54:50 INFO - Installing mozrunner script to /builds/slave/test/build/venv/bin 12:54:50 INFO - Running setup.py install for moztest 12:54:50 INFO - Running setup.py install for mozversion 12:54:51 INFO - Installing mozversion script to /builds/slave/test/build/venv/bin 12:54:51 INFO - Successfully installed manifestparser mozcrash mozdevice mozfile mozhttpd mozinfo mozInstall mozlog moznetwork mozprocess mozprofile mozrunner moztest mozversion 12:54:51 INFO - Cleaning up... 12:54:51 INFO - Return code: 0 12:54:51 INFO - Installing None into virtualenv /builds/slave/test/build/venv 12:54:51 INFO - retry: Calling <bound method DesktopUnittest.run_command of <__main__.DesktopUnittest object at 0x9aa3d2c>> with args: [['/builds/slave/test/build/venv/bin/pip', 'install', '--download-cache', '/builds/slave/test/build/venv/cache', '--timeout', '120', '-r', '/builds/slave/test/build/tests/config/mozbase_requirements.txt', '--no-index', '--find-links', 'http://pypi.pvt.build.mozilla.org/pub', '--find-links', 'http://pypi.pub.build.mozilla.org/pub']], kwargs: {'error_list': [{'substr': 'not found or a compiler error:', 'level': 'warning'}, {'regex': <_sre.SRE_Pattern object at 0xb72af808>, 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0xb728ae30>, 'level': 'warning'}, {'regex': <_sre.SRE_Pattern object at 0x994c850>, 'level': 'debug'}, {'substr': 'command not found', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x9948be8>, 'level': 'warning'}, {'substr': 'Traceback (most recent call last)', 'level': 'error'}, {'substr': 'SyntaxError: ', 'level': 'error'}, {'substr': 'TypeError: ', 'level': 'error'}, {'substr': 'NameError: ', 'level': 'error'}, {'substr': 'ZeroDivisionError: ', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x993d280>, 'level': 'critical'}, {'regex': <_sre.SRE_Pattern object at 0x993ef60>, 'level': 'critical'}], 'cwd': '/builds/slave/test/build/tests/config', 'success_codes': [0]}, attempt #1 12:54:51 INFO - Running command: ['/builds/slave/test/build/venv/bin/pip', 'install', '--download-cache', '/builds/slave/test/build/venv/cache', '--timeout', '120', '-r', '/builds/slave/test/build/tests/config/mozbase_requirements.txt', '--no-index', '--find-links', 'http://pypi.pvt.build.mozilla.org/pub', '--find-links', 'http://pypi.pub.build.mozilla.org/pub'] in /builds/slave/test/build/tests/config 12:54:51 INFO - Copy/paste: /builds/slave/test/build/venv/bin/pip install --download-cache /builds/slave/test/build/venv/cache --timeout 120 -r /builds/slave/test/build/tests/config/mozbase_requirements.txt --no-index --find-links http://pypi.pvt.build.mozilla.org/pub --find-links http://pypi.pub.build.mozilla.org/pub 12:54:51 INFO - Ignoring indexes: https://pypi.python.org/simple/ 12:54:51 INFO - Unpacking /builds/slave/test/build/tests/mozbase/manifestparser 12:54:51 INFO - Running setup.py (path:/tmp/pip-HwXqUy-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/manifestparser 12:54:51 INFO - Requirement already satisfied (use --upgrade to upgrade): manifestparser==0.6 from file:///builds/slave/test/build/tests/mozbase/manifestparser in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 1)) 12:54:51 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozcrash 12:54:51 INFO - Running setup.py (path:/tmp/pip-lvml41-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozcrash 12:54:51 INFO - Requirement already satisfied (use --upgrade to upgrade): mozcrash==0.12 from file:///builds/slave/test/build/tests/mozbase/mozcrash in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 2)) 12:54:51 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozdevice 12:54:51 INFO - Running setup.py (path:/tmp/pip-bfKqnF-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozdevice 12:54:52 INFO - Requirement already satisfied (use --upgrade to upgrade): mozdevice==0.37 from file:///builds/slave/test/build/tests/mozbase/mozdevice in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 3)) 12:54:52 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozfile 12:54:52 INFO - Running setup.py (path:/tmp/pip-47dRiC-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozfile 12:54:52 INFO - Requirement already satisfied (use --upgrade to upgrade): mozfile==1.1 from file:///builds/slave/test/build/tests/mozbase/mozfile in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 4)) 12:54:52 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozhttpd 12:54:52 INFO - Running setup.py (path:/tmp/pip-zRtv0A-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozhttpd 12:54:52 INFO - Requirement already satisfied (use --upgrade to upgrade): mozhttpd==0.7 from file:///builds/slave/test/build/tests/mozbase/mozhttpd in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 5)) 12:54:52 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozinfo 12:54:52 INFO - Running setup.py (path:/tmp/pip-Trz1AN-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozinfo 12:54:52 INFO - Requirement already satisfied (use --upgrade to upgrade): mozinfo==0.7 from file:///builds/slave/test/build/tests/mozbase/mozinfo in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 6)) 12:54:52 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozinstall 12:54:52 INFO - Running setup.py (path:/tmp/pip-E_vg7Q-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozinstall 12:54:52 INFO - Requirement already satisfied (use --upgrade to upgrade): mozInstall==1.10 from file:///builds/slave/test/build/tests/mozbase/mozinstall in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 7)) 12:54:52 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozlog 12:54:52 INFO - Running setup.py (path:/tmp/pip-rfkIat-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozlog 12:54:52 INFO - Requirement already satisfied (use --upgrade to upgrade): mozlog==2.0 from file:///builds/slave/test/build/tests/mozbase/mozlog in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 8)) 12:54:52 INFO - Unpacking /builds/slave/test/build/tests/mozbase/moznetwork 12:54:52 INFO - Running setup.py (path:/tmp/pip-xc5CEb-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/moznetwork 12:54:52 INFO - Requirement already satisfied (use --upgrade to upgrade): moznetwork==0.24 from file:///builds/slave/test/build/tests/mozbase/moznetwork in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 9)) 12:54:52 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozprocess 12:54:52 INFO - Running setup.py (path:/tmp/pip-dJE71f-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozprocess 12:54:53 INFO - Requirement already satisfied (use --upgrade to upgrade): mozprocess==0.19 from file:///builds/slave/test/build/tests/mozbase/mozprocess in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 10)) 12:54:53 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozprofile 12:54:53 INFO - Running setup.py (path:/tmp/pip-2wUdfL-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozprofile 12:54:53 INFO - Requirement already satisfied (use --upgrade to upgrade): mozprofile==0.21 from file:///builds/slave/test/build/tests/mozbase/mozprofile in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 11)) 12:54:53 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozrunner 12:54:53 INFO - Running setup.py (path:/tmp/pip-auw0iC-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozrunner 12:54:53 INFO - Requirement already satisfied (use --upgrade to upgrade): mozrunner==6.1 from file:///builds/slave/test/build/tests/mozbase/mozrunner in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 12)) 12:54:53 INFO - Unpacking /builds/slave/test/build/tests/mozbase/moztest 12:54:53 INFO - Running setup.py (path:/tmp/pip-2UJaUK-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/moztest 12:54:53 INFO - Requirement already satisfied (use --upgrade to upgrade): moztest==0.6 from file:///builds/slave/test/build/tests/mozbase/moztest in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 13)) 12:54:53 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozversion 12:54:53 INFO - Running setup.py (path:/tmp/pip-rNUkOK-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozversion 12:54:53 INFO - Requirement already satisfied (use --upgrade to upgrade): mozversion==0.6 from file:///builds/slave/test/build/tests/mozbase/mozversion in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 14)) 12:54:53 INFO - Requirement already satisfied (use --upgrade to upgrade): mozfile>=1.0 in /builds/slave/test/build/venv/lib/python2.7/site-packages (from mozcrash==0.12->-r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 2)) 12:54:53 INFO - Requirement already satisfied (use --upgrade to upgrade): mozlog in /builds/slave/test/build/venv/lib/python2.7/site-packages (from mozcrash==0.12->-r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 2)) 12:54:53 INFO - Requirement already satisfied (use --upgrade to upgrade): moznetwork>=0.24 in /builds/slave/test/build/venv/lib/python2.7/site-packages (from mozdevice==0.37->-r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 3)) 12:54:53 INFO - Requirement already satisfied (use --upgrade to upgrade): mozprocess>=0.19 in /builds/slave/test/build/venv/lib/python2.7/site-packages (from mozdevice==0.37->-r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 3)) 12:54:53 INFO - Cleaning up... 12:54:53 INFO - Return code: 0 12:54:53 INFO - Installing mock into virtualenv /builds/slave/test/build/venv 12:54:53 INFO - retry: Calling <bound method DesktopUnittest.run_command of <__main__.DesktopUnittest object at 0x9aa3d2c>> with args: [['/builds/slave/test/build/venv/bin/pip', 'install', '--download-cache', '/builds/slave/test/build/venv/cache', '--timeout', '120', '--no-index', '--find-links', 'http://pypi.pvt.build.mozilla.org/pub', '--find-links', 'http://pypi.pub.build.mozilla.org/pub', 'mock']], kwargs: {'error_list': [{'substr': 'not found or a compiler error:', 'level': 'warning'}, {'regex': <_sre.SRE_Pattern object at 0xb72af808>, 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0xb728ae30>, 'level': 'warning'}, {'regex': <_sre.SRE_Pattern object at 0x994c850>, 'level': 'debug'}, {'substr': 'command not found', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x9948be8>, 'level': 'warning'}, {'substr': 'Traceback (most recent call last)', 'level': 'error'}, {'substr': 'SyntaxError: ', 'level': 'error'}, {'substr': 'TypeError: ', 'level': 'error'}, {'substr': 'NameError: ', 'level': 'error'}, {'substr': 'ZeroDivisionError: ', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x993d280>, 'level': 'critical'}, {'regex': <_sre.SRE_Pattern object at 0x993ef60>, 'level': 'critical'}], 'cwd': '/builds/slave/test/build', 'success_codes': [0]}, attempt #1 12:54:53 INFO - Running command: ['/builds/slave/test/build/venv/bin/pip', 'install', '--download-cache', '/builds/slave/test/build/venv/cache', '--timeout', '120', '--no-index', '--find-links', 'http://pypi.pvt.build.mozilla.org/pub', '--find-links', 'http://pypi.pub.build.mozilla.org/pub', 'mock'] in /builds/slave/test/build 12:54:53 INFO - Copy/paste: /builds/slave/test/build/venv/bin/pip install --download-cache /builds/slave/test/build/venv/cache --timeout 120 --no-index --find-links http://pypi.pvt.build.mozilla.org/pub --find-links http://pypi.pub.build.mozilla.org/pub mock 12:54:54 INFO - Ignoring indexes: https://pypi.python.org/simple/ 12:54:54 INFO - Downloading/unpacking mock 12:54:54 INFO - http://pypi.pvt.build.mozilla.org/pub uses an insecure transport scheme (http). Consider using https if pypi.pvt.build.mozilla.org has it available 12:54:54 INFO - http://pypi.pub.build.mozilla.org/pub uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available 12:54:54 INFO - http://pypi.pvt.build.mozilla.org/pub uses an insecure transport scheme (http). Consider using https if pypi.pvt.build.mozilla.org has it available 12:54:54 INFO - http://pypi.pub.build.mozilla.org/pub uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available 12:54:57 INFO - Storing download in cache at ./venv/cache/http%3A%2F%2Fpypi.pvt.build.mozilla.org%2Fpub%2Fmock-1.0.1.tar.gz 12:54:57 INFO - Running setup.py (path:/builds/slave/test/build/venv/build/mock/setup.py) egg_info for package mock 12:54:57 INFO - warning: no files found matching '*.png' under directory 'docs' 12:54:57 INFO - warning: no files found matching '*.css' under directory 'docs' 12:54:57 INFO - warning: no files found matching '*.html' under directory 'docs' 12:54:57 INFO - warning: no files found matching '*.js' under directory 'docs' 12:54:57 INFO - Installing collected packages: mock 12:54:57 INFO - Running setup.py install for mock 12:54:57 INFO - warning: no files found matching '*.png' under directory 'docs' 12:54:57 INFO - warning: no files found matching '*.css' under directory 'docs' 12:54:57 INFO - warning: no files found matching '*.html' under directory 'docs' 12:54:57 INFO - warning: no files found matching '*.js' under directory 'docs' 12:54:57 INFO - Successfully installed mock 12:54:57 INFO - Cleaning up... 12:54:57 INFO - Return code: 0 12:54:57 INFO - Installing simplejson into virtualenv /builds/slave/test/build/venv 12:54:57 INFO - retry: Calling <bound method DesktopUnittest.run_command of <__main__.DesktopUnittest object at 0x9aa3d2c>> with args: [['/builds/slave/test/build/venv/bin/pip', 'install', '--download-cache', '/builds/slave/test/build/venv/cache', '--timeout', '120', '--no-index', '--find-links', 'http://pypi.pvt.build.mozilla.org/pub', '--find-links', 'http://pypi.pub.build.mozilla.org/pub', 'simplejson']], kwargs: {'error_list': [{'substr': 'not found or a compiler error:', 'level': 'warning'}, {'regex': <_sre.SRE_Pattern object at 0xb72af808>, 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0xb728ae30>, 'level': 'warning'}, {'regex': <_sre.SRE_Pattern object at 0x994c850>, 'level': 'debug'}, {'substr': 'command not found', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x9948be8>, 'level': 'warning'}, {'substr': 'Traceback (most recent call last)', 'level': 'error'}, {'substr': 'SyntaxError: ', 'level': 'error'}, {'substr': 'TypeError: ', 'level': 'error'}, {'substr': 'NameError: ', 'level': 'error'}, {'substr': 'ZeroDivisionError: ', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x993d280>, 'level': 'critical'}, {'regex': <_sre.SRE_Pattern object at 0x993ef60>, 'level': 'critical'}], 'cwd': '/builds/slave/test/build', 'success_codes': [0]}, attempt #1 12:54:57 INFO - Running command: ['/builds/slave/test/build/venv/bin/pip', 'install', '--download-cache', '/builds/slave/test/build/venv/cache', '--timeout', '120', '--no-index', '--find-links', 'http://pypi.pvt.build.mozilla.org/pub', '--find-links', 'http://pypi.pub.build.mozilla.org/pub', 'simplejson'] in /builds/slave/test/build 12:54:57 INFO - Copy/paste: /builds/slave/test/build/venv/bin/pip install --download-cache /builds/slave/test/build/venv/cache --timeout 120 --no-index --find-links http://pypi.pvt.build.mozilla.org/pub --find-links http://pypi.pub.build.mozilla.org/pub simplejson 12:54:58 INFO - Ignoring indexes: https://pypi.python.org/simple/ 12:54:58 INFO - Downloading/unpacking simplejson 12:54:58 INFO - http://pypi.pvt.build.mozilla.org/pub uses an insecure transport scheme (http). Consider using https if pypi.pvt.build.mozilla.org has it available 12:54:58 INFO - http://pypi.pub.build.mozilla.org/pub uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available 12:54:58 INFO - http://pypi.pvt.build.mozilla.org/pub uses an insecure transport scheme (http). Consider using https if pypi.pvt.build.mozilla.org has it available 12:54:58 INFO - http://pypi.pub.build.mozilla.org/pub uses an insecure transport scheme (http). Consider using https if pypi.pub.build.mozilla.org has it available 12:55:01 INFO - Storing download in cache at ./venv/cache/http%3A%2F%2Fpypi.pvt.build.mozilla.org%2Fpub%2Fsimplejson-3.3.0.tar.gz 12:55:01 INFO - Running setup.py (path:/builds/slave/test/build/venv/build/simplejson/setup.py) egg_info for package simplejson 12:55:01 INFO - Installing collected packages: simplejson 12:55:01 INFO - Running setup.py install for simplejson 12:55:01 INFO - building 'simplejson._speedups' extension 12:55:01 INFO - gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -c simplejson/_speedups.c -o build/temp.linux-i686-2.7/simplejson/_speedups.o 12:55:03 INFO - gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro build/temp.linux-i686-2.7/simplejson/_speedups.o -o build/lib.linux-i686-2.7/simplejson/_speedups.so 12:55:03 INFO - Successfully installed simplejson 12:55:03 INFO - Cleaning up... 12:55:03 INFO - Return code: 0 12:55:03 INFO - Installing None into virtualenv /builds/slave/test/build/venv 12:55:03 INFO - retry: Calling <bound method DesktopUnittest.run_command of <__main__.DesktopUnittest object at 0x9aa3d2c>> with args: [['/builds/slave/test/build/venv/bin/pip', 'install', '--no-deps', '--download-cache', '/builds/slave/test/build/venv/cache', '--timeout', '120', '-r', '/builds/slave/test/build/tests/config/mozbase_requirements.txt', '--no-index', '--find-links', 'http://pypi.pvt.build.mozilla.org/pub', '--find-links', 'http://pypi.pub.build.mozilla.org/pub']], kwargs: {'error_list': [{'substr': 'not found or a compiler error:', 'level': 'warning'}, {'regex': <_sre.SRE_Pattern object at 0xb72af808>, 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0xb728ae30>, 'level': 'warning'}, {'regex': <_sre.SRE_Pattern object at 0x994c850>, 'level': 'debug'}, {'substr': 'command not found', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x9948be8>, 'level': 'warning'}, {'substr': 'Traceback (most recent call last)', 'level': 'error'}, {'substr': 'SyntaxError: ', 'level': 'error'}, {'substr': 'TypeError: ', 'level': 'error'}, {'substr': 'NameError: ', 'level': 'error'}, {'substr': 'ZeroDivisionError: ', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x993d280>, 'level': 'critical'}, {'regex': <_sre.SRE_Pattern object at 0x993ef60>, 'level': 'critical'}], 'cwd': '/builds/slave/test/build/tests/config', 'success_codes': [0]}, attempt #1 12:55:03 INFO - Running command: ['/builds/slave/test/build/venv/bin/pip', 'install', '--no-deps', '--download-cache', '/builds/slave/test/build/venv/cache', '--timeout', '120', '-r', '/builds/slave/test/build/tests/config/mozbase_requirements.txt', '--no-index', '--find-links', 'http://pypi.pvt.build.mozilla.org/pub', '--find-links', 'http://pypi.pub.build.mozilla.org/pub'] in /builds/slave/test/build/tests/config 12:55:03 INFO - Copy/paste: /builds/slave/test/build/venv/bin/pip install --no-deps --download-cache /builds/slave/test/build/venv/cache --timeout 120 -r /builds/slave/test/build/tests/config/mozbase_requirements.txt --no-index --find-links http://pypi.pvt.build.mozilla.org/pub --find-links http://pypi.pub.build.mozilla.org/pub 12:55:03 INFO - Ignoring indexes: https://pypi.python.org/simple/ 12:55:03 INFO - Unpacking /builds/slave/test/build/tests/mozbase/manifestparser 12:55:03 INFO - Running setup.py (path:/tmp/pip-VcEP87-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/manifestparser 12:55:03 INFO - Requirement already satisfied (use --upgrade to upgrade): manifestparser==0.6 from file:///builds/slave/test/build/tests/mozbase/manifestparser in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 1)) 12:55:03 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozcrash 12:55:03 INFO - Running setup.py (path:/tmp/pip-Mdf7gx-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozcrash 12:55:04 INFO - Requirement already satisfied (use --upgrade to upgrade): mozcrash==0.12 from file:///builds/slave/test/build/tests/mozbase/mozcrash in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 2)) 12:55:04 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozdevice 12:55:04 INFO - Running setup.py (path:/tmp/pip-dEdWBL-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozdevice 12:55:04 INFO - Requirement already satisfied (use --upgrade to upgrade): mozdevice==0.37 from file:///builds/slave/test/build/tests/mozbase/mozdevice in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 3)) 12:55:04 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozfile 12:55:04 INFO - Running setup.py (path:/tmp/pip-q83Af8-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozfile 12:55:04 INFO - Requirement already satisfied (use --upgrade to upgrade): mozfile==1.1 from file:///builds/slave/test/build/tests/mozbase/mozfile in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 4)) 12:55:04 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozhttpd 12:55:04 INFO - Running setup.py (path:/tmp/pip-n6B4Mv-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozhttpd 12:55:04 INFO - Requirement already satisfied (use --upgrade to upgrade): mozhttpd==0.7 from file:///builds/slave/test/build/tests/mozbase/mozhttpd in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 5)) 12:55:04 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozinfo 12:55:04 INFO - Running setup.py (path:/tmp/pip-EXBjIz-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozinfo 12:55:04 INFO - Requirement already satisfied (use --upgrade to upgrade): mozinfo==0.7 from file:///builds/slave/test/build/tests/mozbase/mozinfo in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 6)) 12:55:04 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozinstall 12:55:04 INFO - Running setup.py (path:/tmp/pip-4kkTcb-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozinstall 12:55:04 INFO - Requirement already satisfied (use --upgrade to upgrade): mozInstall==1.10 from file:///builds/slave/test/build/tests/mozbase/mozinstall in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 7)) 12:55:04 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozlog 12:55:04 INFO - Running setup.py (path:/tmp/pip-oU5XgO-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozlog 12:55:04 INFO - Requirement already satisfied (use --upgrade to upgrade): mozlog==2.0 from file:///builds/slave/test/build/tests/mozbase/mozlog in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 8)) 12:55:04 INFO - Unpacking /builds/slave/test/build/tests/mozbase/moznetwork 12:55:04 INFO - Running setup.py (path:/tmp/pip-mKT4yN-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/moznetwork 12:55:05 INFO - Requirement already satisfied (use --upgrade to upgrade): moznetwork==0.24 from file:///builds/slave/test/build/tests/mozbase/moznetwork in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 9)) 12:55:05 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozprocess 12:55:05 INFO - Running setup.py (path:/tmp/pip-WI0aiM-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozprocess 12:55:05 INFO - Requirement already satisfied (use --upgrade to upgrade): mozprocess==0.19 from file:///builds/slave/test/build/tests/mozbase/mozprocess in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 10)) 12:55:05 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozprofile 12:55:05 INFO - Running setup.py (path:/tmp/pip-VqsN2z-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozprofile 12:55:05 INFO - Requirement already satisfied (use --upgrade to upgrade): mozprofile==0.21 from file:///builds/slave/test/build/tests/mozbase/mozprofile in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 11)) 12:55:05 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozrunner 12:55:05 INFO - Running setup.py (path:/tmp/pip-Jzf8EM-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozrunner 12:55:05 INFO - Requirement already satisfied (use --upgrade to upgrade): mozrunner==6.1 from file:///builds/slave/test/build/tests/mozbase/mozrunner in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 12)) 12:55:05 INFO - Unpacking /builds/slave/test/build/tests/mozbase/moztest 12:55:05 INFO - Running setup.py (path:/tmp/pip-pxSMin-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/moztest 12:55:05 INFO - Requirement already satisfied (use --upgrade to upgrade): moztest==0.6 from file:///builds/slave/test/build/tests/mozbase/moztest in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 13)) 12:55:05 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozversion 12:55:05 INFO - Running setup.py (path:/tmp/pip-cQFsfA-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozversion 12:55:05 INFO - Requirement already satisfied (use --upgrade to upgrade): mozversion==0.6 from file:///builds/slave/test/build/tests/mozbase/mozversion in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 14)) 12:55:05 INFO - Cleaning up... 12:55:05 INFO - Return code: 0 12:55:05 INFO - Installing None into virtualenv /builds/slave/test/build/venv 12:55:05 INFO - retry: Calling <bound method DesktopUnittest.run_command of <__main__.DesktopUnittest object at 0x9aa3d2c>> with args: [['/builds/slave/test/build/venv/bin/pip', 'install', '--download-cache', '/builds/slave/test/build/venv/cache', '--timeout', '120', '-r', '/builds/slave/test/build/tests/config/mozbase_requirements.txt', '--no-index', '--find-links', 'http://pypi.pvt.build.mozilla.org/pub', '--find-links', 'http://pypi.pub.build.mozilla.org/pub']], kwargs: {'error_list': [{'substr': 'not found or a compiler error:', 'level': 'warning'}, {'regex': <_sre.SRE_Pattern object at 0xb72af808>, 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0xb728ae30>, 'level': 'warning'}, {'regex': <_sre.SRE_Pattern object at 0x994c850>, 'level': 'debug'}, {'substr': 'command not found', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x9948be8>, 'level': 'warning'}, {'substr': 'Traceback (most recent call last)', 'level': 'error'}, {'substr': 'SyntaxError: ', 'level': 'error'}, {'substr': 'TypeError: ', 'level': 'error'}, {'substr': 'NameError: ', 'level': 'error'}, {'substr': 'ZeroDivisionError: ', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x993d280>, 'level': 'critical'}, {'regex': <_sre.SRE_Pattern object at 0x993ef60>, 'level': 'critical'}], 'cwd': '/builds/slave/test/build/tests/config', 'success_codes': [0]}, attempt #1 12:55:05 INFO - Running command: ['/builds/slave/test/build/venv/bin/pip', 'install', '--download-cache', '/builds/slave/test/build/venv/cache', '--timeout', '120', '-r', '/builds/slave/test/build/tests/config/mozbase_requirements.txt', '--no-index', '--find-links', 'http://pypi.pvt.build.mozilla.org/pub', '--find-links', 'http://pypi.pub.build.mozilla.org/pub'] in /builds/slave/test/build/tests/config 12:55:05 INFO - Copy/paste: /builds/slave/test/build/venv/bin/pip install --download-cache /builds/slave/test/build/venv/cache --timeout 120 -r /builds/slave/test/build/tests/config/mozbase_requirements.txt --no-index --find-links http://pypi.pvt.build.mozilla.org/pub --find-links http://pypi.pub.build.mozilla.org/pub 12:55:06 INFO - Ignoring indexes: https://pypi.python.org/simple/ 12:55:06 INFO - Unpacking /builds/slave/test/build/tests/mozbase/manifestparser 12:55:06 INFO - Running setup.py (path:/tmp/pip-Pgq0zn-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/manifestparser 12:55:06 INFO - Requirement already satisfied (use --upgrade to upgrade): manifestparser==0.6 from file:///builds/slave/test/build/tests/mozbase/manifestparser in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 1)) 12:55:06 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozcrash 12:55:06 INFO - Running setup.py (path:/tmp/pip-_J70VZ-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozcrash 12:55:06 INFO - Requirement already satisfied (use --upgrade to upgrade): mozcrash==0.12 from file:///builds/slave/test/build/tests/mozbase/mozcrash in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 2)) 12:55:06 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozdevice 12:55:06 INFO - Running setup.py (path:/tmp/pip-vdwqOY-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozdevice 12:55:06 INFO - Requirement already satisfied (use --upgrade to upgrade): mozdevice==0.37 from file:///builds/slave/test/build/tests/mozbase/mozdevice in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 3)) 12:55:06 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozfile 12:55:06 INFO - Running setup.py (path:/tmp/pip-rUfS_q-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozfile 12:55:06 INFO - Requirement already satisfied (use --upgrade to upgrade): mozfile==1.1 from file:///builds/slave/test/build/tests/mozbase/mozfile in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 4)) 12:55:06 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozhttpd 12:55:06 INFO - Running setup.py (path:/tmp/pip-wfE8Ew-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozhttpd 12:55:06 INFO - Requirement already satisfied (use --upgrade to upgrade): mozhttpd==0.7 from file:///builds/slave/test/build/tests/mozbase/mozhttpd in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 5)) 12:55:06 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozinfo 12:55:06 INFO - Running setup.py (path:/tmp/pip-DJjIRQ-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozinfo 12:55:07 INFO - Requirement already satisfied (use --upgrade to upgrade): mozinfo==0.7 from file:///builds/slave/test/build/tests/mozbase/mozinfo in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 6)) 12:55:07 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozinstall 12:55:07 INFO - Running setup.py (path:/tmp/pip-LwRehV-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozinstall 12:55:07 INFO - Requirement already satisfied (use --upgrade to upgrade): mozInstall==1.10 from file:///builds/slave/test/build/tests/mozbase/mozinstall in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 7)) 12:55:07 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozlog 12:55:07 INFO - Running setup.py (path:/tmp/pip-zjh0LA-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozlog 12:55:07 INFO - Requirement already satisfied (use --upgrade to upgrade): mozlog==2.0 from file:///builds/slave/test/build/tests/mozbase/mozlog in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 8)) 12:55:07 INFO - Unpacking /builds/slave/test/build/tests/mozbase/moznetwork 12:55:07 INFO - Running setup.py (path:/tmp/pip-Ywkwqp-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/moznetwork 12:55:07 INFO - Requirement already satisfied (use --upgrade to upgrade): moznetwork==0.24 from file:///builds/slave/test/build/tests/mozbase/moznetwork in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 9)) 12:55:07 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozprocess 12:55:07 INFO - Running setup.py (path:/tmp/pip-GtRDVK-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozprocess 12:55:07 INFO - Requirement already satisfied (use --upgrade to upgrade): mozprocess==0.19 from file:///builds/slave/test/build/tests/mozbase/mozprocess in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 10)) 12:55:07 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozprofile 12:55:07 INFO - Running setup.py (path:/tmp/pip-6E860D-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozprofile 12:55:07 INFO - Requirement already satisfied (use --upgrade to upgrade): mozprofile==0.21 from file:///builds/slave/test/build/tests/mozbase/mozprofile in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 11)) 12:55:07 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozrunner 12:55:07 INFO - Running setup.py (path:/tmp/pip-dvxSJx-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozrunner 12:55:07 INFO - Requirement already satisfied (use --upgrade to upgrade): mozrunner==6.1 from file:///builds/slave/test/build/tests/mozbase/mozrunner in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 12)) 12:55:07 INFO - Unpacking /builds/slave/test/build/tests/mozbase/moztest 12:55:07 INFO - Running setup.py (path:/tmp/pip-QWgtrt-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/moztest 12:55:08 INFO - Requirement already satisfied (use --upgrade to upgrade): moztest==0.6 from file:///builds/slave/test/build/tests/mozbase/moztest in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 13)) 12:55:08 INFO - Unpacking /builds/slave/test/build/tests/mozbase/mozversion 12:55:08 INFO - Running setup.py (path:/tmp/pip-ykZovS-build/setup.py) egg_info for package from file:///builds/slave/test/build/tests/mozbase/mozversion 12:55:08 INFO - Requirement already satisfied (use --upgrade to upgrade): mozversion==0.6 from file:///builds/slave/test/build/tests/mozbase/mozversion in /builds/slave/test/build/venv/lib/python2.7/site-packages (from -r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 14)) 12:55:08 INFO - Requirement already satisfied (use --upgrade to upgrade): mozfile>=1.0 in /builds/slave/test/build/venv/lib/python2.7/site-packages (from mozcrash==0.12->-r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 2)) 12:55:08 INFO - Requirement already satisfied (use --upgrade to upgrade): mozlog in /builds/slave/test/build/venv/lib/python2.7/site-packages (from mozcrash==0.12->-r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 2)) 12:55:08 INFO - Requirement already satisfied (use --upgrade to upgrade): moznetwork>=0.24 in /builds/slave/test/build/venv/lib/python2.7/site-packages (from mozdevice==0.37->-r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 3)) 12:55:08 INFO - Requirement already satisfied (use --upgrade to upgrade): mozprocess>=0.19 in /builds/slave/test/build/venv/lib/python2.7/site-packages (from mozdevice==0.37->-r /builds/slave/test/build/tests/config/mozbase_requirements.txt (line 3)) 12:55:08 INFO - Cleaning up... 12:55:08 INFO - Return code: 0 12:55:08 INFO - Done creating virtualenv /builds/slave/test/build/venv. 12:55:08 INFO - Getting output from command: ['/builds/slave/test/build/venv/bin/pip', 'freeze'] 12:55:08 INFO - Copy/paste: /builds/slave/test/build/venv/bin/pip freeze 12:55:08 INFO - Reading from file tmpfile_stdout 12:55:08 INFO - Current package versions: 12:55:08 INFO - mozInstall == 1.10 12:55:08 INFO - mozfile == 1.1 12:55:08 INFO - mozlog == 2.0 12:55:08 INFO - manifestparser == 0.6 12:55:08 INFO - moznetwork == 0.24 12:55:08 INFO - mozsystemmonitor == 0.0 12:55:08 INFO - mozdevice == 0.37 12:55:08 INFO - mozprocess == 0.19 12:55:08 INFO - mozrunner == 6.1 12:55:08 INFO - mock == 1.0.1 12:55:08 INFO - mozprofile == 0.21 12:55:08 INFO - mozinfo == 0.7 12:55:08 INFO - simplejson == 3.3.0 12:55:08 INFO - blobuploader == 1.2.1 12:55:08 INFO - moztest == 0.6 12:55:08 INFO - mozcrash == 0.12 12:55:08 INFO - argparse == 1.2.1 12:55:08 INFO - mozhttpd == 0.7 12:55:08 INFO - wsgiref == 0.1.2 12:55:08 INFO - psutil == 0.7.1 12:55:08 INFO - mozversion == 0.6 12:55:08 INFO - requests == 1.2.3 12:55:08 INFO - docopt == 0.6.1 12:55:08 INFO - Running post-action listener: _resource_record_post_action 12:55:08 INFO - Running post-action listener: _start_resource_monitoring 12:55:08 INFO - Starting resource monitoring. 12:55:08 INFO - ##### 12:55:08 INFO - ##### Running pull step. 12:55:08 INFO - #####
Assignee | ||
Comment 959•10 years ago
|
||
Comment on attachment 8471588 [details] [diff] [review] Last tweak, I promise... Production: https://hg.mozilla.org/build/mozharness/rev/86669bc515cb
Attachment #8471588 -
Flags: checked-in+
Assignee | ||
Comment 960•10 years ago
|
||
10 hours without a comment from TBPL Robot - I'm assuming it worked! :D Closing bug.
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Comment 961•10 years ago
|
||
Thank you :-)
Comment hidden (Legacy TBPL/Treeherder Robot) |
Updated•6 years ago
|
Component: General Automation → General
You need to log in
before you can comment on or make changes to this bug.
Description
•