Closed Bug 1592773 Opened Last month Closed 13 days ago

Intermittent awsy/test_memory_usage.py TestMemoryUsage.test_open_tabs | OSError: [Errno 2] No such file or directory [mozproxy]

Categories

(Testing :: Mozbase, defect, P5)

defect

Tracking

(Not tracked)

RESOLVED INCOMPLETE

People

(Reporter: intermittent-bug-filer, Unassigned)

Details

(Keywords: intermittent-failure)

Filed by: rmaries [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=273763492&repo=autoland
Full log: https://queue.taskcluster.net/v1/task/ML8AZgEbSEGLGxSudr6AQA/runs/0/artifacts/public/logs/live_backing.log


[task 2019-10-30T20:23:09.225Z] 20:23:09 INFO - TEST-START | awsy/test_memory_usage.py TestMemoryUsage.test_open_tabs
[task 2019-10-30T20:25:41.434Z] 20:25:41 INFO - mozproxy browser path: /builds/worker/workspace/build/application/firefox/firefox
[task 2019-10-30T20:25:41.435Z] 20:25:41 INFO - mozproxy Starting mitmproxy playback using env path: /builds/worker/workspace/build/application/firefox:/builds/worker/workspace/build/venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/builds/worker/bin
[task 2019-10-30T20:25:41.436Z] 20:25:41 INFO - mozproxy Starting mitmproxy playback using command: /builds/worker/workspace/build/tests/html/testing/mozproxy/mitmdump-4.0.4/mitmdump -v --set upstream_cert=false --set websocket=false --set server_replay_files=/builds/worker/workspace/build/tests/html/testing/mozproxy/yandex.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/apple.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/yahoo-mail.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/imgur.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/imdb.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/ebay.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/twitter.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/youtube.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/fandom.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/google-sheets.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/google-docs.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/google-search.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/google-mail.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/pinterest.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/tumblr.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/microsoft.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/reddit.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/paypal.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/google-slides.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/yahoo-news.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/instagram.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/amazon.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/facebook.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/bing.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/wikipedia.mp --scripts /builds/worker/workspace/build/venv/lib/python2.7/site-packages/mozproxy/backends/mitm/scripts/alternate-server-replay-4.0.4.py
[task 2019-10-30T20:25:41.440Z] 20:25:41 INFO - [u'/builds/worker/workspace/build/tests/html/testing/mozproxy/mitmdump-4.0.4/mitmdump', '-v', '--set', 'upstream_cert=false', '--set', 'websocket=false', '--set', 'server_replay_files=/builds/worker/workspace/build/tests/html/testing/mozproxy/yandex.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/apple.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/yahoo-mail.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/imgur.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/imdb.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/ebay.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/twitter.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/youtube.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/fandom.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/google-sheets.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/google-docs.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/google-search.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/google-mail.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/pinterest.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/tumblr.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/microsoft.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/reddit.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/paypal.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/google-slides.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/yahoo-news.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/instagram.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/amazon.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/facebook.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/bing.mp,/builds/worker/workspace/build/tests/html/testing/mozproxy/wikipedia.mp', '--scripts', '/builds/worker/workspace/build/venv/lib/python2.7/site-packages/mozproxy/backends/mitm/scripts/alternate-server-replay-4.0.4.py']
[task 2019-10-30T20:25:41.524Z] 20:25:41 INFO - TEST-UNEXPECTED-ERROR | awsy/test_memory_usage.py TestMemoryUsage.test_open_tabs | OSError: [Errno 2] No such file or directory
[task 2019-10-30T20:25:41.524Z] 20:25:41 INFO - Traceback (most recent call last):
[task 2019-10-30T20:25:41.524Z] 20:25:41 INFO - File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_harness/marionette_test/testcases.py", line 140, in run
[task 2019-10-30T20:25:41.524Z] 20:25:41 INFO - self.setUp()
[task 2019-10-30T20:25:41.524Z] 20:25:41 INFO - File "/builds/worker/workspace/build/tests/awsy/awsy/test_memory_usage.py", line 136, in setUp
[task 2019-10-30T20:25:41.525Z] 20:25:41 INFO - self.setupTp6()
[task 2019-10-30T20:25:41.525Z] 20:25:41 INFO - File "/builds/worker/workspace/build/tests/awsy/awsy/test_memory_usage.py", line 115, in setupTp6
[task 2019-10-30T20:25:41.525Z] 20:25:41 INFO - self._playback.start()
[task 2019-10-30T20:25:41.525Z] 20:25:41 INFO - File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/mozproxy/backends/mitm/mitm.py", line 123, in start
[task 2019-10-30T20:25:41.526Z] 20:25:41 INFO - self.start_mitmproxy_playback(self.mitmdump_path, self.browser_path)
[task 2019-10-30T20:25:41.526Z] 20:25:41 INFO - File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/mozproxy/backends/mitm/mitm.py", line 237, in start_mitmproxy_playback
[task 2019-10-30T20:25:41.526Z] 20:25:41 INFO - self.mitmproxy_proc.run()
[task 2019-10-30T20:25:41.526Z] 20:25:41 INFO - File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/mozprocess/processhandler.py", line 811, in run
[task 2019-10-30T20:25:41.526Z] 20:25:41 INFO - self.proc = self.Process([self.cmd] + self.args, **args)
[task 2019-10-30T20:25:41.526Z] 20:25:41 INFO - File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/mozprocess/processhandler.py", line 123, in init
[task 2019-10-30T20:25:41.526Z] 20:25:41 INFO - universal_newlines, startupinfo, creationflags)
[task 2019-10-30T20:25:41.526Z] 20:25:41 INFO - File "/usr/lib/python2.7/subprocess.py", line 711, in init
[task 2019-10-30T20:25:41.527Z] 20:25:41 INFO - errread, errwrite)
[task 2019-10-30T20:25:41.527Z] 20:25:41 INFO - File "/usr/lib/python2.7/subprocess.py", line 1343, in _execute_child
[task 2019-10-30T20:25:41.527Z] 20:25:41 INFO - raise child_exception
[task 2019-10-30T20:25:41.527Z] 20:25:41 INFO - TEST-INFO took 152286ms
[task 2019-10-30T20:25:41.528Z] 20:25:41 INFO -
[task 2019-10-30T20:25:41.528Z] 20:25:41 INFO - SUMMARY
[task 2019-10-30T20:25:41.528Z] 20:25:41 INFO - -------
[task 2019-10-30T20:25:41.528Z] 20:25:41 INFO - passed: 0
[task 2019-10-30T20:25:41.528Z] 20:25:41 INFO - failed: 1
[task 2019-10-30T20:25:41.528Z] 20:25:41 INFO - todo: 0
[task 2019-10-30T20:25:41.528Z] 20:25:41 INFO -
[task 2019-10-30T20:25:41.528Z] 20:25:41 INFO - FAILED TESTS
[task 2019-10-30T20:25:41.528Z] 20:25:41 INFO - -------
[task 2019-10-30T20:25:41.528Z] 20:25:41 INFO - test_memory_usage.py test_memory_usage.TestMemoryUsage.test_open_tabs

It looks like mozproxy is failing to launch mitmproxy, which makes sense given earlier in the log we failed to install it:

[task 2019-10-30T20:23:09.268Z] 20:23:09     INFO - mozproxy downloading mitmproxy binary
[task 2019-10-30T20:23:09.324Z] 20:23:09     INFO - mozproxy INFO - File mitmproxy-4.0.4-linux.tar.gz not present in local cache folder /builds/worker/tooltool-cache
[task 2019-10-30T20:23:09.326Z] 20:23:09     INFO - mozproxy INFO - Attempting to fetch from 'https://tooltool.mozilla-releng.net/'...
[task 2019-10-30T20:25:16.634Z] 20:25:16     INFO - mozproxy INFO - ...failed to fetch 'mitmproxy-4.0.4-linux.tar.gz' from https://tooltool.mozilla-releng.net/
[task 2019-10-30T20:25:16.635Z] 20:25:16     INFO - mozproxy Traceback (most recent call last):
[task 2019-10-30T20:25:16.635Z] 20:25:16     INFO - mozproxy   File "/builds/worker/workspace/mozharness/external_tools/tooltool.py", line 731, in fetch_file
[task 2019-10-30T20:25:16.635Z] 20:25:16     INFO - mozproxy     f = urllib2.urlopen(req)
[task 2019-10-30T20:25:16.635Z] 20:25:16     INFO - mozproxy   File "/usr/lib/python2.7/urllib2.py", line 154, in urlopen
[task 2019-10-30T20:25:16.636Z] 20:25:16     INFO - mozproxy     return opener.open(url, data, timeout)
[task 2019-10-30T20:25:16.636Z] 20:25:16     INFO - mozproxy   File "/usr/lib/python2.7/urllib2.py", line 429, in open
[task 2019-10-30T20:25:16.636Z] 20:25:16     INFO - mozproxy     response = self._open(req, data)
[task 2019-10-30T20:25:16.636Z] 20:25:16     INFO - mozproxy   File "/usr/lib/python2.7/urllib2.py", line 447, in _open
[task 2019-10-30T20:25:16.637Z] 20:25:16     INFO - mozproxy     '_open', req)
[task 2019-10-30T20:25:16.637Z] 20:25:16     INFO - mozproxy   File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
[task 2019-10-30T20:25:16.637Z] 20:25:16     INFO - mozproxy     result = func(*args)
[task 2019-10-30T20:25:16.637Z] 20:25:16     INFO - mozproxy   File "/usr/lib/python2.7/urllib2.py", line 1241, in https_open
[task 2019-10-30T20:25:16.638Z] 20:25:16     INFO - mozproxy     context=self._context)
[task 2019-10-30T20:25:16.638Z] 20:25:16     INFO - mozproxy   File "/usr/lib/python2.7/urllib2.py", line 1198, in do_open
[task 2019-10-30T20:25:16.638Z] 20:25:16     INFO - mozproxy     raise URLError(err)
[task 2019-10-30T20:25:16.638Z] 20:25:16     INFO - mozproxy URLError: <urlopen error [Errno 110] Connection timed out>
[task 2019-10-30T20:25:16.639Z] 20:25:16     INFO - mozproxy ERROR - The following files failed: 'mitmproxy-4.0.4-linux.tar.gz'
Component: DMD → Mozbase
Product: Core → Testing

The failure as visible in the Treeherder log viewer should really be:

[task 2019-10-30T20:25:16.639Z] 20:25:16 INFO - mozproxy ERROR - The following files failed to download: 'mitmproxy-4.0.4-linux.tar.gz'

Maybe having a retry mechanism as what we have for other to download files would be good to have.

Summary: Intermittent awsy/test_memory_usage.py TestMemoryUsage.test_open_tabs | OSError: [Errno 2] No such file or directory → Intermittent awsy/test_memory_usage.py TestMemoryUsage.test_open_tabs | OSError: [Errno 2] No such file or directory [mozproxy]
Status: NEW → RESOLVED
Closed: 13 days ago
Resolution: --- → INCOMPLETE
You need to log in before you can comment on or make changes to this bug.