Closed Bug 1567668 Opened 6 years ago Closed 6 years ago

Intermittent raptor-fennec subprocess.CalledProcessError: Command '['/builds/task_1563599232/workspace/testing/mozproxy/host-utils-68.0a1.en-US.linux-x86_64/certutil', '-d', 'sql:/tmp/tmpn2Ah9x.mozrunner/', '-L']' returned non-zero exit status 255

Categories

(Testing :: Raptor, defect, P5)

Product:
Testing
Component:
Raptor
Version:
Version 3
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED DUPLICATE of bug 1553932

People

(Reporter: intermittent-bug-filer, Unassigned)

References

(Regression)

Details

(Keywords: intermittent-failure, regression)

Filed by: apavel [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=257508428&repo=mozilla-central
Full log: https://queue.taskcluster.net/v1/task/Nx-xAvkURVWuuNhTO2L38g/runs/0/artifacts/public/logs/live_backing.log

05:08:46 INFO - raptor-main Info: starting raptor test: raptor-tp6m-instagram-fennec64
05:08:46 INFO - raptor-main Info: test settings: {'expected_browser_cycles': 1, 'alert_threshold': '2.0', 'playback_pageset_manifest': 'mitmproxy-recordings-raptor-tp6m-instagram-mobile.manifest', 'ancestor-manifest': '/builds/task_1563599232/workspace/build/tests/raptor/raptor/raptor.ini', 'here': '/builds/task_1563599232/workspace/build/tests/raptor/raptor/tests/tp6/mobile', 'measure': ['fnbpaint', 'dcf', 'loadtime'], 'path': '/builds/task_1563599232/workspace/build/tests/raptor/raptor/tests/tp6/mobile/raptor-tp6m-instagram-fennec64', 'cold': False, 'playback_binary_manifest': 'mitmproxy-rel-bin-{platform}.manifest', 'unit': 'ms', 'alert_on': ['loadtime'], 'type': 'pageload', 'name': 'raptor-tp6m-instagram-fennec64', 'playback_recordings': 'instagram-mobile.mp', 'page_cycles': '15', 'apps': 'fennec', 'test_url': 'https://www.instagram.com', 'manifest': '/builds/task_1563599232/workspace/build/tests/raptor/raptor/tests/tp6/mobile/raptor-tp6m-2-fennec64.ini', 'page_timeout': '60000', 'playback': 'mitmproxy-android', 'browser_cycle': 1, 'expected': 'pass', 'lower_is_better': 'true', 'relpath': 'tests/tp6/mobile/raptor-tp6m-instagram-fennec64'}
05:08:46 INFO - raptor-main Info: raptor config: {'binary': 'org.mozilla.firefox', 'local_profile_dir': '/tmp/tmpn2Ah9x.mozrunner', 'symbols_path': 'https://queue.taskcluster.net/v1/task/eEYlFYtdSpiDiFHLD0Zfog/artifacts/public/build/target.crashreporter-symbols.zip', 'memory_test': False, 'cpu_test': False, 'enable_control_server_wait': False, 'e10s': True, 'app': 'fennec', 'gecko_profile_entries': None, 'power_test': False, 'run_local': False, 'platform': 'linux', 'host': '127.0.0.1', 'is_release_build': False, 'intent': None, 'enable_webrender': False, 'activity': None, 'gecko_profile_interval': None, 'processor': 'x86_64', 'gecko_profile': False, 'obj_path': None}
05:08:46 INFO - raptor-gen-test-config Info: writing test settings into background js, so webext can get it
05:08:46 INFO - raptor-gen-test-config Info: finished writing test config to /builds/task_1563599232/workspace/build/tests/raptor/webext/raptor/auto_gen_test_config.js
05:08:46 INFO - raptor-main Info: installing webext /builds/task_1563599232/workspace/build/tests/raptor/raptor/../webext/raptor
05:08:46 INFO - raptor-main Info: making the raptor control server port available to device
05:08:46 INFO - adb command_output: adb -s FA8221A01968 wait-for-device reverse tcp:58091 tcp:58091, timeout: None, timedout: None, exitcode: 0, output:
05:08:46 INFO - raptor-main Info: making the raptor playback server port available to device
05:08:46 INFO - adb command_output: adb -s FA8221A01968 wait-for-device reverse tcp:8080 tcp:8080, timeout: None, timedout: None, exitcode: 0, output:
05:08:46 INFO - raptor-main Info: test uses playback tool: mitmproxy-android

05:08:54 INFO - mozproxy certutil is found at: /builds/task_1563599232/workspace/testing/mozproxy/host-utils-68.0a1.en-US.linux-x86_64/certutil
05:08:54 INFO - mozproxy checking if the nss cert db already exists in the android browser profile
05:08:54 INFO - certutil: function failed: SEC_ERROR_BAD_DATABASE: security library: bad database.
05:08:54 INFO - mozproxy nss cert db doesn't exist yet
05:09:05 INFO - mozproxy creating nss cert database using command: /builds/task_1563599232/workspace/testing/mozproxy/host-utils-68.0a1.en-US.linux-x86_64/certutil -N -v -d sql:/tmp/tmpn2Ah9x.mozrunner/ --empty-password
05:09:05 INFO - certutil: function failed: SEC_ERROR_BAD_DATABASE: security library: bad database.
05:09:15 INFO - mozproxy importing mitmproxy cert into db using command: /builds/task_1563599232/workspace/testing/mozproxy/host-utils-68.0a1.en-US.linux-x86_64/certutil -A -d sql:/tmp/tmpn2Ah9x.mozrunner/ -n mitmproxy-cert -t TC,, -a -i /builds/worker/.mitmproxy/mitmproxy-ca-cert.cer
05:09:25 INFO - mozproxy command to import mitmproxy cert into cert db failed to complete
05:09:25 INFO - mozproxy verifying that the mitmproxy ca cert is installed on android
05:09:25 INFO - mozproxy getting the list of certs in the nss cert db in the android browser profile
05:09:26 INFO - certutil: function failed: SEC_ERROR_BAD_DATABASE: security library: bad database.
05:09:26 INFO - mozproxy certutil command failed
05:09:26 INFO - mozproxy Stopping mitmproxy playback, killing process 686
05:09:26 INFO - mozproxy Successfully killed the mitmproxy playback process
05:09:26 INFO - raptor-main Info: removing reverse socket connections
05:09:26 INFO - adb command_output: adb -s FA8221A01968 wait-for-device reverse --remove-all, timeout: None, timedout: None, exitcode: 0, output:
05:09:26 INFO - raptor-main Info: skipping check_for_crashes: application has not been launched
05:09:26 INFO - raptor-main Info: removing webext /builds/task_1563599232/workspace/build/tests/raptor/raptor/../webext/raptor
05:09:26 INFO - raptor-main Info: removing test folder for raptor: /sdcard/raptor
05:09:26 INFO - adb shell_output: adb -s FA8221A01968 wait-for-device shell rm -r /sdcard/raptor; echo adb_returncode=$?, timeout: None, root: False, timedout: None, exitcode: 0, output:
05:09:26 INFO - raptor-control-server Info: shutting down control server
05:09:26 INFO - raptor-main Info: finished
05:09:26 ERROR - Traceback (most recent call last):
05:09:26 INFO - File "/builds/task_1563599232/workspace/build/tests/raptor/raptor/raptor.py", line 1381, in <module>
05:09:26 INFO - main()
05:09:26 INFO - File "/builds/task_1563599232/workspace/build/tests/raptor/raptor/raptor.py", line 1349, in main
05:09:26 INFO - success = raptor.run_tests(raptor_test_list, raptor_test_names)
05:09:26 INFO - File "/builds/task_1563599232/workspace/build/tests/raptor/raptor/raptor.py", line 1084, in run_tests
05:09:26 INFO - return super(RaptorAndroid, self).run_tests(tests, test_names)
05:09:26 INFO - File "/builds/task_1563599232/workspace/build/tests/raptor/raptor/raptor.py", line 186, in run_tests
05:09:26 INFO - self.run_test(test, timeout=int(test.get('page_timeout')))
05:09:26 INFO - File "/builds/task_1563599232/workspace/build/tests/raptor/raptor/raptor.py", line 1116, in run_test
05:09:26 INFO - self.__run_test_warm(test, timeout)
05:09:26 INFO - File "/builds/task_1563599232/workspace/build/tests/raptor/raptor/raptor.py", line 1229, in __run_test_warm
05:09:26 INFO - self.start_playback(test)
05:09:26 INFO - File "/builds/task_1563599232/workspace/build/tests/raptor/raptor/raptor.py", line 392, in start_playback
05:09:26 INFO - self.playback.start()
05:09:26 INFO - File "/builds/task_1563599232/workspace/build/venv/lib/python2.7/site-packages/mozproxy/backends/mitm.py", line 125, in start
05:09:26 INFO - self.setup()
05:09:26 INFO - File "/builds/task_1563599232/workspace/build/venv/lib/python2.7/site-packages/mozproxy/backends/mitm.py", line 384, in setup
05:09:26 INFO - self.install_mitmproxy_cert(self.browser_path)
05:09:26 INFO - File "/builds/task_1563599232/workspace/build/venv/lib/python2.7/site-packages/mozproxy/backends/mitm.py", line 535, in install_mitmproxy_cert
05:09:26 INFO - if not self.is_mitmproxy_cert_installed():
05:09:26 INFO - File "/builds/task_1563599232/workspace/build/venv/lib/python2.7/site-packages/mozproxy/backends/mitm.py", line 552, in is_mitmproxy_cert_installed
05:09:26 INFO - cmd_output = subprocess.check_output(command, env=os.environ.copy())
05:09:26 INFO - File "/usr/lib/python2.7/subprocess.py", line 574, in check_output
05:09:26 INFO - raise CalledProcessError(retcode, cmd, output=output)
05:09:26 INFO - subprocess.CalledProcessError: Command '['/builds/task_1563599232/workspace/testing/mozproxy/host-utils-68.0a1.en-US.linux-x86_64/certutil', '-d', 'sql:/tmp/tmpn2Ah9x.mozrunner/', '-L']' returned non-zero exit status 255
05:09:30 ERROR - Return code: 1
05:09:30 WARNING - setting return code to 1
05:09:30 INFO - Killing logcat pid 506.
05:09:30 INFO - copying raptor results to upload dir:
05:09:30 INFO - /builds/task_1563599232/workspace/build/blobber_upload_dir/perfherder-data.json
05:09:30 INFO - copying raptor results from /builds/task_1563599232/workspace/build/raptor.json to /builds/task_1563599232/workspace/build/blobber_upload_dir/perfherder-data.json
05:09:30 CRITICAL - Error copying results /builds/task_1563599232/workspace/build/raptor.json to upload dir /builds/task_1563599232/workspace/build/blobber_upload_dir/perfherder-data.json
05:09:30 INFO - [Errno 2] No such file or directory: u'/builds/task_1563599232/workspace/build/raptor.json'
05:09:30 INFO - Running post-action listener: _package_coverage_data
05:09:30 INFO - Running post-action listener: _resource_record_post_action
05:09:30 INFO - Running post-action listener: process_java_coverage_data
05:09:30 INFO - Running post-action listener: stop_device

Marian could you please take a look? This seems to be from bug 1564046

Flags: needinfo?(marian.raiciof)
Regressed by: 1564046

Hi Andreea,

I did a push to try
https://treeherder.mozilla.org/#/jobs?repo=try&revision=e718d6fa75ccff8a9fb8b16394805f0796f8f84d
to be sure that the problem is not caused by the e10s=true, as described in this bug:
https://bugzilla.mozilla.org/show_bug.cgi?id=1565908

Also, Bebe is doing the rerecordings for the mobile tests using mitm4, instagram included. This will change the test's mp and manifest files.

Thank you.

Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → DUPLICATE
Flags: needinfo?(marian.raiciof)
Has Regression Range: --- → yes
You need to log in before you can comment on or make changes to this bug.