1553932 - Intermittent subprocess.CalledProcessError: Command '['/builds/task_1558633286/workspace/testing/mozproxy/host-utils-68.0a1.en-US.linux-x86_64/certutil', '-d', 'sql:/tmp/tmp5784Or.mozrunner/', '-L']' returned non-zero exit status 255

Status: RESOLVED FIXED

(Testing :: Raptor, defect, P1)

Description

[Treeherder Bug Filer]

Filed by: rmaries [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=248011710&repo=autoland
Full log: https://queue.taskcluster.net/v1/task/Vr3PMQEzRDag1hGsMtsexw/runs/0/artifacts/public/logs/live_backing.log

---

17:43:27 ERROR - Traceback (most recent call last):
17:43:27 INFO - File "/builds/task_1558633286/workspace/build/tests/raptor/raptor/raptor.py", line 1230, in <module>
17:43:27 INFO - main()
17:43:27 INFO - File "/builds/task_1558633286/workspace/build/tests/raptor/raptor/raptor.py", line 1198, in main
17:43:27 INFO - success = raptor.run_tests(raptor_test_list, raptor_test_names)
17:43:27 INFO - File "/builds/task_1558633286/workspace/build/tests/raptor/raptor/raptor.py", line 960, in run_tests
17:43:27 INFO - return super(RaptorAndroid, self).run_tests(tests, test_names)
17:43:27 INFO - File "/builds/task_1558633286/workspace/build/tests/raptor/raptor/raptor.py", line 186, in run_tests
17:43:27 INFO - self.run_test(test, timeout=int(test['page_timeout']))
17:43:27 INFO - File "/builds/task_1558633286/workspace/build/tests/raptor/raptor/raptor.py", line 979, in run_test
17:43:27 INFO - self.run_test_cold(test, timeout)
17:43:27 INFO - File "/builds/task_1558633286/workspace/build/tests/raptor/raptor/raptor.py", line 1034, in run_test_cold
17:43:27 INFO - self.start_playback(test)
17:43:27 INFO - File "/builds/task_1558633286/workspace/build/tests/raptor/raptor/raptor.py", line 357, in start_playback
17:43:27 INFO - self.playback.start()
17:43:27 INFO - File "/builds/task_1558633286/workspace/build/venv/lib/python2.7/site-packages/mozproxy/backends/mitm.py", line 125, in start
17:43:27 INFO - self.setup()
17:43:27 INFO - File "/builds/task_1558633286/workspace/build/venv/lib/python2.7/site-packages/mozproxy/backends/mitm.py", line 384, in setup
17:43:27 INFO - self.install_mitmproxy_cert(self.browser_path)
17:43:27 INFO - File "/builds/task_1558633286/workspace/build/venv/lib/python2.7/site-packages/mozproxy/backends/mitm.py", line 535, in install_mitmproxy_cert
17:43:27 INFO - if not self.is_mitmproxy_cert_installed():
17:43:27 INFO - File "/builds/task_1558633286/workspace/build/venv/lib/python2.7/site-packages/mozproxy/backends/mitm.py", line 552, in is_mitmproxy_cert_installed
17:43:27 INFO - cmd_output = subprocess.check_output(command, env=os.environ.copy())
17:43:27 INFO - File "/usr/lib/python2.7/subprocess.py", line 574, in check_output
17:43:27 INFO - raise CalledProcessError(retcode, cmd, output=output)
17:43:27 INFO - subprocess.CalledProcessError: Command '['/builds/task_1558633286/workspace/testing/mozproxy/host-utils-68.0a1.en-US.linux-x86_64/certutil', '-d', 'sql:/tmp/tmp5784Or.mozrunner/', '-L']' returned non-zero exit status 255
17:43:27 INFO - certutil: could not change trust on certificate: SEC_ERROR_PKCS11_DEVICE_ERROR: A PKCS #11 module returned CKR_DEVICE_ERROR, indicating that a problem has occurred with the token or slot.
17:43:27 ERROR - Return code: 1
17:43:27 WARNING - setting return code to 1
17:43:27 INFO - Killing logcat pid 465.
17:43:27 CRITICAL - PERFHERDER_DATA was seen 0 times, expected 1.
17:43:27 INFO - copying raptor results to upload dir:
17:43:27 INFO - /builds/task_1558633286/workspace/build/blobber_upload_dir/perfherder-data.json
17:43:27 INFO - copying raptor results from /builds/task_1558633286/workspace/build/raptor.json to /builds/task_1558633286/workspace/build/blobber_upload_dir/perfherder-data.json
17:43:27 CRITICAL - Error copying results /builds/task_1558633286/workspace/build/raptor.json to upload dir /builds/task_1558633286/workspace/build/blobber_upload_dir/perfherder-data.json
17:43:27 INFO - [Errno 2] No such file or directory: u'/builds/task_1558633286/workspace/build/raptor.json'
17:43:27 INFO - Running post-action listener: _package_coverage_data
17:43:27 INFO - Running post-action listener: _resource_record_post_action
17:43:27 INFO - Running post-action listener: process_java_coverage_data
17:43:27 INFO - Running post-action listener: stop_device
17:43:30 INFO - Killing logcat pid 465.
17:43:30 INFO - [mozharness: 2019-05-23 17:43:30.031711Z] Finished run-tests step (success)
17:43:30 INFO - Running post-run listener: _resource_record_post_run
17:43:30 INFO - Total resource usage - Wall time: 71s; CPU: 41.0%; Read bytes: 0; Write bytes: 7399153664; Read time: 0; Write time: 86768584
17:43:30 INFO - TinderboxPrint: CPU usage<br/>41.2%
17:43:30 INFO - TinderboxPrint: I/O read bytes / time<br/>0 / 0
17:43:30 INFO - TinderboxPrint: I/O write bytes / time<br/>7,399,153,664 / 86,768,584
17:43:30 INFO - TinderboxPrint: CPU idle<br/>123.9 (43.0%)
17:43:30 INFO - TinderboxPrint: CPU iowait<br/>45.8 (15.9%)
17:43:30 INFO - TinderboxPrint: CPU softirq<br/>6.8 (2.4%)
17:43:30 INFO - TinderboxPrint: CPU system<br/>21.8 (7.6%)
17:43:30 INFO - TinderboxPrint: CPU user<br/>89.7 (31.2%)
17:43:30 INFO - TinderboxPrint: Swap in / out<br/>0 / 0
17:43:30 INFO - install - Wall time: 13s; CPU: 41.0%; Read bytes: 0; Write bytes: 159391744; Read time: 0; Write time: 2196820
17:43:30 INFO - run-tests - Wall time: 56s; CPU: 40.0%; Read bytes: 0; Write bytes: 6496309248; Read time: 0; Write time: 83958844
17:43:30 WARNING - returning nonzero exit status 1
cleanup

• cleanup
• local rv=1
• [[ -s /builds/worker/.xsession-errors ]]
• false
• exit 1

Comment 1

[Intermittent Failures Robot]

8 failures in 4730 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-inbound: 3
• autoland: 4
• mozilla-central: 1

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16: 2
• android-hw-p2-8-0-android-aarch64: 6

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1553932&startday=2019-05-20&endday=2019-05-26&tree=all

Comment 2

[Intermittent Failures Robot]

2 failures in 4337 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1
• autoland: 1

Platform breakdown:

• android-hw-p2-8-0-android-aarch64: 1
• android-hw-g5-7-0-arm7-api-16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1553932&startday=2019-05-27&endday=2019-06-02&tree=all

Comment 4

[Intermittent Failures Robot]

11 failures in 5197 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 3
• autoland: 6
• try: 2

Platform breakdown:

• android-hw-p2-8-0-arm7-api-16: 5
• android-hw-g5-7-0-arm7-api-16: 4
• android-hw-p2-8-0-android-aarch64: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1553932&startday=2019-06-03&endday=2019-06-09&tree=all

Comment 5

[Intermittent Failures Robot]

2 failures in 4753 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1
• autoland: 1

Platform breakdown:

• android-hw-p2-8-0-arm7-api-16: 1
• android-hw-p2-8-0-android-aarch64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1553932&startday=2019-06-10&endday=2019-06-16&tree=all

Comment 6

[Intermittent Failures Robot]

6 failures in 2581 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1
• autoland: 4
• try: 1

Platform breakdown:

• android-hw-p2-8-0-android-aarch64: 3
• android-hw-g5-7-0-arm7-api-16: 2
• android-hw-p2-8-0-arm7-api-16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1553932&startday=2019-06-17&endday=2019-06-23&tree=all

Comment 7

[Intermittent Failures Robot]

8 failures in 4409 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 5
• mozilla-inbound: 3

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16: 5
• android-hw-p2-8-0-arm7-api-16: 2
• android-hw-p2-8-0-android-aarch64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1553932&startday=2019-06-24&endday=2019-06-30&tree=all

Comment 8

[Intermittent Failures Robot]

11 failures in 3501 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 6
• mozilla-central: 2
• mozilla-inbound: 3

Platform breakdown:

• android-hw-p2-8-0-arm7-api-16: 3
• android-hw-p2-8-0-android-aarch64: 4
• android-hw-g5-7-0-arm7-api-16: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1553932&startday=2019-07-01&endday=2019-07-07&tree=all

Comment 9

[Intermittent Failures Robot]

15 failures in 837 pushes (0.018 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 10
• mozilla-inbound: 2
• mozilla-central: 3

Platform breakdown:

• android-hw-p2-8-0-arm7-api-16: 8
• android-hw-g5-7-0-arm7-api-16: 4
• android-hw-p2-8-0-android-aarch64: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1553932&startday=2019-07-12&endday=2019-07-12&tree=all

Comment 10

[Intermittent Failures Robot]

16 failures in 179 pushes (0.089 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-central: 14
• mozilla-inbound: 2

Platform breakdown:

• android-hw-p2-8-0-arm7-api-16: 7
• android-hw-g5-7-0-arm7-api-16: 4
• android-hw-p2-8-0-android-aarch64: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1553932&startday=2019-07-13&endday=2019-07-13&tree=all

Comment 11

[Daniel Varga [:dvarga]]

In the last 7 days there were 49 failures associated with this bug: https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?startday=2019-07-07&endday=2019-07-14&tree=trunk&bug=1553932
Recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=256417309&repo=mozilla-inbound&lineNumber=1256
Occurs on android-hw-p2-8-0-android-aarch64, android-hw-p2-8-0-arm7-api-16, android-hw-g5-7-0-arm7-api-16 build type.

Comment 12

[Intermittent Failures Robot]

51 failures in 4710 pushes (0.011 failures/push) were associated with this bug in the last 7 days.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 14
• mozilla-central: 27
• mozilla-inbound: 8
• try: 2

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16: 12
• android-hw-p2-8-0-arm7-api-16: 22
• android-hw-p2-8-0-android-aarch64: 17

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1553932&startday=2019-07-08&endday=2019-07-14&tree=all

Comment 13

[Robert Wood [:rwood]]

:bebe can you please have a look and investigate this as it pertains to mozproxy, thanks!

Comment 14

[Intermittent Failures Robot]

18 failures in 792 pushes (0.023 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-central: 7
• try: 2
• autoland: 9

Platform breakdown:

• android-hw-p2-8-0-arm7-api-16: 9
• android-hw-g5-7-0-arm7-api-16: 3
• android-hw-p2-8-0-android-aarch64: 6

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1553932&startday=2019-07-15&endday=2019-07-15&tree=all

Comment 15

[Intermittent Failures Robot]

19 failures in 816 pushes (0.023 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 12
• mozilla-central: 4
• mozilla-inbound: 3

Platform breakdown:

• android-hw-p2-8-0-android-aarch64: 7
• android-hw-p2-8-0-arm7-api-16: 6
• android-hw-g5-7-0-arm7-api-16: 6

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1553932&startday=2019-07-16&endday=2019-07-16&tree=all

Comment 16

[Intermittent Failures Robot]

20 failures in 945 pushes (0.021 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 8
• mozilla-central: 6
• mozilla-inbound: 4
• try: 2

Platform breakdown:

• android-hw-p2-8-0-android-aarch64: 9
• android-hw-p2-8-0-arm7-api-16: 7
• android-hw-g5-7-0-arm7-api-16: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1553932&startday=2019-07-17&endday=2019-07-17&tree=all

Comment 17

[Intermittent Failures Robot]

16 failures in 811 pushes (0.02 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-central: 12
• autoland: 2
• mozilla-inbound: 2

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16: 7
• android-hw-p2-8-0-android-aarch64: 8
• android-hw-p2-8-0-arm7-api-16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1553932&startday=2019-07-18&endday=2019-07-18&tree=all

Comment 18

[Intermittent Failures Robot]

26 failures in 644 pushes (0.04 failures/push) were associated with this bug yesterday.

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. **

Repository breakdown:

• autoland: 14
• mozilla-central: 8
• mozilla-inbound: 4

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16: 9
• android-hw-p2-8-0-arm7-api-16: 10
• android-hw-p2-8-0-android-aarch64: 7

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1553932&startday=2019-07-19&endday=2019-07-19&tree=all

Comment 19

[Razvan Maries]

There are 123 total failures in the last 7 days: https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?startday=2019-07-13&endday=2019-07-20&tree=trunk&bug=1553932

Recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=257518529&repo=mozilla-central&lineNumber=1262

Florin, do you have any updates on this?

Comment 20

[Intermittent Failures Robot]

114 failures in 4298 pushes (0.027 failures/push) were associated with this bug in the last 7 days.

This is the #8 most frequent failure this week.

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-central: 47
• try: 4
• autoland: 47
• mozilla-inbound: 16

Platform breakdown:

• android-hw-p2-8-0-arm7-api-16: 39
• android-hw-g5-7-0-arm7-api-16: 33
• android-hw-p2-8-0-android-aarch64: 42

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1553932&startday=2019-07-15&endday=2019-07-21&tree=all

Comment 21

[Florin Strugariu [:Bebe]]

try build to better investigate the fail:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=8cb71850ac691e02e681ea6bf4a61bc17763b2b8

Comment 22

[Florin Strugariu [:Bebe]]

Attachment: Bug 1553932 Update mozproxy certutil usage

Comment 24

[Intermittent Failures Robot]

15 failures in 1010 pushes (0.015 failures/push) were associated with this bug yesterday.

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. **

Repository breakdown:

• mozilla-central: 6
• try: 1
• autoland: 7
• mozilla-inbound: 1

Platform breakdown:

• android-hw-p2-8-0-android-aarch64: 5
• android-hw-p2-8-0-arm7-api-16: 6
• android-hw-g5-7-0-arm7-api-16: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1553932&startday=2019-07-24&endday=2019-07-24&tree=all

Comment 25

[Intermittent Failures Robot]

25 failures in 954 pushes (0.026 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 12
• mozilla-inbound: 3
• mozilla-beta: 1
• mozilla-central: 9

Platform breakdown:

• android-hw-p2-8-0-android-aarch64: 12
• android-hw-p2-8-0-arm7-api-16: 8
• android-hw-g5-7-0-arm7-api-16: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1553932&startday=2019-07-25&endday=2019-07-25&tree=all

Comment 26

[Intermittent Failures Robot]

16 failures in 147 pushes (0.109 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 7
• mozilla-central: 6
• try: 1
• mozilla-inbound: 2

Platform breakdown:

• android-hw-p2-8-0-arm7-api-16: 7
• android-hw-p2-8-0-android-aarch64: 8
• android-hw-g5-7-0-arm7-api-16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1553932&startday=2019-07-27&endday=2019-07-27&tree=all

Comment 27

[Intermittent Failures Robot]

85 failures in 4608 pushes (0.018 failures/push) were associated with this bug in the last 7 days.

This is the #13 most frequent failure this week.

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 44
• mozilla-central: 28
• mozilla-inbound: 10
• try: 2
• mozilla-beta: 1

Platform breakdown:

• android-hw-p2-8-0-android-aarch64: 42
• android-hw-g5-7-0-arm7-api-16: 17
• android-hw-p2-8-0-arm7-api-16: 26

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1553932&startday=2019-07-22&endday=2019-07-28&tree=all

Comment 28

[Cosmin Sabou [:CosminS]]

Robert, Dave, could you please review Florin's patch for these failures? This is on the disabled recommeded list for quite a while now and it has 281 total failures in the last 30 days. Thank you.

Comment 29

[Intermittent Failures Robot]

19 failures in 869 pushes (0.022 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-inbound: 1
• autoland: 13
• mozilla-central: 4
• mozilla-beta: 1

Platform breakdown:

• android-hw-p2-8-0-android-aarch64: 6
• android-hw-g5-7-0-arm7-api-16: 8
• android-hw-p2-8-0-arm7-api-16: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1553932&startday=2019-07-29&endday=2019-07-29&tree=all

Comment 30

[Dave Hunt [:davehunt] [he/him] ⌚BST]

(In reply to Cosmin Sabou [:CosminS] from comment #28)

Robert, Dave, could you please review Florin's patch for these failures? This is on the disabled recommeded list for quite a while now and it has 281 total failures in the last 30 days. Thank you.

I've reviewed the patch, but I'm not clear how this would resolve the error. It looks to me that it might highlight the error message in the logs, which will make it easier to identify, but I don't see a fix here.

Comment 31

[Intermittent Failures Robot]

18 failures in 657 pushes (0.027 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 8
• mozilla-central: 9
• mozilla-inbound: 1

Platform breakdown:

• android-hw-p2-8-0-android-aarch64: 9
• android-hw-g5-7-0-arm7-api-16: 7
• android-hw-p2-8-0-arm7-api-16: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1553932&startday=2019-07-31&endday=2019-07-31&tree=all

Comment 32

[Andreea Pavel [:apavel]]

Hi Florin, Are there any updates here?

Comment 33

[Intermittent Failures Robot]

28 failures in 866 pushes (0.032 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-central: 8
• mozilla-inbound: 2
• autoland: 17
• try: 1

Platform breakdown:

• android-hw-p2-8-0-android-aarch64: 8
• android-hw-p2-8-0-arm7-api-16: 8
• android-hw-g5-7-0-arm7-api-16: 12

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1553932&startday=2019-08-02&endday=2019-08-02&tree=all

Comment 34

[Intermittent Failures Robot]

22 failures in 231 pushes (0.095 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 12
• mozilla-central: 8
• mozilla-inbound: 2

Platform breakdown:

• android-hw-p2-8-0-android-aarch64: 9
• android-hw-g5-7-0-arm7-api-16: 8
• android-hw-p2-8-0-arm7-api-16: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1553932&startday=2019-08-03&endday=2019-08-03&tree=all

Comment 35

[Intermittent Failures Robot]

116 failures in 4439 pushes (0.026 failures/push) were associated with this bug in the last 7 days.

This is the #9 most frequent failure this week.

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-inbound: 8
• autoland: 59
• mozilla-central: 45
• mozilla-beta: 2
• try: 2

Platform breakdown:

• android-hw-p2-8-0-android-aarch64: 42
• android-hw-g5-7-0-arm7-api-16: 44
• android-hw-p2-8-0-arm7-api-16: 30

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1553932&startday=2019-07-29&endday=2019-08-04&tree=all

Comment 36

[Intermittent Failures Robot]

18 failures in 553 pushes (0.033 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 8
• mozilla-central: 9
• mozilla-inbound: 1

Platform breakdown:

• android-hw-p2-8-0-android-aarch64: 8
• android-hw-g5-7-0-arm7-api-16: 2
• android-hw-p2-8-0-arm7-api-16: 8

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1553932&startday=2019-08-05&endday=2019-08-05&tree=all

Comment 37

[Andreea Pavel [:apavel]]

Hi Joel, can you please take a look at this bug?

Comment 38

[Intermittent Failures Robot]

23 failures in 858 pushes (0.027 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 8
• mozilla-central: 11
• try: 1
• mozilla-inbound: 3

Platform breakdown:

• android-hw-p2-8-0-arm7-api-16: 8
• android-hw-g5-7-0-arm7-api-16: 9
• android-hw-p2-8-0-android-aarch64: 5
• windows7-32: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1553932&startday=2019-08-06&endday=2019-08-06&tree=all

Comment 39

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4fcbb2234b02
Update mozproxy certutil usage r=tarek

Comment 40

[Stefan Hindli [:stefan_hindli]]

https://hg.mozilla.org/mozilla-central/rev/4fcbb2234b02

Comment 41

[Intermittent Failures Robot]

54 failures in 4666 pushes (0.012 failures/push) were associated with this bug in the last 7 days.

This is the #32 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 20
• mozilla-central: 28
• mozilla-inbound: 5
• try: 1

Platform breakdown:

• android-hw-p2-8-0-android-aarch64: 17
• android-hw-g5-7-0-arm7-api-16: 17
• android-hw-p2-8-0-arm7-api-16: 19
• windows7-32: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1553932&startday=2019-08-05&endday=2019-08-11&tree=all

Comment 42

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/b529d6d87af4