1533221 - Intermittent testing/marionette/harness/marionette_harness/tests/unit/test_capabilities.py TestCapabilities.test_additional_capabilities | AssertionError: u'/storage/emulated/0/tests/profile' != '/sdcard/tests/profile'

Status: RESOLVED FIXED

(Testing :: Marionette Client and Harness, defect, P1)

Description

[Treeherder Bug Filer]

#[markdown(off)]
Filed by: dluca [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=232326854&repo=autoland

https://queue.taskcluster.net/v1/task/HNOA3IR-RVOf4CInbuSZ_Q/runs/0/artifacts/public/logs/live_backing.log

TEST-START | testing/marionette/harness/marionette_harness/tests/unit/test_capabilities.py TestCapabilities.test_additional_capabilities
[task 2019-03-07T01:39:03.649Z] 01:39:03 WARNING - TEST-UNEXPECTED-FAIL | testing/marionette/harness/marionette_harness/tests/unit/test_capabilities.py TestCapabilities.test_additional_capabilities | AssertionError: u'/storage/emulated/0/tests/profile' != '/sdcard/tests/profile'
[task 2019-03-07T01:39:03.650Z] 01:39:03 INFO - Traceback (most recent call last):
[task 2019-03-07T01:39:03.650Z] 01:39:03 INFO - File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_harness/marionette_test/testcases.py", line 159, in run
[task 2019-03-07T01:39:03.651Z] 01:39:03 INFO - testMethod()
[task 2019-03-07T01:39:03.651Z] 01:39:03 INFO - File "/builds/worker/workspace/build/tests/marionette/tests/testing/marionette/harness/marionette_harness/tests/unit/test_capabilities.py", line 102, in test_additional_capabilities
[task 2019-03-07T01:39:03.651Z] 01:39:03 INFO - self.assertEqual(self.caps["moz:profile"].lower(), current_profile.lower())
[task 2019-03-07T01:39:03.652Z] 01:39:03 INFO - TEST-INFO took 335ms
[task 2019-03-07T01:39:03.862Z] 01:39:03 INFO - TEST-START | testing/marionette/harness/marionette_harness/tests/unit/test_capabilities.py TestCapabilities.test_disable_webdriver_click
[task 2019-03-07T01:39:04.110Z] 01:39:04 INFO - TEST-PASS | testing/marionette/harness/marionette_harness/tests/unit/test_capabilities.py TestCapabilities.test_disable_webdriver_click | took 250ms
[task 2019-03-07T01:39:04.316Z] 01:39:04 INFO - TEST-START | testing/marionette/harness/marionette_harness/tests/unit/test_capabilities.py TestCapabilities.test_mandated_capabilities
[task 2019-03-07T01:39:04.453Z] 01:39:04 INFO - TEST-PASS | testing/marionette/harness/marionette_harness/tests/unit/test_capabilities.py TestCapabilities.test_mandated_capabilities | took 137ms
[task 2019-03-07T01:39:04.660Z] 01:39:04 INFO - TEST-START | testing/marionette/harness/marionette_harness/tests/unit/test_capabilities.py TestCapabilities.test_supported_features
[task 2019-03-07T01:39:04.796Z] 01:39:04 INFO - TEST-PASS | testing/marionette/harness/marionette_harness/tests/unit/test_capabilities.py TestCapabilities.test_supported_features | took 135ms
[task 2019-03-07T01:39:05.001Z] 01:39:05 INFO - TEST-START | testing/marionette/harness/marionette_harness/tests/unit/test_capabilities.py TestCapabilities.test_use_non_spec_compliant_pointer_origin
[task 2019-03-07T01:39:05.249Z] 01:39:05 INFO - TEST-PASS | testing/marionette/harness/marionette_harness/tests/unit/test_capabilities.py TestCapabilities.test_use_non_spec_compliant_pointer_origin | took 248ms
[task 2019-03-07T01:39:05.455Z] 01:39:05 INFO - TEST-START | testing/marionette/harness/marionette_harness/tests/unit/test_capabilities.py TestCapabilities.test_we_get_valid_uuid4_when_creating_a_session
[task 2019-03-07T01:39:05.584Z] 01:39:05 INFO - TEST-PASS | testing/marionette/harness/marionette_harness/tests/unit/test_capabilities.py TestCapabilities.test_we_get_valid_uuid4_when_creating_a_session | took 129ms
[task 2019-03-07T01:39:05.792Z] 01:39:05 INFO - TEST-START | testing/marionette/harness/marionette_harness/tests/unit/test_capabilities.py TestCapabilityMatching.test_accept_insecure_certs
[task 2019-03-07T01:39:05.910Z] 01:39:05 INFO - type <type 'str'>
[task 2019-03-07T01:39:06.017Z] 01:39:06 INFO - type <type 'int'>
[task 2019-03-07T01:39:06.125Z] 01:39:06 INFO - type <type 'dict'>
[task 2019-03-07T01:39:06.234Z] 01:39:06 INFO - type <type 'list'>
[task 2019-03-07T01:39:06.476Z] 01:39:06 INFO - TEST-PASS | testing/marionette/harness/marionette_harness/tests/unit/test_capabilities.py TestCapabilityMatching.test_accept_insecure_certs | took 685ms
[task 2019-03-07T01:39:06.681Z] 01:39:06 INFO - TEST-START | testing/marionette/harness/marionette_harness/tests/unit/test_capabilities.py TestCapabilityMatching.test_page_load_strategy

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Looks like bug 1526084 didn't fix it completely. Geoff, can you please have a look?

Comment 2

[Intermittent Failures Robot]

2 failures in 2818 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 2

Platform breakdown:

• android-em-7-0-x86_64: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1533221&startday=2019-03-04&endday=2019-03-10&tree=all

Comment 3

[Intermittent Failures Robot]

2 failures in 2771 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-inbound: 1
• try: 1

Platform breakdown:

• android-em-7-0-x86_64: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1533221&startday=2019-03-11&endday=2019-03-17&tree=all

Comment 4

[Geoff Brown [:gbrown]]

Attachment: Bug 1533221 - In Mn test_capabilites.py, always use custom code to find fennec profile path; r=whimboo

This is quite speculative: What if one of the profile paths being compared here changes, such
that the condition is not satisfied in get_fennec_profile and links are not followed, and then the
unmodified profile path fails the assertion later? I do not see how that can happen, but maybe?
Also, removing the condition reduces the possible code paths, which might help us focus attention
if the intermittent persists.

Comment 5

[Geoff Brown [:gbrown]]

Attachment: Bug 1533221 - In Mn test_capabilities.py, compare only the basename of profile paths; r=whimboo

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ffe777450100
In Mn test_capabilities.py, compare only the basename of profile paths; r=whimboo

Comment 8

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/ffe777450100

Comment 9

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/086a215d4caa

Comment 10

[BMO Automation]

Moving bug to Testing::Marionette Client and Harness component per bug 1815831.