1361325 - Autophone Mdm4 perma fail on Android 5.1/Nexus 6

Status: RESOLVED FIXED

(Core :: WebRTC, defect, P3)

Description

[Bob Clary [:bc] (inactive)]

Attachment: failures

The following tests in Mdm4 fail on Android 5.1/Nexus 6

dom/media/tests/mochitest/identity/test_fingerprints.html
dom/media/tests/mochitest/identity/test_getIdentityAssertion.html
dom/media/tests/mochitest/identity/test_idpproxy.html
dom/media/tests/mochitest/identity/test_idpproxy.html
dom/media/tests/mochitest/identity/test_idpproxy.html
dom/media/tests/mochitest/identity/test_idpproxy.html
dom/media/tests/mochitest/identity/test_idpproxy.html
dom/media/tests/mochitest/identity/test_loginNeeded.html
dom/media/tests/mochitest/identity/test_loginNeeded.html:32:13
dom/media/tests/mochitest/identity/test_peerConnection_asymmetricIsolation.html
dom/media/tests/mochitest/identity/test_peerConnection_peerIdentity.html
dom/media/tests/mochitest/identity/test_setIdentityProvider.html
dom/media/tests/mochitest/identity/test_setIdentityProviderWithErrors.html


https://autophone.s3.amazonaws.com/v1/task/ByjGz98ZR5u342zVyaIDsA/runs/0/artifacts/public/build/mochitest-dom-media-tests-identity-mochitests-dom-media-tests-identity-settings-debug.ini-1-nexus-6-1-a9927acd-1dc2-4123-a6c0-ffd9d497cdac.log

These will be disabled in bug 1358876.

Comment 1

[Dan Minor [:dminor]]

Any idea what is going on with these tests? We had them disabled in the old Mw job, but they run fine locally and I had a green try push with them enabled back in April (https://treeherder.mozilla.org/#/jobs?repo=try&revision=c49df13eb73f7774f64063dc078c80ef4de0a49e).

Comment 2

[Bob Clary [:bc] (inactive)]

No, I don't but that is certainly very interesting.

Comment 3

[Dan Minor [:dminor]]

Oops, linked the wrong try job: https://treeherder.mozilla.org/#/jobs?repo=try&revision=ab5648b64a8a941a7260d1c193f01076942e1ad0

Comment 4

[Wesley Huang [:wesley_huang] (EPM) (NI me)]

Blake, will media team triage this?

Comment 5

[Blake Wu [:bwu][:blakewu]]

We (TPE media team) don't triage this. It looks like related to WebRTC's signaling part.

Comment 6

[Nils Ohlmeier [:drno]]

Martin, do you have any idea what could cause these test failure messages:

Error in test execution: IdpError: Load failed: 2153398266

Comment 7

[Martin Thomson [:mt:]]

I'd be more concerned about this:

5 INFO TEST-UNEXPECTED-FAIL | dom/media/tests/mochitest/identity/test_fingerprints.html | error in test: Promise rejection value is a non-unwrappable cross-compartment wrapper.

That message means that the real error is being hidden.  We'd want to add some logging to determine what that is.

The number appears to be an nsresult value, but I can't find its definition anywhere.

Comment 8

[Bob Clary [:bc] (inactive)]

Attachment: bug-1361325-autophone-disable-Mdm4-nexus-6.patch

The Mdm4 tests are completely disabled in the in-tree manifests for Android 5.1 which causes them to be perma orange on mozilla-central. This patch changes the autophone manifests so the test is no longer scheduled for Android 5.1 on mozilla-central. It will still be available on try.

Comment 9

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8865864 [details] [diff] [review]
bug-1361325-autophone-disable-Mdm4-nexus-6.patch

Review of attachment 8865864 [details] [diff] [review]:
-----------------------------------------------------------------

lgtm

Comment 10

[Bob Clary [:bc] (inactive)]

https://github.com/mozilla/autophone/commit/b711351ad2bb06746773138b18d7a594b05d2446

leaving open to resolve the real issue.

Comment 11

[Bob Clary [:bc] (inactive)]

https://github.com/mozilla/autophone/commit/8c7e0b9e9f43033d5f282242022f93f026795754

I missed debug Mdm4 for Nexus 6. r=self.

Comment 12

[Intermittent Failures Robot]

14 failures in 879 pushes (0.016 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-central: 14

Platform breakdown:
* android-5-1-armv7-api15: 14

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1361325&startday=2017-05-08&endday=2017-05-14&tree=all

Comment 13

[Joel Maher ( :jmaher ) (UTC -8)]

can we mark this as closed?

Comment 14

[Dan Minor [:dminor]]

(In reply to Joel Maher ( :jmaher) from comment #13)
> can we mark this as closed?

I think we should leave this open as it appears there is a real issue here that requires investigation, as per comment 7.

Comment 15

[Bulk Bug Changes for mreavy's org]

Mass change P2->P3 to align with new Mozilla triage process.

Comment 16

[BugBot [:suhaib / :marco/ :calixte]]

There's a r+ patch which didn't land and no activity in this bug for 2 weeks.
:bc, could you have a look please?

Comment 17

[Bob Clary [:bc] (inactive)]

This patch did land but in the autophone github repo. Autophone no longer exists and we do not run the mdm4 tests on android-hw. I'll defer to dminor as to the resolution considering his comment 14.

Comment 18

[Dan Minor [:dminor]]

I think we can mark this as FIXED (since the patch landed on autophone) and open a new bug in the future if necessary.