Closed Bug 1371924 Opened 7 years ago Closed 7 years ago

[Android] Permaorange in test_execute_script.py TestExecuteContent.test_return_value_on_alert when Gecko 55 merges to beta on 2017-06-12

Categories

(Remote Protocol :: Marionette, defect)

Product:
Remote Protocol
Component:
Marionette
Platform:
Unspecified
Android
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(firefox-esr52 unaffected, firefox53 unaffected, firefox54 unaffected, firefox55+ fixed, firefox56 fixed)

Status:
RESOLVED FIXED
Milestone:
mozilla56
Tracking Flags:
Tracking Status
firefox-esr52 --- unaffected
firefox53 --- unaffected
firefox54 --- unaffected
firefox55 + fixed
firefox56 --- fixed

People

(Reporter: RyanVM, Assigned: RyanVM)

References

Details

[Tracking Requested - why for this release]: Permaorange

Tier 2, so not blocking the merge, but a permafail nonetheless.

https://treeherder.mozilla.org/logviewer.html#?job_id=106008959&repo=try
Hard to say without proper logs but I would assume this is due to bug 1341466.

Geoff, what do you think?
Flags: needinfo?(hskupin) → needinfo?(gbrown)
I do not think this is really bug 1341466. The emulator is generally reliable > 95% of the time, so it seems unlikely that sudden emulator failure is causing a permaorange for this job. Also, I see socket timeouts in the log, rather than adb timeouts.

On the other hand, of course there is a similar issue, in that the job is running > 3600 seconds because it doesn't give up and keeps encountering long-running failures. Since the job times out after 3600 seconds, there are fewer diagnostics available. :(

It would be nice to see a logcat for this -- it might tell us what is happening on the device.
Flags: needinfo?(gbrown)
Ok, so I had another look at this and you are right, Geoff! The problematic test in question here is `test_return_value_on_alert`. And it has been added recently by Andreas on bug 1370850.

In general we do not have any modal/tabmodal dialog test running for Fennec. But the patch added a test for tab modal dialogs which also gets run in Fennec. So this should not have been done. Whereby I still wonder why we do not have a permanent failure on Android for Nightly!

Andreas, can you please disable the test? Thanks.
Blocks: 1370850
Flags: needinfo?(ato)
Summary: Permaorange in test_execute_script.py on Android debug when Gecko 55 merges to beta on 2017-06-12 → [Android] Permaorange in test_execute_script.py TestExecuteContent.test_return_value_on_alert when Gecko 55 merges to beta on 2017-06-12
Per IRC
Assignee: nobody → ryanvm
Flags: needinfo?(ato)
Pushed by ryanvm@gmail.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/def7c269d7af
Skip test_execute_script.py TestExecuteContent.test_return_value_on_alert on Android. r=ato
https://hg.mozilla.org/mozilla-central/rev/def7c269d7af
Status: NEW → RESOLVED
Closed: 7 years ago
status-firefox56: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla56
Product: Testing → Remote Protocol
You need to log in before you can comment on or make changes to this bug.