Closed Bug 1261270 Opened 4 years ago Closed 4 years ago

Various Android robocop tests are going to permafail when Gecko 48 merges to Aurora

Categories

(Firefox for Android :: Testing, defect)

Unspecified
Android
defect
Not set

Tracking

()

RESOLVED FIXED
Firefox 48
Tracking Status
firefox46 --- unaffected
firefox47 --- unaffected
firefox48 --- fixed

People

(Reporter: RyanVM, Assigned: RyanVM)

References

Details

Attachments

(1 file)

[Tracking Requested - why for this release]: Android robocop permafails when Gecko 48 merges to Aurora.

Geoff, any idea what might be going on here?

https://treeherder.mozilla.org/logviewer.html#?job_id=18725473&repo=try
https://treeherder.mozilla.org/logviewer.html#?job_id=18725437&repo=try
Flags: needinfo?(gbrown)
Sorry, not really. 

These tests were failing for a long time; I recently enabled them when I noticed they were passing - bug 1144874 - but I have no idea what changed, or why they fail on Aurora.
Flags: needinfo?(gbrown)
Could APZ be held behind a nightly flag?
(In reply to Michael Comella (:mcomella) from comment #2)
> Could APZ be held behind a nightly flag?

Good idea! http://hg.mozilla.org/mozilla-central/annotate/c40c0b2f3b4c/mobile/android/confvars.sh#l100
I verified that testAxisLocking and testPanCorrectness fail on 48 if APZ is disabled.

What needs to happen here? Can those tests be disabled on Aurora after the merge, or do the tests need to be disabled on trunk until APZ rides the trains?
I'll double-check, but I think mozinfo actually has nightly_build in it, so we can skip it that way. Though a fails-if = !nightly_build is probably better so we'll remember to re-enable the tests when APZ is ready to ride the trains.
Turns out mozinfo has a release_build option, but not nightly_build. Seems like we could easily add it, though. Which I think is a reasonable option as long as we use it with a fails-if condition so the annotation doesn't stay there in perpetuity.
Excitingly, while fail-if is perfectly valid syntax for regular mochitests, it apparently isn't for robocop. My patch with |fail-if = !nightly_build| doesn't work, but skip-if does.
Going off https://hg.mozilla.org/mozilla-central/rev/102371151dc6, I guess that's expected though. *sigh*
Looks like this is the best we can do here. I've confirmed that this makes robocop green on Try simulations.
Assignee: nobody → ryanvm
Status: NEW → ASSIGNED
Attachment #8738358 - Flags: review?(gbrown)
See Also: → 1262433
Comment on attachment 8738358 [details] [diff] [review]
skip the 3 failing tests on !nightly_build

Review of attachment 8738358 [details] [diff] [review]:
-----------------------------------------------------------------

Thanks! I'll look into the robocop fail-if situation; for now, this is the best we can do.
Attachment #8738358 - Flags: review?(gbrown) → review+
https://hg.mozilla.org/mozilla-central/rev/86c187556f7a
Status: ASSIGNED → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 48
Looks like bug 1265418 removed the nightly ifdefs for APZ.
Flags: needinfo?(gbrown)
Yes. I think we can back out the test skipping.
Flags: needinfo?(gbrown)
You need to log in before you can comment on or make changes to this bug.