1132209 - Stop running robocop on Android x86

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Geoff Brown [:gbrown]]

In today's mobile testing meeting, we considered bug 1127916 -- Android x86 S4 is currently hidden, due to excessive robocop failures -- and asserted:
 - we would like *some* x86 tests running and visible to avoid catastrophic x86 failures in releases
 - we are having trouble finding time/people to resolve the robocop failures
 - we don't expect significant differences in robocop (java) behavior between x86 and arm, so robocop is arguably of limited importance to x86.

The Android x86 S4 "set" consists of xpcshell, robocop-1, robocop-2, and robocop-3 test suites. If we stop running robocop-1/2/3, xpcshell tests should continue to run well and then S4 (which will be xpcshell only) can be un-hidden.

Comment 1

[Geoff Brown [:gbrown]]

Attachment: Stop running robocop-1/2/3 in Android x86 set 4

I think this is all that is necessary.

There's other "cleanup" to remove un-used robocop configuration, but maybe we should wait to see how this plays out before cleaning up everything?

Comment 2

[Jordan Lund (:jlund)]

In production: https://hg.mozilla.org/build/buildbot-configs/rev/4db57119f2a1

Comment 3

[Geoff Brown [:gbrown]]

Verified on mozilla-inbound, mozilla-central, mozilla-beta and release: Robocop is no longer running in Android x86 S4.

Comment 4

[Geoff Brown [:gbrown]]

Sorry - I wanted to keep this open, to clean up robocop.ini, update robocop x86-specific bugs, and possibly clean up x86 mozharness configuration.

Comment 5

[Geoff Brown [:gbrown]]

Attachment: clean up robocop.ini

This updates robocop.ini to remove all the references to x86, to make the manifest easier to read.

I would also update all the referenced bugs, noting that we no longer run robocop on x86. Should I close those bugs?

Comment 6

[James Willcox (:snorp) ([e-mail]) (he/him)]

Comment on attachment 8572086 [details] [diff] [review]
clean up robocop.ini

Review of attachment 8572086 [details] [diff] [review]:
-----------------------------------------------------------------

I guess this means we won't try to fix it then? :/

Comment 7

[Geoff Brown [:gbrown]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/e7f734a505ea

Comment 8

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/e7f734a505ea

Comment 9

[Geoff Brown [:gbrown]]

(In reply to Geoff Brown [:gbrown] from comment #5)
> I would also update all the referenced bugs, noting that we no longer run
> robocop on x86. Should I close those bugs?

Updated. I closed a few bugs but many were cross-platform.

Comment 10

[Geoff Brown [:gbrown]]

Attachment: remove robocop references from androidx86 mozharness config

Just to keep it tidy!

Comment 11

[Geoff Brown [:gbrown]]

https://hg.mozilla.org/build/mozharness/rev/6b2923ca6ef6