917543 - Add a robocop.ini manifest for x86

Status: RESOLVED FIXED

(Firefox for Android Graveyard :: General, defect)

Description

[Geoff Brown [:gbrown]]

I have been hoping that we could use a single manifest for robocop, but we have a couple of tests crashing on x86 only and at least one intermittent failure that occurs much more frequently on x86. I imagine the crashes will be sorted out in a few weeks (delayed because we cannot currently get crash dumps) but differing frequencies of intermittent failures could be a long-term issue.

Comment 1

[Geoff Brown [:gbrown]]

Attachment: add robocop_x86.ini

Adds the x86-specific robocop.ini file, with x86-failing tests disabled.

$ diff robocop.ini robocop_x86.ini
5c5
< [testBookmarkKeyword]
---
> # [testBookmarkKeyword] # frequent fail on x86 only; see bug 915350
36c36
< [testSystemPages]
---
> # [testSystemPages] # crash on x86 only; see bug 907383
40c40
< [testFindInPage]
---
> # [testFindInPage] # crash on x86 only; see bug 907303

Once this lands, we can update the mozharness x86 script to point to the new manifest and turn robocop green.

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

can we skip these via the manifest support instead?

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 806270 [details] [diff] [review]
add robocop_x86.ini

Review of attachment 806270 [details] [diff] [review]:
-----------------------------------------------------------------

I see nothing wrong with this manifest.

Comment 4

[Geoff Brown [:gbrown]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/eba531854f3a

Comment 5

[Geoff Brown [:gbrown]]

(In reply to Joel Maher (:jmaher) from comment #2)
> can we skip these via the manifest support instead?

Sorry, I missed this earlier...and I'm not sure I understand.

afaik, the robocop manifest does not allow for skip-if/fail-if.

Comment 6

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/eba531854f3a

Comment 7

[Geoff Brown [:gbrown]]

Attachment: use robocop_x86.ini

Comment 8

[Armen [:armenzg]]

Comment on attachment 806805 [details] [diff] [review]
use robocop_x86.ini

Review of attachment 806805 [details] [diff] [review]:
-----------------------------------------------------------------

lgtm!
After you land this, if a Cedar job is not using your revision of mozharness you can just re-trigger.

Comment 9

[Geoff Brown [:gbrown]]

https://hg.mozilla.org/build/mozharness/rev/de5fafe78778

Comment 10

[Geoff Brown [:gbrown]]

I forgot to update the Makefiles.

https://tbpl.mozilla.org/php/getParsedLog.php?id=28054773&tree=Cedar&full=1

13:23:08     INFO - Mochi-Remote ERROR | Unable to find specified robocop .ini manifest 'robocop_x86.ini'
13:23:08     INFO - Mochi-Remote ERROR | Invalid options specified, use --help for a list of valid options

Comment 11

[Geoff Brown [:gbrown]]

Attachment: add new manifest to makefiles

Build at http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/[e-mail]-c7ed8ed497e5/try-android-x86/fennec-27.0a1.en-US.android-i386.tests.zip verifies that robocop_x86.ini is added to the tests.zip -- my main motivation here.

Comment 12

[Gregory Szorc [:gps]]

Comment on attachment 806978 [details] [diff] [review]
add new manifest to makefiles

Review of attachment 806978 [details] [diff] [review]:
-----------------------------------------------------------------

You don't need my review to add files to existing lists.

Comment 13

[Geoff Brown [:gbrown]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/0172bbcf8903

Comment 14

[Ed Morley [:emorley]]

https://hg.mozilla.org/mozilla-central/rev/0172bbcf8903

Comment 15

[Geoff Brown [:gbrown]]

This hasn't really worked out. It is easy to update either robocop.ini or robocop_x86.ini and forget to update the other one. 

Now Robocop understands skip-if in manifests -- let's use that feature instead and eliminate robocop_x86.ini

Comment 16

[Geoff Brown [:gbrown]]

Attachment: update mozharness to use robocop.ini instead of robocop_x86.ini

Comment 17

[Geoff Brown [:gbrown]]

Attachment: remove robocop_x86.ini and use skip-if in robocop.ini

Comment 18

[Geoff Brown [:gbrown]]

https://tbpl.mozilla.org/?tree=Try&rev=b281f7c3f0b2

Comment 19

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 815917 [details] [diff] [review]
remove robocop_x86.ini and use skip-if in robocop.ini

Review of attachment 815917 [details] [diff] [review]:
-----------------------------------------------------------------

oh this makes me happy

Comment 20

[Geoff Brown [:gbrown]]

https://hg.mozilla.org/build/mozharness/rev/b4cabe359fad

I want to generate some test failures (like testSystemPages) on Cedar/x86...then I'll push the m-c changes to green up robocop on x86 again.

Comment 21

[Justin Wood (:Callek)]

something here is in mozharness production

Comment 22

[Geoff Brown [:gbrown]]

remove robocop_x86.ini:

https://hg.mozilla.org/integration/mozilla-inbound/rev/a03b6f30006f

Comment 23

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/a03b6f30006f