843107 - split robocop (rc) into 2 chunks

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

we need to split robocop into 2 chunks, too many tests!

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: add robocop chunks to android config (0.8)

this is an example of what I think the patch might look like for this.  The problem is we need to only do this on mozilla-central.  I could land the harness support for this on aurora/beta if you would like to make it easier, but this will need to be split for ESR/Release at the very least.

Comment 2

[Kim Moir [:kmoir] ET]

Comment on attachment 716572 [details] [diff] [review]
add robocop chunks to android config (0.8)

ltgm 

We can do something like we did in bug 811723 to only have it apply to specific branches.

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

looking at what is done in bug 811723 and what is currently an active patch on bug 843105, I think we can let this chunk the tests for all branches that we do reftest small.  

I believe this would be:
mozilla-central (and related branches like mozilla-inbound, cedar, ash, etc...)
mozilla-aurora
mozilla-beta
mozilla-release

I would need confirmation that we will not be doing the reftestsmall runs on:
ESR17
ESR10

I can land the harness fixes on aurora and beta, I am not sure about release.  We might have to hack the release branch in buildbot-configs to not run robocop in chunks until the next merge.

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

Attachment: add chunking support for robocop (1.0)

this patch adds robocop chunking support into the configs for m-c/m-a/m-b.  It ignores mozilla-release specifically as the harness code there doesn't support chunking.  

Please let me know if my code which manages the exclusion of mozilla-release is valid, that is the most questionable part of the patch.

Comment 5

[Kim Moir [:kmoir] ET]

Comment on attachment 719479 [details] [diff] [review]
add chunking support for robocop (1.0)

I tested the patch on my dev-master and found some errors that needed to be fixed.  We don't run tests on pandas on the release branch yet so it doesn't apply to that branch.  Also, when you're creating ANDROID_PLAIN_UNITTEST_DICT,  suite[0] matches robocop not mochitest-robocop to exclude them.  Otherwise,you get both sets of robocop tests included.  I'll attach a new patch.

Comment 6

[Kim Moir [:kmoir] ET]

Attachment: patch

Comment 7

[Joel Maher ( :jmaher ) (UTC -8)]

kmoir- thanks for testing this and providing an updated patch!

Comment 8

[Justin Wood (:Callek)]

Comment on attachment 719626 [details] [diff] [review]
patch

Review of attachment 719626 [details] [diff] [review]:
-----------------------------------------------------------------

::: mozilla-tests/mobile_config.py
@@ +558,5 @@
>          del BRANCHES[branch]['platforms']['android']['panda_android']
>          BRANCHES[branch]['platforms']['android']['slave_platforms'] = ['tegra_android']
>  
> +# Ignore robocop chunks for mozilla-release, robocop-chunks is defined in ANDROID_PLAIN_UNITTEST_DICT
> +BRANCHES['mozilla-release']["platforms"]["android"]["tegra_android"]["opt_unittest_suites"] = deepcopy(ANDROID_UNITTEST_DICT["opt_unittest_suites"])

won't this then pull in the non reftestsmall stuff as well?

Comment 9

[Kim Moir [:kmoir] ET]

Yes.  So that was a question I have.  

I was rereading 
https://bugzilla.mozilla.org/show_bug.cgi?id=840229#c5

and thought that comment indicated that the non-reftestssmall stuff should be there for release branches.  But I'm unsure if this is a correct interpretation.

Comment 10

[Joel Maher ( :jmaher ) (UTC -8)]

my patch was not intended to revert to regular reftests.  If it becomes a pain, maybe we could revert to regular reftests for mozilla-release as that is a branch with little to no traffic and next merge day mozilla-release will have the robocop chunking code and work all happy!

Comment 11

[Joel Maher ( :jmaher ) (UTC -8)]

any ideas for how to make this not affect the other tests?

Comment 12

[Kim Moir [:kmoir] ET]

Attachment: add chunking support for robocop (v3.0)

I think this fixes the issue with the reftest plain not being run on tegras in the mozilla-release branch.  I looked good on my dev-master.

Comment 13

[Justin Wood (:Callek)]

Comment on attachment 720083 [details] [diff] [review]
add chunking support for robocop (v3.0)

Review of attachment 720083 [details] [diff] [review]:
-----------------------------------------------------------------

The mere fact that this small patch was incredibly hard to figure out what it was doing is enough to make me rage at this files layout.

Waaaaay too painful to change this mess.

Comment 14

[Kim Moir [:kmoir] ET]

green builds!

Comment 15

[Ed Morley [:emorley]]

Up until now, robocop has been hidden on Panda; this landing changed the buildernames, which caused them to become unhidden (normal for this type of bug) - on inspection, Pandas are failing bug 843947 a lot - but other than that, I think we can actually leave them unhidden after all.

Joel, thoughts?

Comment 16

[Ed Morley [:emorley]]

(Discussed on IRC, have disabled the test and left Panda robocop unhidden, see bug 843947).