1466077 - Mac coverage build is scheduled despite run-on-projects only containing "try"

Status: RESOLVED FIXED

(Testing :: Code Coverage, enhancement)

Description

[Marco Castelluccio [:marco]]

https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=9900cebb1f9000bd05731ba67736b7c51f7eb812.

It's OK if it's just the build, but we definitely shouldn't schedule the tests too.

Comment 1

[Marco Castelluccio [:marco]]

Attachment: Disable test-verify on Mac coverage builds

This will make the build stop running in mozilla-inbound.
I don't know if it will fix mozilla-central too.

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8982498 [details] [diff] [review]
Disable test-verify on Mac coverage builds

Review of attachment 8982498 [details] [diff] [review]:
-----------------------------------------------------------------

nice, and this keeps this uniform between all coverage types right now:)

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/92f6a3f162e7
Don't schedule test-verify on any ccov build. r=jmaher

Comment 4

[Marco Castelluccio [:marco]]

Attachment: run_on_projects

I think the bug for mozilla-central is that we're always overwriting the "run-on-projects" with "['mozilla-central', 'try']". This means that even if Mac had "['try']" as "run-on-projects", we were overwriting it with "['mozilla-central', 'try']".

With this patch, we will overwrite "run-on-projects" of the test definitions with "built-projects". As far as I undestand, this means that they will inherit the "run-on-projects" from the build defintions.

After this patch, we can no longer select mac ccov from the "mach try fuzzy" list, but we will need to use "mach try fuzzy --full".

The other solution would be to replace:
>            # Ensure we don't run on inbound/autoland/beta, but if the test is try only, ignore it
>            if 'mozilla-central' in test['run-on-projects'] or \
>                    test['run-on-projects'] == 'built-projects':
>                test['run-on-projects'] = ['mozilla-central', 'try']

with:
>            # Ensure we don't run on inbound/autoland/beta, but if the test is try only, ignore it
>            if 'mozilla-central' in test['run-on-projects'] or \
>                    test['run-on-projects'] == 'built-projects':
>                if 'mac' in test['build-platform']:
>                    test['run-on-projects'] = ['try']
>                else:
>                    test['run-on-projects'] = ['mozilla-central', 'try']

Shorter, but it keeps adding special cases to the enable_code_coverage transform, while the proposed solution removes some special cases.

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8982516 [details] [diff] [review]
run_on_projects

Review of attachment 8982516 [details] [diff] [review]:
-----------------------------------------------------------------

thanks for doing this.  I am fine with requiring --full for osx;  Once we get further with coverage and using the data to help us we can maybe find a "reduced set of tests per platform" that can help us gather full coverage while not exhausting resources.

The only unknown is using built-projects, I think that will work well, but it could lend a hand towards mozilla-beta, mozilla-inbound, autoland.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/c78a67ef0566
Make all ccov test suites inherit the run-on-projects from their ccov build. r=jmaher

Comment 7

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/92f6a3f162e7

Comment 8

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/c78a67ef0566

Comment 9

[Marco Castelluccio [:marco]]

Looks like it's working!