1362010 - Figure out why jittest suite is not run in the linux64-ccov scheduled build

Status: RESOLVED FIXED

(Testing :: Code Coverage, defect)

Description

[Marco Castelluccio [:marco]]

They were run on the try build from bug 1314305.

Comment 1

[Marco Castelluccio [:marco]]

For some reason, it was run today: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=0b255199db9d6a6f189b89b7906f99155bde3726&filter-searchStr=ccov.
Until yesterday, it wasn't run: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=d7e40bb852ea047a0e5f530bcdc04d29a1765001&filter-searchStr=ccov.

Comment 2

[Greg Mierzwinski [:sparky]]

Joel, do you know if there is another bug for this? It's not running on any of the linux builds (32,64,pgo,etc.) on mozilla-central.

Comment 3

[Greg Mierzwinski [:sparky]]

Jsreftest is another test suite that should also be running but is not.

Comment 4

[Marco Castelluccio [:marco]]

It's quite strange as in try builds they are run (e.g. https://treeherder.mozilla.org/#/jobs?repo=try&revision=cc4406472ddec2910ca157ae4cabc838898526e1).

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

there is an explanation for this:
https://dxr.mozilla.org/mozilla-central/source/taskcluster/ci/test/tests.yml

both jittest and jsreftest have a when clause:
    when:
        files-changed:
            - js/src/**
            - js/public/**

so that explains why sometimes they are not run on any platforms.  Now you will see them run all the time for windows, but not all the time on linux.  We schedule all linux and osx through taskcluster where the 'when' clause will affect scheduling.

How can we avoid the 'when' clause for ccov runs?

Comment 6

[Greg Mierzwinski [:sparky]]

Thanks!

Here's a parameter that I think we could use to disable task optimization: https://dxr.mozilla.org/mozilla-central/source/taskcluster/taskgraph/parameters.py#25

The file that takes care of these optimizations is: https://dxr.mozilla.org/mozilla-central/source/taskcluster/taskgraph/optimize.py#236

I'll can start working on a patch for this.

Comment 7

[Greg Mierzwinski [:sparky]]

Here's a patch that I think will work: https://hg.mozilla.org/try/rev/0eefd6664758a24e5ad9f530cfb9406897476c57

It's a bit hard to test on try though since they are not disabled there (only on m-c). But the 'task-graph.json' changed and no longer has the 'files-changed' optimization. [1] is the new graph and [2] is the old one.

Can we try this patch? 

[1]: https://public-artifacts.taskcluster.net/WHjaDzZMS-GvCq_66eO_3w/0/public/task-graph.json
[2]: https://public-artifacts.taskcluster.net/dfL72MLqQhic7-SKcqZFMA/0/public/task-graph.json

Comment 8

[Greg Mierzwinski [:sparky]]

Attachment: Bug 1362010 - Allow all test suites to run regardless of 'files-changed' on linux64-ccov.

This patch will allow jittest, jsreftest, and any other test suite to run on linux64-ccov regardless of whether or not the files in 'files-changed' have been matched with changes in a patch.

Review commit: https://reviewboard.mozilla.org/r/143306/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/143306/

Comment 9

[Greg Mierzwinski [:sparky]]

From comment 7, [1] & [2] give an example of what changes in the task-graph. In this case, the 'files-changed' optimization is removed.

Comment 10

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8871831 [details]
Bug 1362010 - Allow all test suites to run regardless of 'files-changed' on linux64-ccov.

https://reviewboard.mozilla.org/r/143306/#review147032

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c0f923fcce01
Allow all test suites to run regardless of 'files-changed' on linux64-ccov. r=jmaher

Comment 12

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/c0f923fcce01