Schedule code coverage to run 4x per day (up from 1x per day)

RESOLVED FIXED in Firefox 56

Status

enhancement
RESOLVED FIXED
2 years ago
2 years ago

People

(Reporter: ekyle, Assigned: sparky)

Tracking

Trunk
mozilla56
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox56 fixed)

Details

Attachments

(1 attachment)

Please run coverage runs 4x per day.

We may want to consider when is best to schedule these jobs:

* We could schedule the jobs so they land during non-peak times.
* We could schedule the coverage to best quadrisect the pushes
Kyle, would you know of how we could easily count the jobs over time? Is there already a tool in place for this?

Also, is it possible to make a cron job run after X number of commits to m-c and while restricting when it can start up?
we currently use SETA to run every 5th push from taskcluster:
https://dxr.mozilla.org/mozilla-central/source/taskcluster/taskgraph/optimize.py#210

and the code that checks for a 5th push inside of is_low_value:
https://dxr.mozilla.org/mozilla-central/source/taskcluster/taskgraph/util/seta.py#177

we could do something similar for the mozilla-central branch, run every X pushes or Y minutes.  Maybe every 5th push, or every 300 minutes.
Oh nice, thanks for the links. So, based on what I see there it is either one way or the other, and there is no mixing that could restrict the times it runs at while also making it run after X pushes or Y minutes. Is that correct, or did I miss something?
I outlined one option that can help us get more coverage when we have higher load.

Another option is use the cronjob that we currently have:
https://dxr.mozilla.org/mozilla-central/source/.cron.yml#69

we can just put this on a schedule and it will not care about load, it will just run.  The big concern here is that we will get duplicate runs on the same revision if we go a larger timespan with no new commits.
I will add some push counts by tod and maybe total load by tod
Flags: needinfo?(klahnakoski)
The coverage is run on central, and pushes to central are rare and irregular.  I suggest triggering a coverage run upon every push/merge to central.
Flags: needinfo?(klahnakoski)
that would give us much better resolution than once/day and we can see more patterns on intermittent failures/etc.
Great! And when we implement this, should the cron job be removed so that there is no interference with this task? https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=2a63a6c35033b5cbc6c98cabc7657c7290284691

I ask because I am wondering if SETA counts that run as a push.
SETA is only valid on autoland and inbound, we run everything on mozilla-central.
Ok, great. So what I see this task broken down to is to make a new optimization transform, based on the SETA optimization implementation of is_low_value_task, but specifically for the mozilla-central project and only for linux64-ccov.

Other than that, we could modify the opt_seta function to work for m-c as well, but only allow linux64-ccov to go through when m-c is the project. This would involve modifying the is_low_value_task function.

What do you think?
we don't want SETA on mozilla-central.  Is your goal to use the SETA optimization logic for every 5th push?  Possibly we can duplicate that.  I think the volume is low enough on m-c that we could run all linux64-ccov on every m-c push and be ok.  That would mean removing the cronjob and setting up the linux64-ccov builds in taskcluster configs to be default for mozilla-central
Yes, that was the goal. I was thinking that it would be useful to have if we want to change from every push to something like every other push. But if we don't need to have the ability to change it, then what you mentioned will be a better fix. Changing the 'run-on-projects' flag to mozilla-central should be enough to get it through.
I have a patch ready. I can't test this patch on mozilla-central, but it worked on try. In [1], the test is scheduled when the run-on-projects flag is set to 'all' which includes 'try'. (I didn't change the flag for linux64-jsdcov). Then in [2], I set it to 'mozilla-central', and the build and tests were not scheduled. So after this patch is applied, all pushes to mozilla-central should have ccov running alongside them (assuming they run with '-p all -u all' or some other combinations using the '-p all').

Want me to submit the patch for review? I'll remove the cron specifications at the same time.

[1]: https://treeherder.mozilla.org/#/jobs?repo=try&revision=2f1e4f6ad88a20e0b5aff0979b2d70b64f920171
[2]: https://treeherder.mozilla.org/#/jobs?repo=try&revision=445eb9b0e95492a816cd4025fee6423b04bbe479
Comment on attachment 8877243 [details]
Bug 1369908 - Schedule linux64-ccov, and linux64-jsdcov on all mozilla-central pushes.

https://reviewboard.mozilla.org/r/148596/#review153024

good stuff!
Attachment #8877243 - Flags: review?(jmaher) → review+
Pushed by jmaher@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/8cbbafc26747
Schedule linux64-ccov, and linux64-jsdcov on all mozilla-central pushes. r=jmaher
Assignee: nobody → gmierz2
https://hg.mozilla.org/mozilla-central/rev/8cbbafc26747
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla56
You need to log in before you can comment on or make changes to this bug.