Nightly valgrind testing rebuilds and clobbers index for the linux64 build job.

RESOLVED FIXED in Firefox 68

Status

defect
RESOLVED FIXED
11 months ago
4 months ago

People

(Reporter: Callek, Assigned: tomprince)

Tracking

Trunk
mozilla68

Firefox Tracking Flags

(firefox68 fixed)

Details

Attachments

(5 attachments)

In doing unrelated work [1] I found that we rebuild linux64 opt every time we do valgrind testing... 

e.g. https://tools.taskcluster.net/groups/Dcy_LEynSemmeIpMODLhOA was found for push id 34555 which corresponds to https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=8284cddccf78566ca4dc45272940ccae4b1150df

The target tasks only show the mochitest valgrinds...

[1] - https://github.com/mozilla-releng/measuring_ci/issues/33
jmaher, can you clarify our intentions for mochitest-valgrind ?

There's a valgrind-linux64-valgrind/opt job on each m-c push [1], which runs ./mach valgrind-test via a mozharness action, and sets taskcluster routes which don't interfere with the main Linux64 opt (build-linux64/opt) [2]. 

The periodic Vg cron creates a second build-linux64/opt [3], which the mochitest-valgrind tests download and use. The bug here is that the build overwrites some routes of the on-push build, including [4].  

So I'm wondering if the mochitest-valgrind tests are using the right build or no. If they are it seems like there's some task graph generation issue where the previous opt build isn't used.

aside: anyone looking on treeherder should enable tier3 to see the mochitest-valgrind tests [5]

[1] https://tools.taskcluster.net/groups/G8qhxHILS26mCvgdY_8tFg/tasks/M9f_LAe5QX-v19Bhg35hSQ/details
[2] https://tools.taskcluster.net/groups/G8qhxHILS26mCvgdY_8tFg/tasks/Cix9r4t2Rwea4uibDWFnjw/details
[3] https://tools.taskcluster.net/groups/Dcy_LEynSemmeIpMODLhOA/tasks/aWY8Py8gS2WYU2eSgdJGGw/details
[4] index.gecko.v2.mozilla-central.revision.8284cddccf78566ca4dc45272940ccae4b1150df.firefox.linux64-opt
[5] https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=8284cddccf78566ca4dc45272940ccae4b1150df&filter-tier=1&filter-tier=2&filter-tier=3&filter-searchStr=valgrind&selectedJob=197060448
Flags: needinfo?(jmaher)
If we do want to use the `build-linux64/opt for the mochitest-valgrind jobs, then we probably want to do something like rel-pro, where the valgrind-cron tasks pulls in the build-linux64-opt build from the on-push graph.
interesting observations.  Valgrind tests should be using a valgrind build.  Our goal is to keep these running regularly, but not per commit.  This is where the cronjob comes in for the build which should then kick off all the tests.

It is much nicer to schedule per-commit on mozilla-central (<10/day) instead of a cronjob.  I don't think there is enough support for doing this which means running only the crontab and the related valgrind test jobs once/day.
Flags: needinfo?(jmaher)
Assignee: nobody → mozilla

This avoids the surprising result of try_task_config.json overriding explicit
--target-tasks-method from a cron task run against a try push.

Pushed by mozilla@hocat.ca:
https://hg.mozilla.org/integration/autoland/rev/a9500cb7d276
[taskgraph] Move some functions for interacting with existing taskgraph to a more generic location; r=aki
https://hg.mozilla.org/integration/autoland/rev/b14a6538211f
[taskgraph] Add options to reuse on-push tasks in cron graphs; r=dustin,aki
https://hg.mozilla.org/integration/autoland/rev/e7778d636a11
[taskgraph] Reuse on-push build in nightly valgrind graphs; r=dustin
Pushed by mozilla@hocat.ca:
https://hg.mozilla.org/integration/autoland/rev/993025739f8d
[taskgraph] Only look for try parameters in on-push decision tasks; r=dustin
https://hg.mozilla.org/integration/autoland/rev/dcfe21ab348e
[taskgraph] Don't notify sherrifs on cron decision task failures on try; r=dustin
You need to log in before you can comment on or make changes to this bug.