Closed Bug 1352477 Opened 3 years ago Closed 3 years ago
taskgraph ignores run-on-project for fennec-nightlies
59 bytes, text/x-review-board-request
Bug 1317783 showed that even though push-apk specifies "run-on-project" to exclude mozilla-central, it does get included in the central nightly graph.
Comment on attachment 8853461 [details] Bug 1352477 - taskgraph ignores run-on-project for fennec-nightlies Clearing review flag for now... I provided comments in IRC.
Attachment #8853461 - Flags: review?(aki)
This patch makes sure there is no new job created/deleted, with the exception of jamun, where: > build-linux64-nightly/opt-upload-symbols > build-linux-nightly/opt-upload-symbols are now part of the regular jamun decision task. I didn't added them on purpose, initially. However as jamun should be par with beta, I believe we can keep them. TESTING PROTOCOL 1. Download all the parameters.yml from either treeherder (like ), or from the hooks. In my case, I named them parameters-branch-nightly-platform.yml 2. Before the patch is applied, run: > for name in $(find . -name 'parameters-*.yml' | cut -d'-' -f'2-' | cut -d'.' -f1); do echo "$name" ; ./mach taskgraph target -p parameters-$name.yml > $name-before.list ; done This generates the task lists for every parameters file. You may need to tweak the files from beta/release/jamun because they are outdated. You may also need: > export BUILD_NUMBER=1 3. Apply the patch 4. Run: > for name in $(find . -name 'parameters-*.yml' | cut -d'-' -f'2-' | cut -d'.' -f1); do echo "$name" ; ./mach taskgraph target -p parameters-$name.yml > $name-after.list ; done 5. Concatenate the differences between before and after: > rm -f concatenated.diff && for name in $(find . -name 'parameters-*.yml' | cut -d'-' -f'2-' | cut -d'.' -f1); do echo $name >> meta.diff && diff <(sort $name-before.list) <(sort $name-after.list) >> concatenated.diff ; done 6. Read concatenated.diff. In my case, I have 16 different parameters files, which are: * aurora * aurora-nightly-desktop * aurora-nightly-fennec * beta * beta-candidate-fennec * central * central-nightly-desktop * central-nightly-fennec * date * date-nightly-desktop * date-nightly-fennec * inbound * jamun * jamun-candidate-fennec * release * try Like said at the beginning, the only difference spotted was in jamun. All the other remained intact. As a follow up, we need to hardcode the list of expected tasks in an integration test. I haven't looked into it yet.  https://treeherder.mozilla.org/#/jobs?repo=date  https://tools.taskcluster.net/hooks/#project-releng
(In reply to Johan Lorenzo [:jlorenzo] from comment #4) I also tested this patch on top of bug 1317783, where I could see the push-apk tasks (and only these ones) being filtered out of date-nightly, jamun-candidate, and more importantly: central-nightly.
Comment on attachment 8853461 [details] Bug 1352477 - taskgraph ignores run-on-project for fennec-nightlies https://reviewboard.mozilla.org/r/125554/#review128638 r+ with nits. ::: taskcluster/ci/nightly-l10n/kind.yml:36 (Diff revision 2) > + run-on-projects: > + - mozilla-central > + - mozilla-aurora > + - mozilla-beta > + - date > + - jamun As per an earlier review, I think it's cleaner having the real set of branches land, and then update date and jamun to add them. (It may be messy when we land and back out patches, but I think it'll be cleaner overall.) If we plan on having a post-migration sweep to clean up all the jamun and date references once we're done with them, that works. ::: taskcluster/ci/upload-symbols/job-template.yml:10 (Diff revision 2) > deadline-after: 24 hours > run-on-projects: > - try > - - mozilla-beta > - - mozilla-release > + - release > + - date > + - jamun same here. ::: taskcluster/taskgraph/target_tasks.py:36 (Diff revision 2) > - return False > run_on_projects = set(task.attributes.get('run_on_projects', )) > return match_run_on_projects(parameters['project'], run_on_projects) > > > +def filter_upload_symbols(task, parameters): We really should add the 'nightly' parameter to these rather than have this function. Maybe as a followup?
Attachment #8853461 - Flags: review?(aki) → review+
Comment on attachment 8853461 [details] Bug 1352477 - taskgraph ignores run-on-project for fennec-nightlies https://reviewboard.mozilla.org/r/125554/#review128638 > We really should add the 'nightly' parameter to these rather than have this function. Maybe as a followup? I filed bug 1353296
Pushed by firstname.lastname@example.org: https://hg.mozilla.org/integration/autoland/rev/5533bb014569 taskgraph ignores run-on-project for fennec-nightlies r=aki
I had to uplift this patch to aurora, in order to make bug 1357409 happen: https://hg.mozilla.org/releases/mozilla-aurora/rev/4dae386f2c331851b192a3ef493cc139fd562a44
You need to log in before you can comment on or make changes to this bug.