Closed Bug 1353296 Opened 3 years ago Closed 2 years ago

taskgraph: get rid of filter_upload_symbols()

Categories

(Firefox Build System :: Task Configuration, task, P2)

Tracking

(firefox62 fixed)

RESOLVED FIXED
mozilla62
Tracking Status
firefox62 --- fixed

People

(Reporter: jlorenzo, Assigned: ted)

References

Details

Attachments

(1 file)

In bug 1352477, I introduced a clumsy filter[1], which helped maintaining the parity of generated tasks. Like aki suggested[1], it's probably better to rely on the nightly attribute, instead. 

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1352477#c6
Priority: -- → P2
Please also don't use substring matching on labels.
Assignee: nobody → ted
Component: Release Automation → Task Configuration
Product: Release Engineering → Firefox Build System
Comment on attachment 8967132 [details]
bug 1353296 - get rid of filter_upload_symbols.

https://reviewboard.mozilla.org/r/235776/#review241586


Code analysis found 1 defect in this patch:
 - 1 defect found by mozlint

You can run this analysis locally with:
 - `./mach lint path/to/file` (JS/Python)


If you see a problem in this automated review, please report it here: http://bit.ly/2y9N9Vx


::: taskcluster/taskgraph/target_tasks.py:36
(Diff revision 1)
> -    return '-upload-symbols' not in task.label or \
> -        task.attributes.get('nightly') or \
> -        parameters.get('project') in ('mozilla-beta', 'mozilla-release')
> -
> -
>  def filter_beta_release_tasks(task, parameters, ignore_kinds=None, allow_l10n=False):

Error: Expected 2 blank lines, found 1 [flake8: E302]
Diffing the output of `mach taskgraph tasks` before and after this patch shows that the only difference is the removal of the build-android-api-16/opt-upload-symbols, build-linux64/debug-upload-symbols, and build-linux64/opt-upload-symbols tasks (the non-nightly build types that were listed), which were being filtered out by `filter_upload_symbols` anyway:
https://gist.github.com/luser/e67d5fc38df32a664fad06d42b4eed8c

I went one step further and used aki's taskgraph-diff scripts (https://hg.mozilla.org/build/braindump/file/default/taskcluster/taskgraph-diff/) to diff the generated task graph from parameters.yml files from an arbitrary inbound push, a central desktop nightly cron, a central nightly android cron, an arbitrary beta push and an arbitrary release push. The only differences that showed up there were that these non-nightly symbol upload tasks were actually being scheduled on beta/release due to the logic in `filter_upload_symbols`, so this patch fixes that. (Example: https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&selectedJob=170365091 ) I don't think those were ever actually intended to run, and we are running symbol-upload tasks for the actual "nightly" build types on beta/release (Example: https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&filter-searchStr=nightly&selectedJob=170372847 ).
Blocks: 1437577
Comment on attachment 8967132 [details]
bug 1353296 - get rid of filter_upload_symbols.

https://reviewboard.mozilla.org/r/235776/#review241898

modulo the lint (that's cool!)
Attachment #8967132 - Flags: review?(dustin) → review+
https://hg.mozilla.org/mozilla-central/rev/8a03213c74e9
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla62
You need to log in before you can comment on or make changes to this bug.