Closed
Bug 1240184
Opened 8 years ago
Closed 8 years ago
run mozharness tests in the gecko task graph
Categories
(Firefox Build System :: Task Configuration, task)
Firefox Build System
Task Configuration
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: jlund, Assigned: Callek)
References
Details
Attachments
(1 file, 1 obsolete file)
2.84 KB,
patch
|
dustin
:
review+
|
Details | Diff | Splinter Review |
when we put mozharness into the tree we lost github's test automation for mozharness. There are a few places that come to mind where we can add the automation back in: 1) as an hg hook when pushing 2) prior to any mozharness script call 3) in mozharness archiver (relengapi tool for caching subdir archives from large repos) 4) before TC task graph generation my vote is for (4). (3) Archiver is not going to be around for a long time®. (1) is an unknown for me and (2) seems like a waste and messy to check multiple times per checkin. (4) is nice because Taskcluster is the future and most automation jobs should be configured/scheduled through the decision task (mach generate_task_graph). e.g. 1) checkout $gecko && cd testing/mozharness 2) createORsource mozharness/venv; pip install tox 3) run tox 4) if passes: generate_and_submit_tc_graph(); else: fail
Comment 1•8 years ago
|
||
If it is part of the decision job how can we ensure to run the unit tests on all platforms? I would assume there is platform dependent behavior somewhere.
Comment 2•8 years ago
|
||
We should also update the tox.ini to use a version of hg that's supported by version-control-tools. I have some of the extensions on, and tox failed cause hg 2.6 doesn't like them.
Comment 3•8 years ago
|
||
I could do that when trying to fix the regression one of my patches might have been caused via bug 1230150.
Comment 4•8 years ago
|
||
(In reply to Henrik Skupin (:whimboo) from comment #3) > I could do that when trying to fix the regression one of my patches might > have been caused via bug 1230150. Whereby lets make it a separate issue. Feel free to file another bug for it.
Assignee | ||
Comment 5•8 years ago
|
||
fwiw unless mozharness is responsible for running the gecko descision graph generation, I'd argue we should just insert it into the task graph as a dep of everything that uses mozharness (if we don't expect any intermittents from it) or as a side-run thing if there are intermittents from it.
Reporter | ||
Comment 6•8 years ago
|
||
(In reply to Henrik Skupin (:whimboo) from comment #1) > If it is part of the decision job how can we ensure to run the unit tests on > all platforms? I would assume there is platform dependent behavior somewhere. we wouldn't but (1) at least the mh tests failing would get caught and (2) eventually all jobs across all platforms will be through this decision task because it is the futures. (In reply to Justin Wood (:Callek) from comment #5) > fwiw unless mozharness is responsible for running the gecko descision graph > generation, I'd argue we should just insert it into the task graph as a dep > of everything that uses mozharness (if we don't expect any intermittents > from it) or as a side-run thing if there are intermittents from it. sure, valid point.
Comment 7•8 years ago
|
||
I think we should just schedule this as a task in the task graph. I'm not even sure it should be a prerequisite -- that would just increase our e2e time. This would mix nicely with the mach unit tests, for example. And also with the ESLint stuff that's being set up -- tasks that can run straight from the source and don't require or produce a binary.
Summary: gecko decision task should run mozharness unittests prior to task graph generation → gecko decision task should run early in the task graph
Updated•8 years ago
|
Assignee: nobody → dustin
Updated•8 years ago
|
Assignee: dustin → nobody
Summary: gecko decision task should run early in the task graph → run mozharness tests in the gecko task graph
Updated•8 years ago
|
Assignee | ||
Comment 8•8 years ago
|
||
Scratching an itch now that I'm back. Looking for anything that would block this landing. https://treeherder.mozilla.org/#/jobs?repo=try&revision=0593907de955 is my try push... When this lands the file_patterns will be `testing/mozharness/**` to be clear.
Assignee: nobody → bugspam.Callek
Status: NEW → ASSIGNED
Attachment #8733105 -
Flags: feedback?(jlund)
Attachment #8733105 -
Flags: feedback?(dustin)
Comment 9•8 years ago
|
||
Cool! Btw. from where do we install the additional python packages? Their download looks pretty slow so I wonder if that could be from pypi and not from our internal mirror.
Comment 10•8 years ago
|
||
Comment on attachment 8733105 [details] [diff] [review] [mc] mozharness tox Review of attachment 8733105 [details] [diff] [review]: ----------------------------------------------------------------- ::: testing/taskcluster/tasks/tests/mozharness-gecko.yml @@ +7,5 @@ > + name: '[TC] - Mozharness Tox' > + description: 'Mozharness integration tests' > + > + scopes: > + - 'docker-worker:cache:level-{{level}}-{{project}}-dotcache' You don't use this cache, so you don't need this scope @@ +13,5 @@ > + payload: > + image: > + type: 'task-image' > + path: 'public/image.tar' > + taskId: '{{#task_id_for_image}}tester{{/task_id_for_image}}' "tester" is a b2g image, and may go away or change unexpectedly. Better to use desktop-test. @@ +16,5 @@ > + path: 'public/image.tar' > + taskId: '{{#task_id_for_image}}tester{{/task_id_for_image}}' > + > + cache: > + level-{{level}}-{{project}}-tc-vcs: '/home/worker/.tc-vcs' This cache is already included in lint.yml and need not be repeated here
Attachment #8733105 -
Flags: feedback?(dustin) → feedback+
Reporter | ||
Comment 11•8 years ago
|
||
Comment on attachment 8733105 [details] [diff] [review] [mc] mozharness tox Review of attachment 8733105 [details] [diff] [review]: ----------------------------------------------------------------- outside of dustins comments, it looks good to me. one concern below in line ::: testing/taskcluster/tasks/branches/base_jobs.yml @@ +366,5 @@ > + root: true > + when: > + file_patterns: > + # Taskcluster stuff to trigger my try push > + - 'testing/taskcluster/**' should this be 'testing/mozharness/**' ?
Attachment #8733105 -
Flags: feedback?(jlund) → feedback+
Assignee | ||
Comment 12•8 years ago
|
||
Try push with this, and eslint enabled is: https://treeherder.mozilla.org/#/jobs?repo=try&revision=300113fd744e (I enabled by merely marking testing/taskcluster/** as triggering files)
Attachment #8733105 -
Attachment is obsolete: true
Attachment #8733494 -
Flags: review?(dustin)
Updated•8 years ago
|
Attachment #8733494 -
Flags: review?(dustin) → review+
Assignee | ||
Comment 13•8 years ago
|
||
https://hg.mozilla.org/integration/mozilla-inbound/rev/93c1fd856330
Status: ASSIGNED → NEW
Comment 14•8 years ago
|
||
Following this mostly as a learning exercise for myself, and wondering: Should the task also run when the task definitions change? i.e., any of the 4 files here at least? Also spotted workerType: b2gtest in lint.yml, unrelated to this patch, but probably wants a follow-up, re comment 10?
Assignee | ||
Comment 15•8 years ago
|
||
(In reply to Axel Hecht [:Pike] from comment #14) > Following this mostly as a learning exercise for myself, and wondering: > > Should the task also run when the task definitions change? i.e., any of the > 4 files here at least? Probably a good idea (I just didn't add them to avoid extra complexity, since the other files: ones didn't do that either) > Also spotted > workerType: b2gtest > in lint.yml, unrelated to this patch, but probably wants a follow-up, re > comment 10? I suspect a blanket audit of "what uses b2gtest" and then replacing with a better choice is a good idea, however I don't actually know what other worker types exist to know what is more appropriate atm, so I left it alone.
Comment 16•8 years ago
|
||
b2gtest will be fine for now -- it's easy to do an 'hg grep' (and get frustrated that it doesn't work and instead use `grep -r` instead) to find uses of b2gtest when we want to turn it off. As we build out the new task-optimization system in the in-tree configuration system, it will be more careful to cover the task definitions as well.
Comment 17•8 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/93c1fd856330
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Updated•6 years ago
|
Product: TaskCluster → Firefox Build System
You need to log in
before you can comment on or make changes to this bug.
Description
•