Closed Bug 1380338 Opened 7 years ago Closed 7 years ago

Convert |mach taskgraph python-tests| to regular |mach python-test|

Categories

(Firefox Build System :: Task Configuration, task)

task
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED
mozilla56

People

(Reporter: ahal, Assigned: ahal)

Details

Attachments

(1 file)

The taskgraph unittests are currently run with:
./mach taskgraph python-test

I think this was mostly just because the |mach python-test| framework wasn't very polished at the time. We might as well convert them over now. This bug means that to run these tests, the new command would be:

./mach python-test taskcluster/taskgraph

(just passing in taskcluster works too, but would run theoretical future tests as well)
Comment on attachment 8885782 [details]
Bug 1380338 - Convert taskgraph unit tests to the |mach python-test| framework,

https://reviewboard.mozilla.org/r/156576/#review161644

Awesome!  Your guess about why this wasn't done to start with is correct.  Is there any need to have a separate "tg" task now, or could these be mixed in with tests of the other in-tree python?
Attachment #8885782 - Flags: review?(dustin) → review+
In theory we could run them all together, though the other python-tests all run in separate tasks as well (with the exception of the tests that still run in `make check`). Keeping it separate also makes the 'files-changed' attribute more accurate.
Pushed by ahalberstadt@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/54e4545b0fe0
Convert taskgraph unit tests to the |mach python-test| framework, r=dustin
Backed out for breaking spidermonkey pkg due to failing to process taskgraph/test/python.ini:

https://hg.mozilla.org/integration/autoland/rev/c272abf614a3213a31e8f11bf407904d82a23965

Push with failure: https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=54e4545b0fe03e103b464011738f60caa5f621a6&filter-resultStatus=testfailed&filter-resultStatus=busted&filter-resultStatus=exception&filter-resultStatus=retry&filter-resultStatus=usercancel&filter-resultStatus=runnable
Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=113699408&repo=autoland

[task 2017-07-12T16:56:39.851003Z] updating cache ./config.cache
[task 2017-07-12T16:56:39.852492Z] creating ./config.data
[task 2017-07-12T16:56:39.916328Z] Creating config.status
[task 2017-07-12T16:56:40.095366Z] Reticulating splines...
[task 2017-07-12T16:56:40.118640Z] Traceback (most recent call last):
[task 2017-07-12T16:56:40.118683Z]   File "/home/worker/workspace/sm-package/mozjs-56.0a1.0/js/src/../../configure.py", line 124, in <module>
[task 2017-07-12T16:56:40.118705Z]     sys.exit(main(sys.argv))
[task 2017-07-12T16:56:40.118733Z]   File "/home/worker/workspace/sm-package/mozjs-56.0a1.0/js/src/../../configure.py", line 34, in main
[task 2017-07-12T16:56:40.118753Z]     return config_status(config)
[task 2017-07-12T16:56:40.118785Z]   File "/home/worker/workspace/sm-package/mozjs-56.0a1.0/js/src/../../configure.py", line 119, in config_status
[task 2017-07-12T16:56:40.118810Z]     return config_status(args=[], **encode(sanitized_config, encoding))
[task 2017-07-12T16:56:40.118844Z]   File "/home/worker/workspace/sm-package/mozjs-56.0a1.0/python/mozbuild/mozbuild/config_status.py", line 147, in config_status
[task 2017-07-12T16:56:40.118863Z]     definitions = list(definitions)
[task 2017-07-12T16:56:40.118894Z]   File "/home/worker/workspace/sm-package/mozjs-56.0a1.0/python/mozbuild/mozbuild/frontend/emitter.py", line 170, in emit
[task 2017-07-12T16:56:40.118910Z]     for out in output:
[task 2017-07-12T16:56:40.118946Z]   File "/home/worker/workspace/sm-package/mozjs-56.0a1.0/python/mozbuild/mozbuild/frontend/reader.py", line 890, in read_topsrcdir
[task 2017-07-12T16:56:40.118977Z]     for r in self.read_mozbuild(path, self.config):
[task 2017-07-12T16:56:40.119014Z]   File "/home/worker/workspace/sm-package/mozjs-56.0a1.0/python/mozbuild/mozbuild/frontend/reader.py", line 1062, in read_mozbuild
[task 2017-07-12T16:56:40.119029Z]     raise bre
[task 2017-07-12T16:56:40.119843Z] mozbuild.frontend.reader.BuildReaderError: 
[task 2017-07-12T16:56:40.119868Z] ==============================
[task 2017-07-12T16:56:40.119882Z] ERROR PROCESSING MOZBUILD FILE
[task 2017-07-12T16:56:40.119895Z] ==============================
[task 2017-07-12T16:56:40.119903Z] 
[task 2017-07-12T16:56:40.119920Z] The error occurred while processing the following file:
[task 2017-07-12T16:56:40.119931Z] 
[task 2017-07-12T16:56:40.119966Z]     /home/worker/workspace/sm-package/mozjs-56.0a1.0/taskcluster/moz.build
[task 2017-07-12T16:56:40.119982Z] 
[task 2017-07-12T16:56:40.119998Z] The error was triggered on line 30 of this file:
[task 2017-07-12T16:56:40.120012Z] 
[task 2017-07-12T16:56:40.120026Z]     'taskgraph/test/python.ini',
[task 2017-07-12T16:56:40.120037Z] 
[task 2017-07-12T16:56:40.120064Z] An error was encountered as part of executing the file itself. The error appears to be the fault of the script.
[task 2017-07-12T16:56:40.120075Z] 
[task 2017-07-12T16:56:40.120090Z] The error as reported by Python is:
[task 2017-07-12T16:56:40.120101Z] 
[task 2017-07-12T16:56:40.120143Z]     ['IOError: Missing files: /home/worker/workspace/sm-package/mozjs-56.0a1.0/taskcluster/taskgraph/test/python.ini\n']
[task 2017-07-12T16:56:40.120159Z] 
[task 2017-07-12T16:56:40.140729Z] Traceback (most recent call last):
[task 2017-07-12T16:56:40.140777Z]   File "./devtools/automation/autospider.py", line 332, in <module>
[task 2017-07-12T16:56:40.140805Z]     run_command(['sh', '-c', posixpath.join(PDIR.js_src, 'configure') + ' ' + CONFIGURE_ARGS], check=True)
[task 2017-07-12T16:56:40.140830Z]   File "./devtools/automation/autospider.py", line 293, in run_command
[task 2017-07-12T16:56:40.140854Z]     raise subprocess.CalledProcessError(status, command, output=stderr)
Flags: needinfo?(ahalberstadt)
I've been bitten by this before, the sm-pkg task is clown shoes. I filed bug 1380383 to prevent backouts like this from happening in the future.

I'll get a fix up shortly.
Flags: needinfo?(ahalberstadt)
Pushed by ahalberstadt@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/77c1ead3ba58
Convert taskgraph unit tests to the |mach python-test| framework, r=dustin
https://hg.mozilla.org/mozilla-central/rev/77c1ead3ba58
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla56
Product: TaskCluster → Firefox Build System
You need to log in before you can comment on or make changes to this bug.