Open Bug 1342230 Opened 3 years ago Updated 2 years ago

Move testing/xpcshell/selftest.py out of 'make check' and into a separate task

Categories

(Firefox Build System :: General, defect)

defect
Not set

Tracking

(Not tracked)

People

(Reporter: mshal, Unassigned)

References

(Blocks 1 open bug)

Details

If 'make check' is enabled in cross-OSX builds (bug 1340698), testing/xpcshell/selftest.py tries to run despite being listed with 'skip-if = os == "mac"' in the manifest (it is skipped due to bug 1255588). I'm not sure why this isn't skipped in a cross-build, since mozinfo.json still defines os as "mac" in that case.
ahal and I were talking about selftest.py. It shouldn't be incredibly hard to run as a standalone job, it just needs access to the build + the xpcshell test package. My thought was just to stick selftest.py in the xpcshell test package and then run it like it was any other xpcshell job, but ahal thought that would be complicated because of the way the mozharness scripts are written. His idea was to run it as one of the new Python test tasks that the mozbase tests are using, but it would need to also depend on the build task.
I'm about to land the ability to depend on a build task in bug 1336559. Should be in by early next week. I haven't tested pulling these tasks out on OSX yet though. It might *just work* or there might be some taskcluster changes required.
Depends on: 1336559
The weird part to me was that they run at all on a cross build with 'make check' enabled, since it is supposed to be disabled on mac. But since we're not going to enable 'make check' there anyway I don't think this will block us. Still, it's great that we'll have a solution ready in the event that we update our builders to 10.9+ and want to re-enable selftest.py.
I don't think the `python-test` command is loading the generated mozinfo.json file at all, so it's getting the mozinfo defaults, where `os` is the host OS:
https://dxr.mozilla.org/mozilla-central/rev/5069348353f8fc1121e632e3208da33900627214/python/mach_commands.py#50
I did a bit of digging into what would be involved with re-using run-task for this and posted my findings here:
https://bugzilla.mozilla.org/show_bug.cgi?id=1337903#c2

I think jgriffin mentioned that we won't have any buildbot hardware in the new data centre. And as far as I can tell, the in-tree taskcluster config support for the "test kind" (traditional mozharness jobs) hasn't been implemented on OSX either. So even going the mozharness route would require fiddling with taskcluster configs.. Given that, I'd recommend just focusing on the "run-task" route.
Since we're not planning to turn 'make check' on in cross-OSX builds, this no longer blocks the OSX Taskcluster work. However, it is still useful to pull this out into a separate task for performance reasons, so I'm renaming this bug.
No longer blocks: 1340698
Summary: testing/xpcshell/selftest.py tries to run on cross-OSX builds → Move testing/xpcshell/selftest.py out of 'make check' and into a separate task
ahal has some patches in bug 1048446 that will be useful for this as well (specifically, downloading & extracting the packages for these kinds of tests, and a better way of conditionally including the test manifest). So once those are done this bug should be pretty straightforward.
Depends on: 1048446
No longer blocks: 1337903
Depends on: 1337903
Product: Core → Firefox Build System
You need to log in before you can comment on or make changes to this bug.