1176632 - web-platform tests mach command file adds 100ms to mach load time

Status: RESOLVED FIXED

(Testing :: web-platform-tests, defect)

Description

[Gregory Szorc [:gps]]

If you stop loading testing/web-platform/mach_commands.py, mach loads ~100ms faster. The reason is the wptrunner and update packages pull in a lot of dependencies once everything is said and done:

_ast from testing/web-platform/mach_commands.py
ast from testing/web-platform/mach_commands.py
distutils.debug from testing/web-platform/mach_commands.py
distutils.log from testing/web-platform/mach_commands.py
distutils.spawn from testing/web-platform/mach_commands.py
encodings.ascii from testing/web-platform/mach_commands.py
importlib from testing/web-platform/mach_commands.py
importlib.sys from testing/web-platform/mach_commands.py
mach.commands.049eb330187511e5a2aeb8e85631ff68 from testing/web-platform/mach_commands.py
mach.commands.update from testing/web-platform/mach_commands.py
mach.commands.wptrunner from testing/web-platform/mach_commands.py
mozinfo from testing/web-platform/mach_commands.py
mozinfo.mozinfo from testing/web-platform/mach_commands.py
mozinfo.os from testing/web-platform/mach_commands.py
mozinfo.platform from testing/web-platform/mach_commands.py
mozinfo.re from testing/web-platform/mach_commands.py
mozinfo.sys from testing/web-platform/mach_commands.py
mozlog.structured.logging from testing/web-platform/mach_commands.py
mozlog.structured.stdadapter from testing/web-platform/mach_commands.py
plistlib from testing/web-platform/mach_commands.py
pyexpat from testing/web-platform/mach_commands.py
pyexpat.errors from testing/web-platform/mach_commands.py
pyexpat.model from testing/web-platform/mach_commands.py
update from testing/web-platform/mach_commands.py
update.github from testing/web-platform/mach_commands.py
update.json from testing/web-platform/mach_commands.py
update.mozlog from testing/web-platform/mach_commands.py
update.os from testing/web-platform/mach_commands.py
update.re from testing/web-platform/mach_commands.py
update.subprocess from testing/web-platform/mach_commands.py
update.sys from testing/web-platform/mach_commands.py
update.tempfile from testing/web-platform/mach_commands.py
update.tree from testing/web-platform/mach_commands.py
update.update from testing/web-platform/mach_commands.py
update.updatecommandline from testing/web-platform/mach_commands.py
update.upstream from testing/web-platform/mach_commands.py
update.urlparse from testing/web-platform/mach_commands.py
update.wptrunner from testing/web-platform/mach_commands.py
wptrunner from testing/web-platform/mach_commands.py
wptrunner.ConfigParser from testing/web-platform/mach_commands.py
wptrunner.Queue from testing/web-platform/mach_commands.py
wptrunner.StringIO from testing/web-platform/mach_commands.py
wptrunner.abc from testing/web-platform/mach_commands.py
wptrunner.argparse from testing/web-platform/mach_commands.py
wptrunner.ast from testing/web-platform/mach_commands.py
wptrunner.browsers from testing/web-platform/mach_commands.py
wptrunner.collections from testing/web-platform/mach_commands.py
wptrunner.config from testing/web-platform/mach_commands.py
wptrunner.distutils from testing/web-platform/mach_commands.py
wptrunner.environment from testing/web-platform/mach_commands.py
wptrunner.expected from testing/web-platform/mach_commands.py
wptrunner.functools from testing/web-platform/mach_commands.py
wptrunner.imp from testing/web-platform/mach_commands.py
wptrunner.importlib from testing/web-platform/mach_commands.py
wptrunner.json from testing/web-platform/mach_commands.py
wptrunner.logging from testing/web-platform/mach_commands.py
wptrunner.manifestexpected from testing/web-platform/mach_commands.py
wptrunner.manifestinclude from testing/web-platform/mach_commands.py
wptrunner.manifestupdate from testing/web-platform/mach_commands.py
wptrunner.metadata from testing/web-platform/mach_commands.py
wptrunner.mozinfo from testing/web-platform/mach_commands.py
wptrunner.mozlog from testing/web-platform/mach_commands.py
wptrunner.multiprocessing from testing/web-platform/mach_commands.py
wptrunner.os from testing/web-platform/mach_commands.py
wptrunner.products from testing/web-platform/mach_commands.py
wptrunner.shutil from testing/web-platform/mach_commands.py
wptrunner.signal from testing/web-platform/mach_commands.py
wptrunner.socket from testing/web-platform/mach_commands.py
wptrunner.subprocess from testing/web-platform/mach_commands.py
wptrunner.sys from testing/web-platform/mach_commands.py
wptrunner.tempfile from testing/web-platform/mach_commands.py
wptrunner.testloader from testing/web-platform/mach_commands.py
wptrunner.threading from testing/web-platform/mach_commands.py
wptrunner.time from testing/web-platform/mach_commands.py
wptrunner.types from testing/web-platform/mach_commands.py
wptrunner.update from testing/web-platform/mach_commands.py
wptrunner.update.base from testing/web-platform/mach_commands.py
wptrunner.update.cPickle from testing/web-platform/mach_commands.py
wptrunner.update.metadata from testing/web-platform/mach_commands.py
wptrunner.update.mozlog from testing/web-platform/mach_commands.py
wptrunner.update.os from testing/web-platform/mach_commands.py
wptrunner.update.re from testing/web-platform/mach_commands.py
wptrunner.update.shutil from testing/web-platform/mach_commands.py
wptrunner.update.state from testing/web-platform/mach_commands.py
wptrunner.update.subprocess from testing/web-platform/mach_commands.py
wptrunner.update.sync from testing/web-platform/mach_commands.py
wptrunner.update.sys from testing/web-platform/mach_commands.py
wptrunner.update.tree from testing/web-platform/mach_commands.py
wptrunner.update.update from testing/web-platform/mach_commands.py
wptrunner.update.uuid from testing/web-platform/mach_commands.py
wptrunner.urlparse from testing/web-platform/mach_commands.py
wptrunner.uuid from testing/web-platform/mach_commands.py
wptrunner.vcs from testing/web-platform/mach_commands.py
wptrunner.wptcommandline from testing/web-platform/mach_commands.py
wptrunner.wptlogging from testing/web-platform/mach_commands.py
wptrunner.wptmanifest from testing/web-platform/mach_commands.py
wptrunner.wptmanifest.backends from testing/web-platform/mach_commands.py
wptrunner.wptmanifest.backends.conditional from testing/web-platform/mach_commands.py
wptrunner.wptmanifest.backends.operator from testing/web-platform/mach_commands.py
wptrunner.wptmanifest.backends.static from testing/web-platform/mach_commands.py
wptrunner.wptmanifest.cStringIO from testing/web-platform/mach_commands.py
wptrunner.wptmanifest.node from testing/web-platform/mach_commands.py
wptrunner.wptmanifest.parser from testing/web-platform/mach_commands.py
wptrunner.wptmanifest.serializer from testing/web-platform/mach_commands.py
wptrunner.wptmanifest.types from testing/web-platform/mach_commands.py
wptrunner.wpttest from testing/web-platform/mach_commands.py
xml.parsers from testing/web-platform/mach_commands.py
xml.parsers.expat from testing/web-platform/mach_commands.py
xml.parsers.pyexpat from testing/web-platform/mach_commands.py

I'd appreciate if these Python modules could be refactored so populating mach's parser doesn't require importing so many modules.

Preliminary results indicate that lazy module loading doesn't even help us here because many of these imports seem to occur as part of populating the argument parser. Boo.

Comment 1

[Gregory Szorc [:gps]]

107 of the 177 modules loaded by mach_commands.py files that are only used by a single mach_commands.py file are attributed to testing/web-platform/mach_commands.py. Any reduction in import bloat would be appreciated.

Comment 2

[James Graham [:jgraham]]

Right, which is why my proposed solution to this was to lazy-load the argument parsing i.e. instead of just parsing all the arguments once, parse enough arguments to work out which command will be loaded and have a static mapping of commands to modules. I thought there was a bug for that but now I can't see one.

Comment 3

[James Graham [:jgraham]]

And presumably more commands will get the structured logging options over time (I'm actually surprised that isn't already the case). That accounts for some, but not all, of this.

Comment 4

[Gregory Szorc [:gps]]

I filed bug 1176639 for the structured logging bits. I also have bug 1176642 tracking the improvements I'm making on my own. This and bug 1176642 were a bit too out of my comfort zone.

FWIW, I'm looking at lazy module loading and caching the dispatch table. I fear the latter will be necessary to restore performance. But I'm still holding out hope.

Comment 5

[James Graham [:jgraham]]

Attachment: MozReview Request: Bug 1176632 - Defer creating the web-platform-tests command line argument parser for mach

Bug 1176632 - Defer creating the web-platform-tests command line argument parser for mach

Comment 6

[Chris Manchester (limited bugmail, email directly)]

Comment on attachment 8628779 [details]
MozReview Request: Bug 1176632 - Defer creating the web-platform-tests command line argument parser for mach

https://reviewboard.mozilla.org/r/12467/#review10955

Ship It!

::: testing/web-platform/mach_commands.py:123
(Diff revision 1)
> +def create_parser_update():

These names would read a little better to me as "create_<topic>_parser".

Comment 7

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/4fc6bd8ccab0

Comment 8

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/4fc6bd8ccab0