1631950 - Use native tools to extract zip/tar files when possible

Status: RESOLVED INACTIVE

(Release Engineering :: Applications: MozharnessCore, enhancement)

Description

[Chris AtLee [:catlee]]

Python's implementation of tar and zip is slow compared to native tools.

We should use the system's tar/unzip if possible in download-and-extract.

Comment 1

[Chris AtLee [:catlee]]

Attachment: Bug 1631950: Use native tar to extract tarfiles r=glandium

The native tar tool is much faster than python's tarfile implementation. We
should try to extract using it when possible.

Some versions of tar do not support the --wildcards option, so we fallback to
using python's version in this case since extracting a subset of the files
using python is usually faster than extracting all of the files with the native
tool.

Add a special case for web-platform tests to extract everything using the
native tool instead of trying to filter. The filters are global to all test
packages currently being downloaded/extracted, but we need everything from the
web-platform tests archive anyway.