Open Bug 1631950 Opened 5 months ago Updated 5 months ago

Use native tools to extract zip/tar files when possible

Categories

(Release Engineering :: Applications: MozharnessCore, enhancement)

enhancement

Tracking

(Not tracked)

ASSIGNED

People

(Reporter: catlee, Assigned: catlee)

Details

Attachments

(1 file)

Python's implementation of tar and zip is slow compared to native tools.

We should use the system's tar/unzip if possible in download-and-extract.

The native tar tool is much faster than python's tarfile implementation. We
should try to extract using it when possible.

Some versions of tar do not support the --wildcards option, so we fallback to
using python's version in this case since extracting a subset of the files
using python is usually faster than extracting all of the files with the native
tool.

Add a special case for web-platform tests to extract everything using the
native tool instead of trying to filter. The filters are global to all test
packages currently being downloaded/extracted, but we need everything from the
web-platform tests archive anyway.

Assignee: nobody → catlee
Status: NEW → ASSIGNED
You need to log in before you can comment on or make changes to this bug.