Closed Bug 1646244 Opened 4 years ago Closed 4 years ago

pre-commit in Travis takes a long time

Categories

(Tree Management :: Treeherder, enhancement)

enhancement

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: ekyle, Unassigned)

Details

Attachments

(1 file)

I think we are wasting time on travis. pre-commit is a fine tool on a devlopment machine, where the plugins get installed just once, but in CI it must pull the plugin code for every compile.

It is probably faster to install the plugin libraries with the rest of the dev code, and run the checks directly.

a sense of how long it took:

pre-commit took about 2 minutes

2020-07-13 20:12:59.969 | WARNING  | adr.configuration:__init__:118 - Configuration path /home/travis/.config/adr/config.toml is not a file.
System check identified no issues (0 silenced).
py37 run-test: commands[0] | pip check
No broken requirements found.
py37 run-test: commands[1] | /home/travis/build/mozilla/treeherder/lints/queuelint.py
py37 run-test: commands[2] | flake8 --show-source
py37 run-test: commands[3] | pre-commit run --all-files
[INFO] Initializing environment for https://gitlab.com/pycqa/flake8.
[INFO] Initializing environment for https://github.com/shellcheck-py/shellcheck-py.
[INFO] Initializing environment for https://github.com/prettier/prettier.
[INFO] Initializing environment for https://github.com/igorshubovych/markdownlint-cli.
[INFO] Initializing environment for https://github.com/psf/black.
[INFO] Installing environment for https://gitlab.com/pycqa/flake8.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for https://github.com/shellcheck-py/shellcheck-py.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for https://github.com/prettier/prettier.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for https://github.com/igorshubovych/markdownlint-cli.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for https://github.com/psf/black.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
flake8...................................................................Passed
shellcheck...............................................................Passed
prettier.................................................................Passed
markdownlint.............................................................Passed
black....................................................................Passed
py37 run-test: commands[4] | sh -c 'SITE_URL=https://treeherder.dev TREEHERDER_DEBUG=False ./manage.py check --deploy --fail-level WARNING'
2020-07-13 20:15:03.291 | WARNING  | adr.configuration:__init__:118 - Configuration path /home/travis

The new run time is 20seconds

2020-07-14 13:47:36.190 | WARNING  | adr.configuration:__init__:118 - Configuration path /home/travis/.config/adr/config.toml is not a file.
System check identified no issues (0 silenced).
py37 run-test: commands[0] | pip check
No broken requirements found.
py37 run-test: commands[1] | /home/travis/build/mozilla/treeherder/lints/queuelint.py
py37 run-test: commands[2] | flake8 --show-source
py37 run-test: commands[3] | black --check .
All done! ✨ 🍰 ✨
335 files would be left unchanged.
py37 run-test: commands[4] | shellcheck initialize_data.sh
py37 run-test: commands[5] | shellcheck docker/entrypoint.sh
py37 run-test: commands[6] | sh -c 'SITE_URL=https://treeherder.dev TREEHERDER_DEBUG=False ./manage.py check --deploy --fail-level WARNING'
2020-07-14 13:47:56.011 | WARNING  | adr.configuration:__init__:118 - Configuration path /home/travis

The JS tasks took 12sec+4sec, AND run in parallel; so have no impact of total CI time

$ yarn prettier
yarn run v1.22.4
$ npx prettier --check .
npx: installed 1 in 1.059s
Checking formatting...
All matched files use Prettier code style!
The command "yarn prettier" exited with 0.
4.23s$ yarn markdownlint
yarn run v1.22.4
$ npx markdownlint-cli -c .markdownlint.json -p .markdownlintignore .
npx: installed 36 in 2.362s
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
Component: Treeherder: Docs & Development → TreeHerder
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: