moz-phab on mercurial is slow to determine changed file modes
Categories
(Conduit :: moz-phab, defect, P2)
Tracking
(Not tracked)
People
(Reporter: evilpies, Assigned: glob)
Details
(Keywords: conduit-triaged)
Attachments
(2 files)
Apparently 1 minute per patch is not the normal time that moz-phab submit should take.
:padenot already made some suggestions on Matrix, so I made the following changes:
- Switched from mozilla-central to mozilla-unified
- Nuked .mozbuild and re-run ./mach bootstrap
- Updated hg/moz-phab
My hg version is 5.3.1 and moz-phab is 0.1.80.
|
Reporter | |
Comment 1•5 years ago
|
||
perf top shows e.g. 21,40% python2.7 [.] PyEval_EvalFrameEx
If you run with DEBUG env set you'll see a trace of actions performed: DEBUG=1 moz-phab submit ...
21,40% python2.7
That's hg; moz-phab requires python3.5+,
Make sure you have the mercurial evolve extension installed and enabled, as well as watchman.
Are you able to attach the output from DEBUG=1 moz-phab submit ...?
|
|
Reporter | |
Comment 4•5 years ago
|
||
I think watchman might not be working correctly for me. I remember seeing something about timeouts before, but not in this log.
I see you don't have the mercurial evolve extension installed. Run mach bootstrap and say Yes to installing evolve.
Evolve speeds up a lot of operations, particularly those moz-phab uses when creating a new revision.
That command took 33 seconds to run:
4s preamble (before confirmation prompt)
25s prepare (building diff to submit)
2s revision creation (phabricator)
2s cleanup
The bulk of the preparation time is spent running these two commands:
hg --config extensions.rebase= --pager never --config rebase.experimental.inmemory=true manifest -T '{mode} {path}\n' -r dad98a19520e3afe07531db40351b3f53c3312a8
hg --config extensions.rebase= --pager never --config rebase.experimental.inmemory=true manifest -T '{mode} {path}\n' -r a1c6d5e83eda3c7538cbdaccbb2d15540fa90fbd
Which generate file modes for all files in the tree to detect changed filemodes.
On my system this takes 16 seconds.
It would be much quicker if we built a list of files modified between the two revisions, and then run hg files on just those files.
From the command line this would be:
hg files $( hg status --rev dad98a19520e3afe07531db40351b3f53c3312a8:a1c6d5e83eda3c7538cbdaccbb2d15540fa90fbd -T '{path}\n' ) -T '{flags}:{path}\n'
On my system this takes 0.3 seconds.
|
|
Reporter | |
Comment 6•5 years ago
|
||
Are you sure? Maybe the version is wrong?
$ hg evolve
no troubled changesets
$ hg debugextensions
...
evolve (5.3!)
...
Ah, you are correct - my bad.
Older versions of moz-phab used to explicitly enable evolve when calling hg, so it was visible in the logs.
Newer versions only do that when running under --safe-mode.
Instead of generating a list of file mode changes for all files in the
repository, which is expensive on mozilla-central's ~300k files, use
the list of files modified by the commit being submitted.
On my system this reduces the time _get_file_modes takes on a
mozilla-central repo from 17.5 seconds down to 1.5 seconds.
Description
•