1312140 - Autoland stats all checked out files, adds dozens of seconds to operations

Status: RESOLVED WONTFIX

(Conduit Graveyard :: Transplant, defect, P3)

Description

[Gregory Szorc [:gps]]

Autoland is currently taking 70-80s per push. strace appears to indicate it is calling stat() on all (or at least a good portion of) the working directory. I suspect we're doing something sub-optimal in Mercurial API land. If we fix it, autoland will likely complete dozens of seconds faster.

I installed python2.7-dbg on the autoland host. So you can attach gdb to a process and run `py-bt` to get a Python stack. I haven't seen anything obvious yet.

Comment 1

[Mark Côté [:mcote]]

Do you think this is an issue with Lando's use of transplant (or maybe it's not an issue at all anymore)?

Comment 2

[:glob ✱]

how autoland uses mercurial doesn't really change with lando, so it's likely to still be an issue.

it's probably something we should revisit when we move to cloudops as part of the reworking, and might be easier to fix by ensuring we deploy on infra with faster disk.

Comment 3

[Gregory Szorc [:gps]]

Since this issue was reported 2 years ago, watchman integration is now built-in to Mercurial. If we installed watchman and enabled the fsmonitor extension on the transplant server, things should magically get a lot faster.

Comment 4

[:glob ✱]

The time Lando takes to land changes isn't problematic.