Ship watchman and hgwatchman in MozillaBuild

RESOLVED FIXED

Status

mozilla.org
MozillaBuild
RESOLVED FIXED
2 years ago
2 years ago

People

(Reporter: gps, Assigned: RyanVM)

Tracking

Details

hgwatchman makes `hg status` and other commands that walk the working copy much faster. It relies on the Watchman file watching tool, which until recently did not support Windows.

The author of Watchman is standing over my shoulder. Windows support is in "alpha." But that label is only because they don't have any users at scale to call it "beta" or better.

We should:

1) Install watchman in MozillaBuild
2) Ship hgwatchman in MozillaBuild (or vendor in version-control-tools)
3) Enable hgwatchman by default in MozillaBuild's global hgrc

We can find a link to the latest watchman Windows build at https://ci.appveyor.com/project/wez/watchman/branch/master/artifacts. More info at https://github.com/facebook/watchman/issues/19.

The way I see it, we can ship this enabled in a beta of MozillaBuild. If watchman blows up, we just don't enable hgwatchman in the global hgrc.
(Assignee)

Updated

2 years ago
Blocks: 1224680
Good to see this. I thought Windows had a good cache so that even if the repo is large, `hg status` won't be too slow. But it seems that's not true. The command becomes much slower than it was when I started using this Windows machine.
(Assignee)

Comment 2

2 years ago
I'm assuming we need pywatchman as well per https://github.com/facebook/watchman/issues/19 ?
(Assignee)

Comment 3

2 years ago
Looks like the hgwatchman package on pypi includes it, so nevermind.
(Assignee)

Comment 4

2 years ago
I'm not really sure how to test this, but here's a build that I believe has everything setup correctly for it. Can you try it out, Greg?

http://people.mozilla.org/~rvandermeulen/MozillaBuildSetup2.2.0pre1.exe (77.5MB)
SHA-256: f929bffff830d608a81206592458df1bc9a40db4627649f0c8e9d4d8c0e1d2fe
Flags: needinfo?(gps)
I don't have access to a Windows machine this week. But if you want to test this yourself, try running `hg status` a few times. With hgwatchman installed, it should complete in a fraction of the time as without hgwatchman. You can disable watchman on the command invocation via `hg --config watchman.mode=off status`.
Flags: needinfo?(gps)
watchman + hgwatchman appears to be working in this build! `hg status` goes from ~1.680s to ~0.280s on my machine. Not bad. `git status` (from Git for Windows Win32 distribution) is ~0.363s by comparison, making hg the winner.

FWIW, `hg --version` is taking ~0.157s, which feels a bit high. I may have to perform some performance profiling to see what's up.
Disabling all extensions causes `hg --version` to run in ~0.049s. Looks like it's time to refactor some code in version-control-tools to optimize for load time.
I wonder whether it would make sense to use pypy+chg to optimize the performance. (Although it seems chg is not available on Windows, so it is not actually relevant to this bug.)
(Assignee)

Comment 9

2 years ago
http://hg.mozilla.org/mozilla-build/rev/b092537e51b2
http://hg.mozilla.org/mozilla-build/rev/d2b21afc1c0c
http://hg.mozilla.org/mozilla-build/rev/9a412956ed10

This enables it by default for now. Will be trivial to turn off if needed, though I'm not sure how we're going to judge whether that's the case or not.
Assignee: nobody → ryanvm
Status: NEW → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.