1645196 - Teach `mach vcs-setup` to tune git configuration in the same way it tunes hg configuration

Status: RESOLVED FIXED

(Firefox Build System :: Bootstrap Configuration, enhancement, P3)

Description

[Nick Alexander :nalexander [he/him]]

I'm evaluating using git for developing on m-c, and there's a great deal of git performance configuration possible -- and possibly required -- to get acceptable performance. There's code to do some tuning but it seems to be tied to cloning and not used to help get developers into the best possible state.

Concurrently, it would be nice if this was expanded. Locally, I've found

• core.commitGraph
• core.fsmonitor
• core.splitIndex
• core.untrackedCache

seem to make git usable with a fresh clone.

Comment 1

[Ricky Stewart]

fsmonitor has behaved really poorly on my Mac for me so I'm not sure enforcing it globally makes sense. The rest, sure.

Comment 2

[Mike Hommey [:glandium]]

core.commitGraph is enabled by default since git 2.24. The pref itself exists since 2.18. It feels to me we should rely on the default and push people to use a more modern version of git if theirs is too old, like we do for mercurial.

core.untrackedCache had a bug before 2.17, so we probably shouldn't enable it on these older versions.

Comment 3

[Ricky Stewart]

I tried core.splitIndex at the suggestion of comment 0 and it caused all my builds to unconditionally fail in cargo because it depends on libgit2 which appears to not support splitIndex, so that's probably a no-go :)

Comment 4

[Mike Hommey [:glandium]]

What the hell does cargo do with the index?

Comment 5

[Ricky Stewart]

I guess it's using git (via libgit2) to determine whether crates need to be rebuilt? At least, that's the only thing I can possibly conclude from the error:

 1:52.99 error: failed to determine package fingerprint for build script for geckodriver v0.26.0 (/Users/rickystewart/src/mozilla-unified/testing/geckodriver)
 1:52.99 Caused by:
 1:52.99   failed to determine the most recently modified file in /Users/rickystewart/src/mozilla-unified/testing/geckodriver
 1:52.99 Caused by:
 1:52.99   failed to determine list of files in /Users/rickystewart/src/mozilla-unified/testing/geckodriver
 1:52.99 Caused by:
 1:53.00   failed to open git index at /Users/rickystewart/src/mozilla-unified/.git/
 1:53.00 Caused by:
 1:53.00   unsupported mandatory extension: 'link'; class=Index (10)
 1:53.02 make[4]: *** [force-cargo-program-build] Error 101
 1:53.02 make[3]: *** [testing/geckodriver/target] Error 2

If that's true, that's news to me. I had no idea cargo even knew what git is, beyond calling git init for me when I do cargo init.

Comment 6

[Ricky Stewart]

for example

Comment 7

[Mitchell Hentges [:mhentges] 🦀]

Based on the above discussion, I'll change vcs-setup to do:

1. Enable untrackedCache by default
2. Recommend users use a git version that is at least 2.24 to improve performance

Comment 8

[Mitchell Hentges [:mhentges] 🦀]

Attachment: Bug 1645196: Tune git settings to improve performance r?rstewart

Also adds a warning if a user's git version is older than 2.24

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d72c64af2c23
Tune git settings to improve performance r=rstewart

Comment 10

[Bogdan Tara[:bogdan_tara | bogdant]]

https://hg.mozilla.org/mozilla-central/rev/d72c64af2c23

Comment 11

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Backed out due to bug 1651542:

https://hg.mozilla.org/integration/autoland/rev/739056dc4d11ec7953df7559309d5c958e136acd

Comment 12

[Henrik Skupin [:whimboo][⌚️UTC+2]]

The backout didn't help and the tests are still failing. See bug 1651542 comment 14 for details.

Mitchell, feel free to reland your patch.

Comment 13

[Mitchell Hentges [:mhentges] 🦀]

Sounds good!

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/422c5c783008
Tune git settings to improve performance r=rstewart

Comment 15

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Backout got merged: https://hg.mozilla.org/mozilla-central/rev/739056dc4d11

Comment 16

[Bogdan Tara[:bogdan_tara | bogdant]]

https://hg.mozilla.org/mozilla-central/rev/422c5c783008