1710287 - |./mach clang-format| doesn't update clang-format, incorrectly recommends |./mach bootstrap|

Status: RESOLVED FIXED

(Developer Infrastructure :: Lint and Formatting, defect, P3)

Description

[Lars T Hansen [:lth]]

Consider:

$ hg commit --amend
 0:00.56 ERROR: You're using an old or incorrect version (clang-format version 11.0.1 (taskcluster-fQQA3eNxTwiFHaZ6gEXSWQ)) of clang-format binary. Please update to a more recent one (at least > 12.0.0) by running: './mach bootstrap' 
$ ./mach bootstrap
...
$ hg commit --amend
 0:00.56 ERROR: You're using an old or incorrect version (clang-format version 11.0.1 (taskcluster-fQQA3eNxTwiFHaZ6gEXSWQ)) of clang-format binary. Please update to a more recent one (at least > 12.0.0) by running: './mach bootstrap' 

That is, bootstrapping does not install the tool, contrary to the error message. This is on an M1 MacBook Pro, MacOS 11.3.1.

Comment 1

[Mitchell Hentges [:mhentges] 🦀]

A few questions for you:

• What's the output of ~/.mozbuild/clang/bin/clang-format --version?
• If you run ./mach bootstrap, does it print Setting up artifact clang.tar.zst?

Comment 2

[Lars T Hansen [:lth]]

(In reply to Mitchell Hentges [:mhentges] 🦀 from comment #1)

A few questions for you:

• What's the output of ~/.mozbuild/clang/bin/clang-format --version?

$ ~/.mozbuild/clang/bin/clang-format --version
clang-format version 12.0.0 (taskcluster-I-6A7g8WQRCYbWdAC4ZYmA)

There does not seem to be a clang-format in the path.

• If you run ./mach bootstrap, does it print Setting up artifact clang.tar.zst?

No. In fact, unique to the M1, after doing the mercurial setup, bootstrap always exits. Later, the build process may sometimes download tools, but when I tried a build yesterday to see if it would improve the situation, no tool download appeared to happen.

Comment 3

[Mitchell Hentges [:mhentges] 🦀]

Hmm, it looks like your ~/.mozbuild/clang/bin/clang-format is up-to-date.
I wonder how the pre-commit hook is deciding which clang-format to use, perhaps it's checking the PATH before ~/.mozbuild.
Can you show me the output of hg config hooks --debug, or perhaps .hg/hgrc in the repository?

Comment 4

[Lars T Hansen [:lth]]

Ask and ye shall receive:

$ hg config hooks --debug
starting pager for command 'config'
read config from: resource:mercurial.defaultrc.mergetools.rc
read config from: /opt/homebrew/etc/mercurial/hgrc
read config from: /etc/mercurial/hgrc
read config from: /Users/lhansen/.hgrc

I doubt my top-level .hgrc is very interesting but here's an excerpt:

[extensions]
strip = 
purge = 
absorb = 
histedit = 
rebase = 
fsmonitor = 
blackbox = 
firefoxtree = /Users/lhansen/.mozbuild/version-control-tools/hgext/firefoxtree
push-to-try = /Users/lhansen/.mozbuild/version-control-tools/hgext/push-to-try
clang-format = /Users/lhansen/.mozbuild/version-control-tools/hgext/clang-format
shelve = 
show = 
#format-source = /Users/lhansen/.mozbuild/version-control-tools/hgext/format-source

The .hg/hgrc is this:

# example repository config (see 'hg help config' for more info)
[paths]
default = https://hg.mozilla.org/mozilla-unified

# path aliases to other clones of this repo in URLs or filesystem paths
# (see 'hg help config.paths' for more info)
#
# default:pushurl = ssh://[e-mail]/hg/jdoes-fork
# my-fork         = ssh://[e-mail]/hg/jdoes-fork
# my-clone        = /home/jdoe/jdoes-clone

[ui]
# name and email (local to this repository, optional), e.g.
# username = Jane Doe <[e-mail]>

ignore = ~/moz/sandbox/ignore

where the linked ignore file is

js/src/build-release
js/src/build-release-debug
js/src/build-test
js/src/build-fuzz
js/src/build-debug
js/src/build-arm
js/src/build-.*

Comment 5

[Mitchell Hentges [:mhentges] 🦀]

I wonder how the pre-commit hook is deciding which clang-format to use, perhaps it's checking the PATH before ~/.mozbuild.

Ah, it's from the clang-format extension which calls ./mach clang-format -p [paths].
Let me dig into ./mach clang-format's technique of finding the clang-format binary.

Comment 6

[Mitchell Hentges [:mhentges] 🦀]

Ah-hah:

1. clang-format also (?) exists in ~/.mozbuild/clang-tidy/bin/clang-format. I bet that that, on your machine, is version 11, not 12.
2. Arm Macs use the OSXBootstrapperLight, which doesn't download clang-tidy.
   • I'll make a patch to address this.
3. You should be able to work around this in the short term by doing ./mach artifact toolchain --bootstrap --from-build macosx64-clang-tidy

Comment 7

[Mitchell Hentges [:mhentges] 🦀]

Attachment: Bug 1710287: Download clang-tidy on M1 Macs

Most build-related bootstrapping was recently moved to happen
during configure.

However, some of these tools are needed outside of a build.
The bug associated with this patch was reported because the user
needed to commit, but committing is associated with running
"clang-format", which is fetched via bootstrap.

Comment 8

[Lars T Hansen [:lth]]

(In reply to Mitchell Hentges [:mhentges] 🦀 from comment #6)

1. clang-format also (?) exists in ~/.mozbuild/clang-tidy/bin/clang-format. I bet that that, on your machine, is version 11, not 12.

I appear to have no such file. find . -name clang-format from my home dir reports:

./m-u/tools/lint/test/files/clang-format
./m-u/tools/lint/clang-format
./m-u/security/nss/automation/clang-format
./m-u/clang-tidy/bin/clang-format
./m-u/.hg/store/data/tools/lint/test/files/clang-format
./m-u/.hg/store/data/tools/lint/clang-format
./m-u/.hg/store/data/security/nss/automation/clang-format
./.mozbuild/clang/bin/clang-format
./.mozbuild/clang-tools/clang-tidy/bin/clang-format
./.mozbuild/version-control-tools/.hg/store/data/hgext/clang-format
./.mozbuild/version-control-tools/hgext/clang-format

3. You should be able to work around this in the short term by doing ./mach artifact toolchain --bootstrap --from-build macosx64-clang-tidy

Sadly not. I did this, and it did download something:

eeyore:m-u lhansen$ ./mach artifact toolchain --bootstrap --from-build macosx64-clang-tidy
 0:02.60 Setting up artifact clang-tidy.tar.zst
 0:02.60 Downloading artifact to local cache: /Users/lhansen/.mozbuild/toolchains/dbcc5469839e6edd-clang-tidy.tar.zst
 ...
 0:04.92 Downloading... 100.0 %
 0:05.00 untarring "/Users/lhansen/m-u/clang-tidy.tar.zst"

but hg commit still provokes the same error message afterwards.

Comment 9

[Mitchell Hentges [:mhentges] 🦀]

Ah, two mistakes on my part:

1. Right, I missed a part of the tidy clang-format location. It indeed should be at ~/.mozbuild/clang-tools/clang-tidy/bin/clang-format, as you've got there. Thanks for doing that find ... there, that was a smart call 👍.
   2. Speaking of which, if you run that with --version, it should show itself as version 11.
2. TIL that ./mach artifact toolchain ... cares about your working directory. So, you probably have a big clang-tidy directory tree splatted in your source code directory right now, sorry about that. The workaround to make this happy will require that you do:
   • cd ~/.mozbuild
   • path/to/source/directory/mach artifact toolchain --bootstrap --from-build macosx64-clang-tidy
   • Now, ~/.mozbuild/clang-tools/clang-tidy/bin/clang-format --version should show version 12, and you should be good to go.

Sorry for the unfortunate tips, but that should at least get you on your way.

---

The patch to improve this will be a little more work, as I'm going to embrace our "automatic bootstrapping" as glandium recommended in the revision.

Comment 10

[Mitchell Hentges [:mhentges] 🦀]

This isn't affecting a large number of people (to my knowledge), and there's a workaround (rm -r ~/.mozbuild/clang-tools).
Due to build team resource constraints, and because synchronizing bootstrap state between ./mach bootstrap/configure/./mach clang-format is non-trivial, I'm going to defer this.

Comment 13

[Mitchell Hentges [:mhentges] 🦀]

Attachment: Bug 1710287: `./mach clang-format` should update tools if out-of-date

Before, clang tools would only be bootstrapped if they didn't exist.
Now, the version check is moved such that, if the check fails, the tools
are updated.

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/97605bac0987
`./mach clang-format` should update tools if out-of-date r=andi,marco

Comment 16

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/97605bac0987