Closed Bug 1511594 Opened 5 years ago Closed 5 years ago

hg format-source fails with "mach exited with status 1"

Categories

(Developer Infrastructure :: Lint and Formatting, defect)

defect
Not set
normal

Tracking

(Not tracked)

VERIFIED FIXED

People

(Reporter: emk, Assigned: andi)

References

(Blocks 1 open bug)

Details

Attachments

(3 files)

When I tried to rebase with the following command:

  hg rebase -k -b . -d 138f65958545 --debug

I got the follwing error:

 dom/ipc/TabParent.cpp: versions differ -> m (premerge)
Files to be: dom/ipc/TabParent.cpp dom/ipc/TabParent.cpp dom/ipc/TabParent.cpp
running e:\m\mozilla-unified\mach clang-format --assume-filename $HG_FILENAME -p c:\users\kimu\appdata\local\temp\tmpet53ir.cpp
abort: clang-format: mach exited with status 1

Do I have to manually resolve the conflict? It is really painful.
Thanks for running this id debug mode!
Please copy 'dom/ipc/TabParent.cpp' from the base revision to the destination revision, overwrite it and run:

>>mach clang-format --assume-filename dom/ipc/TabParent.cpp -p dom/ipc/TabParent.cpp

Please copy and paste here the output.

Also do did you run 'mach bootstrap', do you have in ~/.mozbuild/clang-tools/ ?
Assignee: nobody → bpostelnicu
Flags: needinfo?(VYV03354)
(In reply to Andi-Bogdan Postelnicu [:andi] from comment #1)
> Please copy 'dom/ipc/TabParent.cpp' from the base revision to the
> destination revision, overwrite it and run:
> 
> >>mach clang-format --assume-filename dom/ipc/TabParent.cpp -p dom/ipc/TabParent.cpp
> 
> Please copy and paste here the output.

I attached the output because it was large.

> Also do did you run 'mach bootstrap', do you have in
> ~/.mozbuild/clang-tools/ ?

Yes, I re-ran `./mach bootstrap` just in case. I also confirmed that I have `~/.mozbuild/clang-tools/` folder. By the way, my `.mozbuild` is an NTFS junction to another drive because my system drive is a small capacity SSD. Is it relevant?
Flags: needinfo?(VYV03354)
Can you upload a remote bookmark with the changeset that needs to be rebased?
https://hg.mozilla.org/try/rev/2d4d022a4a7251bf6598f23ba2f0619676d0f656

If this is not what you want, please teach me a command.
Summary: format-source did not work → hg format-source fails with "mach exited with status 1"
Are you sure that’s the revision since I don’t see the problematic cpp inside it?
See Also: → 1511656
I modified format-source to see error messages from mach:
--- a/hgext/format-source/__init__.py
+++ b/hgext/format-source/__init__.py
@@ -235,17 +235,21 @@ def run_tools(ui, root, tool, cmd, filep
     ui.debug('running %s\n' % format_cmd)
     ui.pushbuffer(subproc=True)
     try:
         ui.system(format_cmd,
                   environ=env,
                   cwd=root,
                   onerr=error.Abort,
                   errprefix=tool)
-    finally:
+    except:
+        newcontent = ui.popbuffer()
+        ui.debug('result %s\n' % newcontent)
+        raise
+    else:
         newcontent = ui.popbuffer()
     return newcontent

 def touched(repo, old_ctx, new_ctx, paths):
     matcher = rootedmatch(repo, new_ctx, paths)
     if any(path in new_ctx for path in paths):
         status = old_ctx.status(other=new_ctx, match=matcher)
         return bool(status.modified or status.added)

Result:
 dom/ipc/TabParent.cpp: versions differ -> m (premerge)
Files to be: dom/ipc/TabParent.cpp dom/ipc/TabParent.cpp dom/ipc/TabParent.cpp
running e:\m\mozilla-unified\mach clang-format --assume-filename $HG_FILENAME -p c:\users\kimu\appdata\local\temp\tmpyuyb9r.cpp
result 'e:\m\mozilla-unified\mach' は、内部コマンドまたは外部コマンド、
操作可能なプログラムまたはバッチ ファイルとして認識されていません。
(English: 'e:\m\mozilla-unified\mach' is not recognized as an internal or external command,
operable program or batch file.)

I added following settings to ~/.hgrc:
[format-source]
clang-format = /e/m/mozilla-unified/mach clang-format --assume-filename $HG_FILENAME -p
clang-format:configpaths = .clang-format, .clang-format-ignore
clang-format:fileext = .cpp, .c, .h

Then I got the following error:
 dom/ipc/TabParent.cpp: versions differ -> m (premerge)
Files to be: dom/ipc/TabParent.cpp dom/ipc/TabParent.cpp dom/ipc/TabParent.cpp
running /e/m/mozilla-unified/mach clang-format --assume-filename $HG_FILENAME -p c:\users\kimu\appdata\local\temp\tmpeaopjl.cpp
result 指定されたパスが見つかりません。
(English: The system cannot find the file specified.)

Maybe this bug is Windows-specific and Windows-to-UNIX (mingw) path conversions are required?
See Also: 1511656
On Windows, the command processor (cmd.exe) has no way to parse and execute shbang, so it could not run mach directly.
This setting worked for me:
[format-source]
clang-format = sh -c mach clang-format --assume-filename $HG_FILENAME -p
clang-format:configpaths = .clang-format, .clang-format-ignore
clang-format:fileext = .cpp, .c, .h

But I found another (probably plaform-independent) problem. I'll file another bug.
(In reply to Masatoshi Kimura [:emk] from comment #7)
> But I found another (probably plaform-independent) problem. I'll file
> another bug.

Filed bug 1511668.
(In reply to undefined from comment #undefined)
> 
Don’t you run from the build environment? Are you running from cmd directly?
(In reply to Andi-Bogdan Postelnicu [:andi] from comment #9)
> (In reply to undefined from comment #undefined)
> > 
> Don’t you run from the build environment? Are you running from cmd directly?

I run `hg rebase` from the msys shell. But apparently ui.system() invokes the command via cmd on Windows. I have no control over that.
This is very strange, is your setup custom? I'm asking this because other engineers have used this on windows without any issues.
As I already said, I installed files to D: as much as possible.
My objdir is on an exFAT drive.
I did not enalbe fsmonitor because it only times out on my environment.
I have the same problem and I'm using the normal MozillaBuild 3.2 build shell (start-shell.bat).
Did the fix from :emk work?
Flags: needinfo?(jh+bugzilla)
I no longer get the clang-format error, but instead some of my changesets were dropped (which I guess could be bug 1511668).
Flags: needinfo?(jh+bugzilla)
Actually even worse - in my case it dropped *all* changes to .cpp/.h files.
Can you poște onto the new bug a log? Does it have the same behavior on other platforms?
(In reply to Andi-Bogdan Postelnicu [:andi] from comment #11)
> I'm asking this because other engineers have used this on windows without any issues.

Are they using native win32 mercurial? I heard Windows developers had to use WSL to deal with Phacricator problems.
You might have a point with the WSL, thanks for mentioning this.
I will come up to you with a resolution, will try to reproduce this in a win vm.
(In reply to Andi-Bogdan Postelnicu [:andi] from comment #17)
> Can you poște onto the new bug a log? Does it have the same behavior on
> other platforms?

Posted my patch set to bug 1511668 as well.

(In reply to Masatoshi Kimura [:emk] from comment #18)
> Are they using native win32 mercurial? I heard Windows developers had to use
> WSL to deal with Phacricator problems.

Good point, I'm not using WSL, either (and couldn't even if I wanted to, because my desktop is still on Windows 7).
Pushed by cosheehan@mozilla.com:
https://hg.mozilla.org/hgcustom/version-control-tools/rev/42179015d473
extensions: Improve hg-format source with default parameters on windows r=sheehan
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
I got `Command Not Found: machclang-format` error with this patch. Proposed fix:
--- a/hgext/format-source/__init__.py
+++ b/hgext/format-source/__init__.py
@@ -90,17 +90,17 @@ if util.safehasattr(registrar, 'configit
 file_storage_path = '.hg-format-source'


 def return_default_clang_format(repo):
     arguments = ['clang-format', '--assume-filename', '$HG_FILENAME', '-p']

     # On windows we need this to call the command in a shell, see Bug 1511594
     if os.name == 'nt':
-        clang_format_cmd = 'sh mach' + ' '.join(arguments)
+        clang_format_cmd = 'sh mach ' + ' '.join(arguments)
     else:
         clang_format_cmd = os.path.join(repo.root, "mach") + ' '.join(arguments)

     clang_format_cfgpaths = ['.clang-format', '.clang-format-ignore']
     clang_fortmat_fileext = ('.cpp', '.c', '.cc', '.h')
     return clang_format_cmd, clang_format_cfgpaths, clang_fortmat_fileext
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Right good catch, otherwise did it work with your fix?
Flags: needinfo?(VYV03354)
Yes, rebase succeeded (including all .cpp files) with my fix. Thanks for the great work!
Flags: needinfo?(VYV03354)
Pushed by cosheehan@mozilla.com:
https://hg.mozilla.org/hgcustom/version-control-tools/rev/efdd5165c82e
extensions: hg-format add missing space between command and arguments r=sheehan
Status: REOPENED → RESOLVED
Closed: 5 years ago5 years ago
Resolution: --- → FIXED
(In reply to Masatoshi Kimura [:emk] from comment #26)
> Yes, rebase succeeded (including all .cpp files) with my fix. Thanks for the
> great work!

Seconded, thanks as well.
Status: RESOLVED → VERIFIED
Product: Firefox Build System → Developer Infrastructure
You need to log in before you can comment on or make changes to this bug.