1272544 - git artifact builds failing: WindowsError: [Error 206] The filename or extension is too long

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Description

[Mark Hammond [:markh] [:mhammond]]

Traceback below. The problem is that _get_hg_revisions_from_git() is executing:

% git rev-list --topo-order HEAD~500..HEAD

and it then hopes to pass the output of that, via the command-line to:

% git cinnabar git2hg ...

The issue is that the rev-list command is returning 2369 full hashes, which appears to exceed a Windows command-line limit (which isn't *too* surprising - the cmdline in this case is 97128 bytes!)

The workarounds:

1) Use --abbrev-commit on the rev-list command. This reduces the command-line down to ~20k and does work, but it takes cinnabar ~30s to output the hg lists - a quick scan of cinnabar's code shows that it makes its own |git rev-list| call to turn them back into long hashes, so I'm guessing this is much slower than it would be if full hashes were passed.

2) Change the initial revlist command to something like |git rev-list --topo-order --max-count=500 HEAD|

3) Change NUM_REVISIONS_TO_QUERY from 500 to, say, 100.

All the above workarounds worked for me. I'm guessing (2) is reasonable - if it is, I'll put up a patch. If it's not, it might be necessary to change cinnabar to allow reading of the revisions from stdin and have artifacts.py pipe them in that way.

The details of the failure are as follows:
> 
> WindowsError: [Error 206] The filename or extension is too long
>   File "o:\src\moz\gecko\python/mozbuild/mozbuild/mach_commands.py", line 1549, in artifact_install
>     return artifacts.install_from(source, self.distdir)
>   File "o:\src\moz\gecko\python/mozbuild\mozbuild\artifacts.py", line 1037, in install_from
>     return self.install_from_recent(distdir)
>   File "o:\src\moz\gecko\python/mozbuild\mozbuild\artifacts.py", line 999, in install_from_recent
>     return self._install_from_hg_pushheads(hg_pushheads, distdir)
>   File "o:\src\moz\gecko\python/mozbuild\mozbuild\artifacts.py", line 977, in _install_from_hg_pushheads
>     for hg_hash in hg_pushheads:
>   File "o:\src\moz\gecko\python/mozbuild\mozbuild\artifacts.py", line 871, in _find_pushheads
>     last_revs = self._get_recent_public_revisions()
>   File "o:\src\moz\gecko\python/mozbuild\mozbuild\artifacts.py", line 857, in _get_recent_public_revisions
>     return self._get_hg_revisions_from_git()
>   File "o:\src\moz\gecko\python/mozbuild\mozbuild\artifacts.py", line 838, in _get_hg_revisions_from_git
>     ] + rev_list.splitlines())
>   File "c:\mozilla-build\python\lib\subprocess.py", line 566, in check_output
>     process = Popen(stdout=PIPE, *popenargs, **kwargs)
>   File "c:\mozilla-build\python\lib\subprocess.py", line 710, in __init__
>     errread, errwrite)
>   File "c:\mozilla-build\python\lib\subprocess.py", line 958, in _execute_child
>     startupinfo)

(line numbers might be off by a line or 2 as I added a couple of prints, but you get the idea)

Comment 1

[Chris Manchester (limited bugmail, email directly)]

Either or both of 2 and 3 sound reasonable.

Comment 2

[Mark Hammond [:markh] [:mhammond]]

Attachment: MozReview Request: Bug 1272544 - artifact builds with git now only consider the last 500 revisions rather than all revisions for the last 500 commits. r?chmanchester

Review commit: https://reviewboard.mozilla.org/r/52403/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/52403/

Comment 3

[Chris Manchester (limited bugmail, email directly)]

Comment on attachment 8752059 [details]
MozReview Request: Bug 1272544 - artifact builds with git now only consider the last 500 revisions rather than all revisions for the last 500 commits. r?chmanchester

https://reviewboard.mozilla.org/r/52403/#review49465

Thank you for the patch.

Comment 4

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/950d344b6e43

Comment 5

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/950d344b6e43