Linux opt format and tidy bustages with bash: ./mach: Argument list too long

RESOLVED FIXED in Firefox 68

Status

defect
P2
normal
RESOLVED FIXED
3 months ago
2 months ago

People

(Reporter: CosminS, Assigned: bastien)

Tracking

(Regression, {intermittent-failure})

unspecified
mozilla68

Firefox Tracking Flags

(firefox68 fixed)

Details

Attachments

(2 attachments)

When the command line is too long, we should split into several executions.
(this is what we are doing for clang-format)

Assignee: nobody → bpostelnicu
Priority: -- → P2

Does clang-format already accept @resource-file arguments? mach whatever could accept those too.

Assignee

Comment 3

3 months ago

This error happens because both ./mach clang-format and ./mach static-analysis check CI tasks need the list of files modified in the try push.
This is done in taskcluster/scripts/misc/source-test-clang-setup.sh by creating an environment variable TRY_MODIFIED_FILES, and you can see the export line in the log file from this bug.

We could do this in the Python code directly to avoid creating huge environment variables by adding a mercurial revision option. Andrew, what do you think of this solution ? I can implement it.

Flags: needinfo?(ahal)

You would need to use all files touched in the push, not just a single revision. Luckily we have a function that gives you that already:
https://searchfox.org/mozilla-central/source/python/mozversioncontrol/mozversioncontrol/__init__.py#309

Another alternative would be to modify the mach_commands.py itself to read that environment variable instead of passing it in on the task command line.

Flags: needinfo?(ahal)
Assignee

Updated

3 months ago
Assignee: bpostelnicu → bastien
Status: NEW → ASSIGNED
Reporter

Comment 6

3 months ago

There is also this failure now on trees: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=236298018&repo=mozilla-inbound&lineNumber=463

[task 2019-03-27T05:22:06.928Z] ++ cd /builds/worker/checkouts/gecko
[task 2019-03-27T05:22:06.928Z] +++ hg status --rev 'max(ancestors(tip) and public())' -q --no-status
[task 2019-03-27T05:22:09.514Z] ++ export TRY_MODIFIED_FILES=
[task 2019-03-27T05:22:09.514Z] ++ TRY_MODIFIED_FILES=
[task 2019-03-27T05:22:09.514Z] + cd /builds/worker/checkouts/gecko
[task 2019-03-27T05:22:09.514Z] + ./mach --log-no-times clang-format --path --output /builds/worker/clang-format.json --format json
[task 2019-03-27T05:22:09.812Z] usage: mach [global arguments] clang-format [command arguments]
[task 2019-03-27T05:22:09.812Z] mach: error: argument --path/-p: expected at least one argument
[taskcluster 2019-03-27 05:22:10.368Z] === Task Finished ===
[taskcluster 2019-03-27 05:22:10.481Z] Artifact "public/code-review/clang-format.json" not found at "/builds/worker/clang-format.json"
[taskcluster 2019-03-27 05:22:11.272Z] Unsuccessful task run with exit code: 2 completed in 338.428 seconds

See Also: → 1539331

Updated

3 months ago
See Also: → 1540325
Comment hidden (Intermittent Failures Robot)
Comment hidden (Intermittent Failures Robot)

Comment 10

3 months ago
Pushed by babadie@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/813b970cd40b
Replace TRY_MODIFIED_FILES by mozversioncontrol usage, r=ahal

Comment 11

3 months ago
bugherder
Status: ASSIGNED → RESOLVED
Closed: 3 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla68

Comment 12

3 months ago
Pushed by babadie@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/a001a531c01e
Remove _conv_to_abspath from mach static analysis, r=ahal
No longer blocks: 1536836
Regressed by: 1536836
Regressions: 1541660
Comment hidden (Intermittent Failures Robot)
Type: enhancement → defect
You need to log in before you can comment on or make changes to this bug.