1904777 - perma "Aborting, could not apply patch buffer for None."

Status: RESOLVED FIXED

(Developer Infrastructure :: Try, defect)

Description

[:gerard-majax]

Since circa yesterday afternoon, none of my mach try fuzzy --push-to-lando have been able to succeed. Directly pushing to try always succeeded.

This is from a git-cinnabar repo, no change whatsoever on my side, uptodate to current central in all cases. This is not related to the branch content itself, i reproduce with multiple branches.

For all the attempts, it fails with:

Your request to land try push with tip commit 'XXX' failed.

Reason:
Aborting, could not apply patch buffer for None.
(1, b'nothing changed', b'')

This is the result of the all mach try --push-to-lando:

$ BUILD_DEBUG=1 ./mach try fuzzy --push-to-lando -q "'linux | 'macosx" toolkit/components/extensions/test/mochitest/test_chrome_native_messaging_paths_*
estimates: Runs 53 tasks (30 selected, 23 dependencies)
estimates: Total task duration 6:23:28
estimates: In the shortest 28% of durations
estimates: Should take about 1:32:01 (Finished around 2024-06-25 18:32)
Auth0 token validated.
Using 653f0dc8442dd9cae70845896eb3c0a0252677b3 as the hg base commit.
Submitting stack of 9 nodes and the try commit.
Patches gathered for submission.
Submitting patches to Lando.
Lando try submission success, took 1.84 seconds. Landing job id: 88356.

This one succeeded as direct push: https://treeherder.mozilla.org/jobs?repo=try&revision=d49bc5ac3f9b52c85b8b41341a6daed7294a0403

$ BUILD_DEBUG=1 ./mach try fuzzy --push-to-lando -q "'linux | 'macosx" toolkit/components/extensions/test/mochitest/test_chrome_native_messaging_paths_*
Task configuration changed, generating target task set
estimates: Runs 53 tasks (30 selected, 23 dependencies)
estimates: Total task duration 6:23:28
estimates: In the shortest 28% of durations
estimates: Should take about 1:32:01 (Finished around 2024-06-25 21:16)
Auth0 token validated.
Using a7d69e48affbb339e57fa5845eadc38192774df4 as the hg base commit.
Submitting stack of 8 nodes and the try commit.
Patches gathered for submission.
Submitting patches to Lando.
Lando try submission success, took 2.06 seconds. Landing job id: 88375.

Direct: https://treeherder.mozilla.org/jobs?repo=try&revision=ad2dfe08564f503e952788c71fdffa5f145bfbae

$ BUILD_DEBUG=1 ./mach try fuzzy --full --push-to-lando -q "'xdg-test"
Task configuration changed, generating full task set
estimates: Runs 15 tasks (5 selected, 10 dependencies)
estimates: Total task duration 1:07:13
estimates: In the shortest 5% of durations
estimates: Should take about 1:01:56 (Finished around 2024-06-26 11:15)
Auth0 token validated.
Using eb8e54ddbc044ecab12d09d7a7417a9d7a9bec8e as the hg base commit.
Submitting stack of 10 nodes and the try commit.
Patches gathered for submission.
Submitting patches to Lando.
Lando try submission success, took 1.69 seconds. Landing job id: 88420.

Direct: https://treeherder.mozilla.org/jobs?repo=try&revision=72bf76dc055ee45894b3bee0893603f7cfd51f5d

Comment 1

[Mark Banner (:standard8)]

I've had a similar issue on a Mac, using the latest m-c with git cinnabar 0.6.3-830a1f2c75fa91cf509020d16d19698a159cf22e and git version 2.45.0. This is the direct push: https://hg.mozilla.org/try/pushloghtml?changeset=521560f9ccfb2309b498005c793d632d7f24c378

I went back to just before bug 1500188 and applied my whitespace only JS change and it worked, moving to just after that and I get the same failure as we see here. So I think these are probably the same - though I noticed that :gerard-majax said on Matrix that backing out bug 1500188 didn't help.

I've tried applied the fixes for the other two regressions from bug 1500188 and they haven't helped.

Dumping the patches from within get_stack_info, I can see my patch being listed correctly. However, the patch for the try chooser output is wrong. This is the python output of the last patch in the array:

b'From c9896870ad94b7149ac67648d47ae09c2af30074 Mon Sep 17 00:00:00 2001\nFrom: Mark Banner <[e-mail]>\nDate: Fri, 28 Jun 2024 09:44:12 +0100\nSubject: [PATCH] Try Chooser Enhanced (1 tasks selected)\n\nmach try command: `./mach -v try chooser --push-to-lando`\n\nPushed via `mach try chooser`\n-- \n2.45.0\n\n'

and this is the python output of that last element from a direct push:

b'From 562aff3286503e4b3b3c010f81578ea9eb482aa4 Mon Sep 17 00:00:00 2001\nFrom: Mark Banner <[e-mail]>\nDate: Fri, 28 Jun 2024 09:44:46 +0100\nSubject: [PATCH] Try Chooser Enhanced (1 tasks selected)\n\nmach try command: `./mach -v try chooser --push-to-lando`\n\nPushed via `mach try chooser`\n---\n try_task_config.json | 14 ++++++++++++++\n 1 file changed, 14 insertions(+)\n create mode 100644 try_task_config.json\n\ndiff --git a/try_task_config.json b/try_task_config.json\nnew file mode 100644\nindex 0000000000000..457007b80c9cf\n--- /dev/null\n+++ b/try_task_config.json\n@@ -0,0 +1,14 @@\n+{\n+    "parameters": {\n+        "optimize_target_tasks": false,\n+        "try_task_config": {\n+            "env": {\n+                "TRY_SELECTOR": "chooser"\n+            },\n+            "tasks": [\n+                "build-linux64/opt"\n+            ]\n+        }\n+    },\n+    "version": 2\n+}\n-- \n2.45.0\n\n'

So somewhere along the way, it looks like the file diff is no longer present.

Comment 2

[Mike Hommey [:glandium]]

Can you try this patch?

diff --git a/tools/tryselect/lando.py b/tools/tryselect/lando.py
index baca2f9fa6d42..74dc4a80cccf9 100644
--- a/tools/tryselect/lando.py
+++ b/tools/tryselect/lando.py
@@ -419,7 +419,7 @@ def push_to_lando_try(
     # Get the time when the push was initiated, not including Auth0 login time.
     push_start_time = time.perf_counter()
 
-    with vcs.try_commit(commit_message) as head:
+    with vcs.try_commit(commit_message, changed_files) as head:
         try:
             base_commit, patches = get_stack_info(vcs, head)
         except ValueError as exc:

Comment 3

[Mark Banner (:standard8)]

That works for me. Thank you.

Comment 4

[:gerard-majax]

(In reply to Mike Hommey [:glandium] from comment #2)

Can you try this patch?

diff --git a/tools/tryselect/lando.py b/tools/tryselect/lando.py
index baca2f9fa6d42..74dc4a80cccf9 100644
--- a/tools/tryselect/lando.py
+++ b/tools/tryselect/lando.py
@@ -419,7 +419,7 @@ def push_to_lando_try(
     # Get the time when the push was initiated, not including Auth0 login time.
     push_start_time = time.perf_counter()
 
-    with vcs.try_commit(commit_message) as head:
+    with vcs.try_commit(commit_message, changed_files) as head:
         try:
             base_commit, patches = get_stack_info(vcs, head)
         except ValueError as exc:

Thansk, it fixed it: https://treeherder.mozilla.org/jobs?repo=try&revision=6fae815c9c71ccc660c0c11952bdbed710a85e07

Comment 5

[Mike Hommey [:glandium]]

Attachment: Bug 1904777 - Add missing changed_files argument to try_commit for push_to_lando_try after bug 1500188. r?sheehan

Comment 6

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1500188

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/491c65d9bd8c
Add missing changed_files argument to try_commit for push_to_lando_try after bug 1500188. r=sheehan

Comment 8

[Atila Butkovits]

https://hg.mozilla.org/mozilla-central/rev/491c65d9bd8c

Comment 10

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1500188