try server: post submission patch queue edits can be problematic

RESOLVED INCOMPLETE

Status

Release Engineering
General
RESOLVED INCOMPLETE
6 years ago
7 months ago

People

(Reporter: joey, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [kanban:engops:https://mozilla.kanbanize.com/ctrl_board/6/2876] )

(Reporter)

Description

6 years ago
Modifying a patch queue [deletions] after a try submission has been made will not have good downstream effects:

http://tbpl.mozilla.org/php/getParsedLog.php?id=9826325&tree=Try

 pulling from http://hg.build.scl1.mozilla.com/try
  abort: unknown revision 'd1fa189b44cea5f8110c2f4789bffaee66ab132b'!

I created a shell script that would create a temporary patch containing only try args, submit then cleanup/remove the patch afterward but did not consider the patch/changeset would still be needed later for checkout.

Users should probably be given a heads up about the behavior.  As a line item in the commit hook message and on the try server wiki page.

ps> this may also be a contributor for the try-limbo job problem
(Reporter)

Updated

6 years ago
OS: Windows 7 → All
Hardware: x86_64 → All
Maybe I'm just being dense right now, but what exactly are the steps to repro this??

Once a patch/cset is pushed to try its in hg.m.o/try and exists! And what you do locally shouldn't matter.
(Reporter)

Comment 2

6 years ago
This bug is tied to another bug/behavior on the try server, which can be produced with simple hg push & queue ops.

In a nutshell I have a try-server script that will:
  o [ after all local edits have been included in a separate patch ]
  o create a temporary patch named "patch.try_$$"
  o Label the patch with hg qref --message "$try_server_ops"
  o hg push -f try
  o # qpop and remove temp patch

The result of these ops will *sometimes* place a job on the try server that will sit idle and not be processed (~try-limbo jobs) or have been in an odd state.  catlee has already sanity checked the script, nothing obvious as a problem source jumped out.

When logs have been available some reported failures attempting to checkout a changeset that was submitted.
Several try-limbo jobs have landed in the queue on the same day.  They sit idle in the queue while jobs submitted prior-to and following are processed.

Only basic hg commands are in use.
Event(s) happen randomly, no detail yet on a source for jobs sitting idle in the queue.

If you want to try out the try-submission script I can upload it -- but the behavior so far has been random.
I would love to at least see the script, and know what version of hg you have locally, and what OS you are using (in the slim case that matters).

Also would love a pushlog link to one or more try pushes where this fails/failed.
(Reporter)

Comment 4

6 years ago
The problem appears related to extra whitespace embedded within the try server args.
try server parser may need a little work:

Multiple jobs were submitted from the same sandbox and produced different results:

   try: -b do -e -p  macosx64 -u none -t none

two jobs were stuck in the queue
one submitted contained no source changes, only a changeset containing the try server args.
7:12 < catlee> so we call parse_known_args with a list of arguments created by processMessage(): http://hg.mozilla.org/build/buildbotcustom/file/a27b54ab52eb/try_parser.py#l133
07:12 < catlee> processMessage() is http://hg.mozilla.org/build/buildbotcustom/file/a27b54ab52eb/try_parser.py#l38
07:12 < catlee> it returns line[1].split(' ')
07:13 < catlee> so you'd end up with ['-p', ' ', 'macosx64']
07:13 < catlee> instead of ['-p', 'macosx64']
(Reporter)

Comment 6

6 years ago
Redundant "try: " checkin messages may be the other problem source.

A try push from a sandbox containing multiple patches:

[phantasm:734139] hg qapplied
patch.680246
patch.734121
patch.merge1
patch.734139
patch.readme

Produced a try submission containing two change sets and a single modified file.

Stray 'try: ' arguments are the only obvious oddity.  'hg qfold' has been run a few times on the patch set to accumulate the dup args.

[phantasm:patches] grep 'try:' *
patch.734139:try: -b do -e -p all -u none -t none
patch.734139:try: -b do -e -p all -u none -t none
patch.734139:try: -b do -e -p all -u all -t none

If multiple try: args are detected in the comment/argument stream might be able to either merge arguments with failure on conflicting switches.  Or fail outright requiring a single try: line in the top patch.
(Reporter)

Comment 7

6 years ago
patch.734139
============
bug 734139: makefiles updated to use mkdir function from bug 680246
* * *
try: -b do -e -p all -u none -t none
* * *
try: -b do -e -p all -u none -t none
* * *
try: -b do -e -p all -u all -t none

diff --git a/b2g/app/Makefile.in b/b2g/app/Makefile.in
--- a/b2g/app/Makefile.in
QA Contact: lsblakk → hwine
(Reporter)

Updated

5 years ago
Duplicate of this bug: 733046
(Assignee)

Updated

4 years ago
Product: mozilla.org → Release Engineering

Updated

3 years ago
Whiteboard: [kanban:engops:https://mozilla.kanbanize.com/ctrl_board/6/2869]

Updated

3 years ago
Whiteboard: [kanban:engops:https://mozilla.kanbanize.com/ctrl_board/6/2869] → [kanban:engops:https://mozilla.kanbanize.com/ctrl_board/6/2874]

Updated

3 years ago
Whiteboard: [kanban:engops:https://mozilla.kanbanize.com/ctrl_board/6/2874] → [kanban:engops:https://mozilla.kanbanize.com/ctrl_board/6/2876]
Status: NEW → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → INCOMPLETE
(Assignee)

Updated

7 months ago
Component: Tools → General
Product: Release Engineering → Release Engineering
You need to log in before you can comment on or make changes to this bug.