1062305 - mcMerge intermittently silently fails to resolve bugs if it has to set the assignee and the hg user != bugzilla email

Status: RESOLVED FIXED

(Tree Management :: Bugherder, defect)

Description

[Ryan VanderMeulen [:RyanVM]]

We've seen reports over the past few weeks of bugs landed on m-c that failed to be resolved by mcMerge. Due to lack of any real debugging output and the fact that bug marking failures occur silently, we've only seen them when notified by developers.

From the few I've seen, I *think* it's related to bugs that aren't assigned to anybody. I'm wondering if it's hitting an error trying to set the assignee that we're not handling. Anyway, just a theory.

I think we need to add more debugging output into mcMerge so we can get visibility into bug marking issues. Once that's done, we can hopefully figure out what's going wrong here.

Comment 1

[Ryan VanderMeulen [:RyanVM]]

99% sure that this due to the author's email address and bmo account email address being different.

https://hg.mozilla.org/mozilla-central/rev/f3b0104fbe97 is another example of it happening. Matt uses his Moco email address for pushing, but his gmail address for bmo. mcMerge claims to have marked the bug and successfully set the assignee to [e-mail], but that obviously didn't take and nothing was done to the bug.

Comment 2

[Ryan VanderMeulen [:RyanVM]]

So the thing we have to be doing until this is fixed is to verify *any* bug where mcMerge is trying to set the assignee to make sure it was actually done successfully.

Comment 3

[Carsten Book [:Tomcat]]

https://tbpl.mozilla.org/?tree=Mozilla-Inbound&rev=467bd8c70ac0 is another example, this time it was hitting ehsan it seems

Comment 4

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

So, here's where assignee gets set via mc-merge: http://mxr.mozilla.org/webtools-central/source/tbpl/mcmerge/js/Step.js#166

But later in the file, it looks like the script should attempt to resubmit the change, dropping the assigned_to field if it failed the first time: http://mxr.mozilla.org/webtools-central/source/tbpl/mcmerge/js/Step.js#242

So if it is a problem with the assignee not matching the commit info, is that retry code not being called at all?

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+1]]

One other example as from the bug Ed duped here: https://hg.mozilla.org/mozilla-central/rev/1a6883cb1a09

Comment 7

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

Three more instances from bug 1074917, bug 1074862, and bug 1074854.

Comment 8

[Ed Morley [:emorley]]

(In reply to Wes Kocher (:KWierso) from comment #4)
> But later in the file, it looks like the script should attempt to resubmit
> the change, dropping the assigned_to field if it failed the first time:
> http://mxr.mozilla.org/webtools-central/source/tbpl/mcmerge/js/Step.js#242
> 
> So if it is a problem with the assignee not matching the commit info, is
> that retry code not being called at all?

Perhaps the bzapi compatibility layer doesn't return the same HTTP error code as the original bzapi did, if the changes were unable to me made due to invalid assignee?

> 242   if (where == 'submit' && msg == 'HTTP status 400') {

Comment 9

[Ed Morley [:emorley]]

s/me/be/

Comment 10

[Ed Morley [:emorley]]

Request URL: 	https://bugzilla.mozilla.org/bzapi/bug/1062305?<snip username and password>
Status Code: 	HTTP/1.1 200 OK

Request Body:
{"comments":[{"creation_time":"2014-10-27T19:43:12.769Z","creator":{"email":"emorley@<snip>.com"},"is_private":0,"text":"Testing"}],"id":"1062305","whiteboard":"","assigned_to":{"name":"trev.saunders@<snip>.com"},"last_change_time":"2014-10-27T19:28:45Z","update_token":"<snip>"}

Response Body:
{"documentation":"https://wiki.mozilla.org/Bugzilla:BzAPI","error":true,"code":51,"message":"There is no user named 'trev.saunders@<snip>.com'. Either you mis-typed the name or that user has not yet registered for a Bugzilla account."}

So seems like the new bzapi compat layer returns 200 even if the bug update PUT failed. We can either handle this in mcMerge or request a change upstream. Guess we should first try the latter and see if this was an intended change.

Comment 11

[Ed Morley [:emorley]]

Filed bug 1089805, let's see how that goes :-)

Comment 12

[Ed Morley [:emorley]]

Now that bug 1089805 has landed, I tried again...

STR:

1) Open https://tbpl.mozilla.org/mcmerge/?tree=mozilla-inbound&cset=c6fa46d62293 (changeset whose author doesn't have an identical Bugzilla username)
2) Remove the initial bug association & add bug 1069823 (has to be a bug with no assignee)
3) Submit changes

Unfortunately we're still getting 200s, see bug 1089805 comment 7.

Comment 13

[Ed Morley [:emorley]]

Return codes now being honoured regardless of "Accept-Encoding: gzip":

/c/src$ curl -s -i -X PUT https://bugzilla.mozilla.org/bzapi/bug/1069823 | head -n 1
HTTP/1.1 401 Authorization Required
/c/src$ curl -s -i -X PUT -H 'Accept-Encoding: gzip' https://bugzilla.mozilla.org/bzapi/bug/1069823 | head -n 1
HTTP/1.1 401 Authorization Required

Repeating the STR in comment 12 now works:

19:32:14.586 GET https://bugzilla.mozilla.org/bzapi/bug [HTTP/1.1 200 OK 705ms]
19:32:52.820 GET https://bugzilla.mozilla.org/bzapi/bug/1069823 [HTTP/1.1 200 OK 777ms]
19:32:53.606 OPTIONS https://bugzilla.mozilla.org/bzapi/bug/1069823 [HTTP/1.1 200 OK 183ms]
19:32:53.800 PUT https://bugzilla.mozilla.org/bzapi/bug/1069823 [HTTP/1.1 400 Bad Request 491ms]
19:32:54.295 GET https://bugzilla.mozilla.org/bzapi/bug/1069823 [HTTP/1.1 200 OK 752ms]
19:32:55.053 OPTIONS https://bugzilla.mozilla.org/bzapi/bug/1069823 [HTTP/1.1 200 OK 181ms]
19:32:55.242 PUT https://bugzilla.mozilla.org/bzapi/bug/1069823 [HTTP/1.1 200 OK 880ms]

...and resulted in bug 1069823 comment 3.