1549693 - Backout of patches causes phabricator revision to get reopened twice

Status: RESOLVED FIXED

(Conduit :: Phabricator, defect, P1)

Description

[Kartikaya Gupta (email:[e-mail])]

See e.g. https://phabricator.services.mozilla.com/D29687

This landed yesterday, then was backed out, and the revision was reopened automatically. I relanded it since it was backed out incorrectly, and it merged to m-c. However phabricator detected "another backout" when the original backout got merged to the rMCU repository (what is that, mozilla-unified?) and reopened the revision even though it should remain closed.

Comment 1

[Mark Banner (:standard8)]

These have also done the same:

https://phabricator.services.mozilla.com/D29862
https://phabricator.services.mozilla.com/D28856

(a landing which didn't get backed out in-between didn't get reopened).

Comment 3

[Steven MacLeod [:smacleod]]

I'll look into a temp workaround until we can fix it properly.

Comment 4

[Steven MacLeod [:smacleod]]

Looks like there isn't an easy fix here (unless we disable re-opening entirely). The best way forward is to start the already planned process of moving commit syncing & reopening to unified, right away. The plan is to:

• Switch integration/autoland to be a publishing repository (Bug 1538695).
• Add integration/autoland to the list of repos in mozilla-unified.
• Update Lando to treat rMCU[1] targeted revisions as landing to integration/autoland.
  • Other branches and uplifts will be done using dummy repo targets on Phabricator.
• Update phabricator-repo-monitor to check for sync lag using rMCU and mozilla-unified rather than rMOZILLACENTRAL[2] and mozilla-central.
• Re-enable auto closing revisions for rMCU.
• Disable commit syncing for rMOZILLACENTRAL.

[1]rMCU - https://phabricator.services.mozilla.com/source/mcu/
[2]rMOZILLACENTRAL - https://phabricator.services.mozilla.com/source/mozilla-central/

Comment 5

[Steven MacLeod [:smacleod]]

Turns out switching the Phabricator repo used will have some problematic complications with Lando. Stacks mixing the repository they are targeting (Lando only allows landing revisions together which target the same repo) will appear when things are uploaded for rMOZILLACENTRAL but then changed to rMCU on close.

Revised plan:

• Switch integration/autoland to be a publishing repository (Bug 1538695).
• Add integration/autoland to the list of repos in mozilla-unified.
• Update Lando to treat rMCU[1] targeted revisions as landing to integration/autoland.
• Keep using rMOZILLACENTRAL[2] to target Lando at integration/autoland.
  • Other branches and uplifts will be done using dummy repo targets on Phabricator.
• Update phabricator-repo-monitor to check for sync lag using rMCU and mozilla-unified rather than rMOZILLACENTRAL and mozilla-central.
• Re-enable auto closing revisions for rMCU.
• Disable commit syncing for rMOZILLACENTRAL.
• Disable commit syncing for rMCU.
• Switch rMOZILLACENTRAL to observe mozilla-unified rather than integration/autoland.

Switching rMOZILLACENTRAL's observation target to unified may cause a temporary delay in some auto-closing / re-opening while the ~60k commit difference is synced. I will talk to :ckolos and attempt this on phabricator-dev to get an idea for the lag.

[1]rMCU - https://phabricator.services.mozilla.com/source/mcu/
[2]rMOZILLACENTRAL - https://phabricator.services.mozilla.com/source/mozilla-central/

Comment 6

[Steven MacLeod [:smacleod]]

(In reply to Steven MacLeod [:smacleod] from comment #5)

• Disable commit syncing for rMCU.

I have done this and it should prevent these double re-opens for now.

• Switch rMOZILLACENTRAL to observe mozilla-unified rather than integration/autoland.

Switching rMOZILLACENTRAL's observation target to unified may cause a temporary delay in some auto-closing / re-opening while the ~60k commit difference is synced. I will talk to :ckolos and attempt this on phabricator-dev to get an idea for the lag.

This is currently running in dev. Based on the current rate I'm seeing, I estimate the sync will take between 5 and 9 hours to complete. This means when run on production we'll have a decent period of time where revisions won't be auto-closed or re-opened.

Comment 7

[Steven MacLeod [:smacleod]]

Documenting the procedure to change the syncing repo:

1. Turn off syncing from integration/autoland for rMOZILLACENTRAL
   • https://phabricator.services.mozilla.com/source/mozilla-central/uri/edit/220/
   • Change "I/O" Type from Observe: Copy from a remote. to No I/O: Do not perform any I/O.
   • "Save Changes"
2. Turn on syncing from mozilla-unified for rMOZILLACENTRAL
   • https://phabricator.services.mozilla.com/source/mozilla-central/uri/edit/291/
   • Change "I/O" Type from No I/O: Do not perform any I/O. to Observe: Copy from a remote.
   • "Save Changes"
3. Monitor progress of commit sync on phd.
   • https://phabricator.services.mozilla.com/daemon/
   • Can watch for PhabricatorRepositoryMercurialCommitChangeParserWorker tasks being queued and processed
4. Verify commit sync has completed.
   • https://phabricator.services.mozilla.com/source/mozilla-central/manage/
   • Check for Updates OK and last update time being recent and repeatedly updating every ~15 seconds.

From start to finish the process will most likely take somewhere between 6 and 10 hours.

Comment 8

[Steven MacLeod [:smacleod]]

This should be fixed now.