1321184 - Rewrite autoland history to remove all csets from bug 1304598

Status: RESOLVED FIXED

(Developer Services :: Mercurial: hg.mozilla.org, defect)

Description

[Daniel Holbert [:dholbert]]

Bug 1304598 renames some source files (with modifications), but as noted in bug 1304598 comment 59, the first round of commits accidentally landed as "delete + recreate from scratch" operations, rather than as file-move operations.

This destroys blame (i.e. the blame ends at this commit). And while it was backed out, the backout didn't (couldn't) help, unfortunately.

That's pretty darn bad. Fortunately this is just on "autoland" right now -- we should rewrite autoland's history to fix this BEFORE the badness leaks to any other branches, if at all possible.

So: filling this bug to strip out all csets associated with bug 1304598. (Particularly the initial landing in https://bugzilla.mozilla.org/show_bug.cgi?id=1304598#c49 and the backout of that initial landing. The second landing in https://bugzilla.mozilla.org/show_bug.cgi?id=1304598#c58 is OK to either leave in or backout, I think - whatever's easier.)

Comment 1

[Gregory Szorc [:gps]]

My plan is to `histedit` out the "bad" changesets by dropping them from history. Depending on if I use the code from bug 1288845, pushlog data may be lost. perfherder may get confused.

I would do this tonight but there is a release build in progress and I don't want to do anything that may potentially interfere with it. So as long as autoland isn't merged into mozilla-central, we should be good.

It's also safe to leave autoland open: everything should rebase properly. We may just lose some pushlog and perfherder data. If people are concerned about that, we can close the tree to mitigate impact.

Comment 2

[Gregory Szorc [:gps]]

flagging me so I remember to do this tomorrow :)

Comment 3

[Daniel Holbert [:dholbert]]

INSTRUCTIONS FOR HISTORY REWRITING:
Please remove these pushes' changesets from history (and from the pushlog):
 https://hg.mozilla.org/integration/autoland/pushloghtml?changeset=7aa621ebbcd8
 https://hg.mozilla.org/integration/autoland/pushloghtml?changeset=738d5a401498

(The latter push there is just a backout of the former; so in total, the fixup here should result in zero source-code changes -- it'll only change history/blame.)

Note also that 1304598's patches re-landed in a "good" form [with actual file-moves], in this push:
 https://hg.mozilla.org/integration/autoland/pushloghtml?changeset=e80263a030ca
That^^ push does NOT need to be removed from history, but feel free to nuke it if it makes things easier for some reason.

Comment 4

[Gregory Szorc [:gps]]

Also, Mercurial has an "automv" extension to detect renames and record the appropriate metadata to prevent this sort of thing. Bug 1313432 is open to have autoland use it. (That's another benefit of abstracting landing away from the user.)

Comment 5

[Daniel Holbert [:dholbert]]

(In reply to Gregory Szorc [:gps] from comment #1)
> It's also safe to leave autoland open: everything should rebase properly. We
> may just lose some pushlog and perfherder data. If people are concerned
> about that, we can close the tree to mitigate impact.

(I'm leaving it closed, to be on the safe side. It shouldn't be a bit deal to have one integration tree closed for ~1 day, given that we've got other places where people can land truly-time-critical stuff.)

Comment 6

[Ting-Yu Lin [:TYLin] (PDT, UTC-7)]

Gregory, Daniel, thank you for helping on the catastrophe that I caused.

Comment 7

[Gregory Szorc [:gps]]

I dropped the changesets using the extension from 1288845.

$ hg drop 04a37d89a469
dropping 04a37d89a469 (requested explicitly): Bug 1304598 Part 1 - Move PresShell to mozilla namespace. r=dholbert
dropping 4b127208e075 (shares push with dropped changeset): Bug 1304598 Part 2 - Strip "dom::" prefix by using namespace in TouchManager.cpp r=dholbert
dropping 25df85d242fc (shares push with dropped changeset): Bug 1304598 Part 3 - Rename nsPresShell.h/cpp to PresShell.h/cpp, and move exported header to mozilla/ subdir. r=dholbert
dropping 4e44b3464585 (shares push with dropped changeset): Bug 1304598 Part 4 - Sort #include statements in TouchManager.cpp and PresShell.h. r=dholbert
dropping 333e95c0f86c (shares push with dropped changeset): Bug 1304598 Part 5 - Move ViewportFrame to mozilla namespace. r=dholbert
dropping c8e3fc3b49ea (shares push with dropped changeset): Bug 1304598 Part 6 - Rename nsViewportFrame.h/cpp to ViewportFrame.h/cpp, and move exported header to mozilla/ subdir. r=dholbert
dropping 7aa621ebbcd8 (shares push with dropped changeset): Bug 1304598 Part 7 - Move BRFrame to mozilla namespace, and rename nsBRFrame.cpp to BRFrame.cpp. r=dholbert
dropping f8ab8a670d83 (changes impacted files): Backed out changeset 7aa621ebbcd8 (bug 1304598) for developer's request
dropping e88ab6472bea (shares push with dropped changeset): Backed out changeset c8e3fc3b49ea (bug 1304598)
dropping 8e817af2bcc9 (shares push with dropped changeset): Backed out changeset 333e95c0f86c (bug 1304598)
dropping 5ed368ce1d94 (shares push with dropped changeset): Backed out changeset 4e44b3464585 (bug 1304598)
dropping 807662e8fab3 (shares push with dropped changeset): Backed out changeset 25df85d242fc (bug 1304598)
dropping f913e55832ea (shares push with dropped changeset): Backed out changeset 4b127208e075 (bug 1304598)
dropping 738d5a401498 (shares push with dropped changeset): Backed out changeset 04a37d89a469 (bug 1304598)
grafted 72eb22d75bca -> a6aabcb07a6e: bug 1320860 adjust textarea style contexts for GTK 3.20 theming ABI changes r=stransky+263117
grafted 47eaf2d43862 -> e6f4b042c540: Merge m-c to autoland
dropping f546165fe714 (changes impacted files): Bug 1304598 Part 1 - Move PresShell to mozilla namespace. r=dholbert
dropping 5dfc74c1cfd5 (shares push with dropped changeset): Bug 1304598 Part 2 - Strip "dom::" prefix by using namespace in TouchManager.cpp r=dholbert
dropping 2f95fbcdc1f5 (shares push with dropped changeset): Bug 1304598 Part 3 - Rename nsPresShell.h/cpp to PresShell.h/cpp, and move exported header to mozilla/ subdir. r=dholbert
dropping d4a309b80849 (shares push with dropped changeset): Bug 1304598 Part 4 - Sort #include statements in TouchManager.cpp and PresShell.h. r=dholbert
dropping e7684174562b (shares push with dropped changeset): Bug 1304598 Part 5 - Move ViewportFrame to mozilla namespace. r=dholbert
dropping c46d257108fd (shares push with dropped changeset): Bug 1304598 Part 6 - Rename nsViewportFrame.h/cpp to ViewportFrame.h/cpp, and move exported header to mozilla/ subdir. r=dholbert
dropping e80263a030ca (shares push with dropped changeset): Bug 1304598 Part 7 - Move BRFrame to mozilla namespace, and rename nsBRFrame.cpp to BRFrame.cpp. r=dholbert
grafted 09b480229ec2 -> 2176b72e7e2f: Bug 1318800 fix systemPrincipal checks so top level tab loads work with webrequests, r=kmag
grafted 0fe1c7f17e68 -> f9278a331057: Bug 1320762 - fix compare-mozconfig checks; r=rail
grafted 698893ea26f8 -> f6cc35920bf2: Backed out changeset 10bb1ad646cd (bug 1321065) for ASan gtest failures
dropping 6d08064edd93 (changes impacted files): Bug 1321157 - Remove some unnecessary stylo NS_ERROR in nsDocument. r=heycam
creating new pushlog entry #34795 by [e-mail] from entry #34788
creating new pushlog entry #34796 by [e-mail] from entry #34789
creating new pushlog entry #34797 by [e-mail] from entry #34791
creating new pushlog entry #34798 by [e-mail] from entry #34792
creating new pushlog entry #34799 by [e-mail] from entry #34793
autoland is CLOSED! Reason: Closed for Work on Bug 1321184 ETA for reopening is sometime Nov. 30th - more details in the bug - Please don't do anything (like merge etc) with autoland!!
But you included the magic words.  Hope you had permission!

View your change here:
  https://hg.mozilla.org/integration/autoland/rev/f6cc35920bf232c5aa2a91606247271214eb6ec8

Follow the progress of your build on Treeherder:
  https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=f6cc35920bf232c5aa2a91606247271214eb6ec8
recorded changegroup in replication log in 0.003s
recorded updates to obsolete in replication log in 0.002s

Comment 8

[Daniel Holbert [:dholbert]]

I'm seeing useful blame here now, for two of the files whose blame was previously horked:
  https://hg.mozilla.org/integration/autoland/annotate/tip/layout/base/nsPresShell.cpp
  https://hg.mozilla.org/integration/autoland/annotate/tip/layout/generic/nsBRFrame.cpp

(And for people wanting to check the blame in the future, those files will soon be renamed in bug 1304598:
  https://hg.mozilla.org/integration/autoland/annotate/tip/layout/base/PresShell.cpp
  https://hg.mozilla.org/integration/autoland/annotate/tip/layout/generic/BRFrame.cpp
...but for now these latter 2 links are to nonexistent files)

So, I think this is fixed, as far as I can tell. Thanks, gps!

Comment 9

[Gregory Szorc [:gps]]

So, `hg drop` worked like a charm! The only real fallout I saw from that is duplicate push entries on Treeherder. But we knew about that (that is tracked in bug 1288845).

Unfortunately, all queued autoland requests failed. The autoland daemon logs reveal a faulty server-side hook:

remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 3 changesets with 21 changes to 21 files
remote:
remote:
remote: ************************** ERROR ****************************
remote: Multiple heads detected on branch 'default'
remote: Only one head per branch is allowed!
remote: *************************************************************
remote:
remote:
remote: transaction abort!
remote: rollback completed
remote: pretxnchangegroup.b_singlehead hook failed

It looks like the multiple heads per branch hook is failing to account for hidden changesets. Doh.

Rather than rewrite the hook on the spot, I decided the safest thing to do was strip the hidden changesets from the repo so the hook would no longer complain. However, when I ran `hg strip`, the stripping failed midway through due to a permissions problem. I was running with `sudo -u hg` and apparently those permissions were not enough for some files in the repo. This essentially corrupted the pushlog on the ssh server. So, I copied the autoland repo from an hgweb machine to the hgssh machine, swapped it in, did a `hg strip` as root, fixed permissions, and everything seems to be back to normal.

I'll see about getting bugs on file for the secondary issues we ran into as a result of doing the rewriting.