1484181 - Bookmarks on mozilla-unified aren't updating (fetching with cinnabar broken)

Status: RESOLVED FIXED

(Developer Services :: Mercurial: hg.mozilla.org, defect)

Description

[James Graham [:jgraham]]

This looks like a repeat of bug 1480034

https://hg.mozilla.org/integration/mozilla-inbound/ shows that tip is b84213ec5a4d, but https://hg.mozilla.org/mozilla-unified/rev/inbound points at 5a0c5138ce9e which suggests this broke sometime in the last 3 hours.

Comment 1

[James Graham [:jgraham]]

It looks like this started working again.

Comment 2

[Connor Sheehan [:sheehan]]

This is definitely a repeat in symptoms of bug 1480034, however the root cause is entirely different. In bug 1480034 we simply hadn't updated the unifyrepo service off of deprecated APIs before upgrading Mercurial to 4.6.2 on the ssh servers. In this instance the service is running to completion with no errors but is not working as intended. Arguably much worse!

I did make a change to the unifyrepo service last night (https://hg.mozilla.org/hgcustom/version-control-tools/rev/66e3ea5ac6ef) but it shouldn't have changed anything related to the repo unification, only the notifications for the service. The service continued to run overnight and only broke ~12h later. I came online this morning and reloaded the systemd units + restarted unifyrepo (`sudo systemctl daemon-reload && sudo systemctl restart unifyrepo.service`) for good measure, and the bookmarks seemed to update as normal shortly after. I doubt the restart actually solved the issue, but I thought I'd report on the off chance it did and someone sees this down the line.

Comment 3

[Kim Moir [:kmoir] ET]

Triaging, can this bug be closed or is there additional work to mitigate this event from occurring again?

Comment 4

[Connor Sheehan [:sheehan]]

I wrote tests for the unifyrepo extension in bug 1484231, through that I found a bug which may have caused this failure (source URL was set to the last repo in alphabetical order during unification, which may have resulted in the process incorrectly determining that no bookmarks needed to be updated). I'll close this bug as resolved and we can re-open/file a new bug and reference this one in the event it happens again.