968636 - set up a bzr server on a bugzilla community server

Status: RESOLVED FIXED

(Bugzilla :: bugzilla.org, defect)

Description

[:glob ✱]

mozilla IT are very keen on decommissioning the bugzilla server (bug 967642).
we're moving to git.mozilla.org (bug 929685).

i would like to propose the hosting of bzr.mozilla.org gets moved onto a bugzilla community server, for both the bzr server and loggerhead.  as the bugzilla project are the only users of bzr at mozilla, i would also like for the existing domain to point to the community hosted server.

this provides the following benefits:

- removes the responsibility of the server from mozilla IT's plate
- guarantees the commit history of the will still be available after the git migration (some repos may prove problematic, especially extensions)
- ensures existing bugzilla sites can continue to upgrade using the establishes processes

ldap auth will need to be replaced with local accounts, but as the server should be in read-only mode we only need accounts for the git->bzr mirroring script, and another account for admin reasons.

Comment 1

[Mark Côté [:mcote]]

This doesn't block the git migration; we can move bzr to a community server later.

Comment 2

[:glob ✱]

as per the bugzilla meeting, assigning to justdave.

we probably don't need loggerhead, which may simplify the deployment.

Comment 3

[Dave Miller [:justdave]]

bzr.bugzilla.org now exists.   http://bzr.bugzilla.org/

Going to need the git-to-bzr mirror set up on this box before we can cut it over.

Comment 4

[Mark Côté [:mcote]]

The script is here: https://github.com/markrcote/bugzilla-git-migration/blob/master/git-to-bzr.pl

You can get fubar to send over the current crontab on bzr.mozilla.org.

Comment 5

[Kendall Libby [:fubar] (he/him)]

*/5 * * * * root /usr/bin/perl /root/bugzilla-git-migration/git-to-bzr.pl --from-repo=https://git.mozilla.org/bugzilla/bugzilla.git --from-branch=4.0 --to-repo=/var/www/html/bzr.mozilla.org/bugzilla/ --to-branch=4.0
*/5 * * * * root /usr/bin/perl /root/bugzilla-git-migration/git-to-bzr.pl --from-repo=https://git.mozilla.org/bugzilla/bugzilla.git --from-branch=4.2 --to-repo=/var/www/html/bzr.mozilla.org/bugzilla/ --to-branch=4.2
*/5 * * * * root /usr/bin/perl /root/bugzilla-git-migration/git-to-bzr.pl --from-repo=https://git.mozilla.org/bugzilla/bugzilla.git --from-branch=4.4 --to-repo=/var/www/html/bzr.mozilla.org/bugzilla/ --to-branch=4.4
*/5 * * * * root /usr/bin/perl /root/bugzilla-git-migration/git-to-bzr.pl --from-repo=https://git.mozilla.org/bugzilla/qa.git --from-branch=4.0 --to-repo=/var/www/html/bzr.mozilla.org/bugzilla/qa/ --to-branch=4.0
*/5 * * * * root /usr/bin/perl /root/bugzilla-git-migration/git-to-bzr.pl --from-repo=https://git.mozilla.org/bugzilla/qa.git --from-branch=4.2 --to-repo=/var/www/html/bzr.mozilla.org/bugzilla/qa/ --to-branch=4.2
*/5 * * * * root /usr/bin/perl /root/bugzilla-git-migration/git-to-bzr.pl --from-repo=https://git.mozilla.org/bugzilla/qa.git --from-branch=4.4 --to-repo=/var/www/html/bzr.mozilla.org/bugzilla/qa/ --to-branch=4.4
*/5 * * * * root /usr/bin/perl /root/bugzilla-git-migration/git-to-bzr.pl --from-repo=https://git.mozilla.org/bugzilla/bugzilla.git --from-branch=master --to-repo=/var/www/html/bzr.mozilla.org/bugzilla/ --to-branch=trunk
*/5 * * * * root /usr/bin/perl /root/bugzilla-git-migration/git-to-bzr.pl --from-repo=https://git.mozilla.org/bugzilla/extensions/Testopia.git --from-branch=master --to-repo=/var/www/html/bzr.mozilla.org/bugzilla/extensions/testopia --to-branch=trunk

Comment 6

[Mark Côté [:mcote]]

Note that I think you can drop trunk.  I really don't see the point in mirroring it anymore, since we have testing set up on Travis CI from GitHub, and we aren't mirroring any new releases over.

Comment 7

[Dave Miller [:justdave]]

OK, the mirroring script and cron jobs are in place, but I'm not sure how to verify if it's working or not...

Comment 8

[Mark Côté [:mcote]]

You could add trunk temporarily and verify that new commits show up.

Comment 9

[Mark Côté [:mcote]]

Mirroring was set up for trunk and appeared to be working for a while (after git was upgraded to support a log option used by the mirroring script), but bzr.bugzilla.org/bugzilla/trunk/ appears to have recently diverged from bzr.mozilla.org/bugzilla/trunk/ and git.mozilla.org/bugzilla/bugzilla.git master.  bzr.b.o has 9352 revisions, whereas bzr.m.o has 9363 revisions: revisions 9351-9353, 9355-9361, and 9363 appear to be missing from bzr.b.o.  These commits are confirmed on git.mozilla.org, so it appears to be a failure of the mirroring script on bzr.b.o.  justdave, did you see any errors?

Comment 10

[Dave Miller [:justdave]]

bzr: ERROR: Unrecognized bug mozilla:123278. Commit refused.
bzr: ERROR: Unrecognized bug mozilla:1093868. Commit refused.
bzr: ERROR: Unrecognized bug mozilla:1061226. Commit refused.
bzr: ERROR: Unrecognized bug mozilla:1143005. Commit refused.

etc.

Comment 11

[Mark Côté [:mcote]]

Those bugs do match up with missing commits on bzr.b.o.  Those bugs exist and are public.  Maybe there are network failures and bzr is giving an unhelpful error message?  What version of bzr is on there, and what version is on bzr.m.o?

In any case, the script shouldn't continue to mirror new revisions when one has failed.  I'll fix that in a separate bug.

Comment 12

[Dave Miller [:justdave]]

Would it make sense to just remove the commit check from the bzr commit hook? Since that would need to be getting done on the git side and nobody can directly commit to bzr anymore anyway?

Comment 13

[:glob ✱]

(In reply to Dave Miller [:justdave] ([e-mail]) from comment #12)
> Would it make sense to just remove the commit check from the bzr commit
> hook? Since that would need to be getting done on the git side and nobody
> can directly commit to bzr anymore anyway?

+1

Comment 14

[Dave Miller [:justdave]]

Next question is how do I do that? :)

Comment 15

[Mark Côté [:mcote]]

Wait, this is due to the bug tracker not being defined at all in the bzr configuration on bzr.b.o.  The error is not particularly clear, but I found where it's being raised in the bzr code, and it's in response to a UnknownBugTrackerAbbreviation exception.  Glad they obfuscated that when the error is actually printed... anyway, we need the config file set up as it is on bzr.m.o (also see http://doc.bazaar.canonical.com/bzr.2.6/en/user-reference/bugs-help.html).

Comment 16

[Dave Miller [:justdave]]

what config does that go in?  The only bazaar.conf on bzr2.scl3 appears to be specific to loggerheadd, I can't find a locations.conf, and every repo has a branch.conf in it already which was included in the original data rsync.

Comment 17

[Mark Côté [:mcote]]

I have no idea, having not set it up or ever used the server directly.  See link in comment 15 if you just want to set it up anew on bzr.b.o.

Comment 18

[Kendall Libby [:fubar] (he/him)]

err, is it /root/.bazaar.conf:
[DEFAULT]
email = Sync <[e-mail]>
bugzilla_mozilla_url = https://bugzilla.mozilla.org

Comment 19

[Mark Côté [:mcote]]

That would be it.

Comment 20

[Teemu Mannermaa (:wicked)]

Oh, sorry, I didn't know there was a bug about this or I would have commented sooner. I actually added /root/.bazaar/bazaar.conf file on bzr.b.o server with the lines from comment 18 (except email one) about three days ago when I grew tired of deleting the webmaster spam. :) So the new server should now support "--fixes mozilla:xx" switch used in the mirror script.

BTW, this configuration information was in https://wiki.mozilla.org/Bugzilla:Committing_Patches page. Only in history now that it got switched to talk about git but luckily wiki has history diffing so I could dig it up from there. :)

Comment 21

[Mark Côté [:mcote]]

justdave is now rsyncing the bzr repo back over from bzr.m.o, since we lost some earlier commits.  We should be good to go then, although at some point I want to fix the mirroring script so that it properly aborts if a commit fails.

Comment 22

[Dave Miller [:justdave]]

I'm not sure there's anything left to do here unless Mark is using this as his reminder to fix that script so it aborts correctly when a commit fails.

Comment 23

[Mark Côté [:mcote]]

There are blockers still open; I wouldn't say this is fully complete until we get a proper cert and redirect in place.

Comment 24

[Mark Côté [:mcote]]

... although, for the purposes of decommissioning bzr.m.o, I think we can call this fixed.  I announced the switchover last week anyway.  I'm unblocking and resolving this; we can track the cert and the dependent redirect separately.