Closed Bug 968636 Opened 6 years ago Closed 5 years ago

set up a bzr server on a bugzilla community server

Categories

(Bugzilla :: bugzilla.org, defect)

defect
Not set

Tracking

()

RESOLVED FIXED

People

(Reporter: glob, Assigned: justdave)

References

(Blocks 1 open bug)

Details

mozilla IT are very keen on decommissioning the bugzilla server (bug 967642).
we're moving to git.mozilla.org (bug 929685).

i would like to propose the hosting of bzr.mozilla.org gets moved onto a bugzilla community server, for both the bzr server and loggerhead.  as the bugzilla project are the only users of bzr at mozilla, i would also like for the existing domain to point to the community hosted server.

this provides the following benefits:

- removes the responsibility of the server from mozilla IT's plate
- guarantees the commit history of the will still be available after the git migration (some repos may prove problematic, especially extensions)
- ensures existing bugzilla sites can continue to upgrade using the establishes processes

ldap auth will need to be replaced with local accounts, but as the server should be in read-only mode we only need accounts for the git->bzr mirroring script, and another account for admin reasons.
This doesn't block the git migration; we can move bzr to a community server later.
No longer blocks: 929685
Blocks: 983209
No longer blocks: 983209
as per the bugzilla meeting, assigning to justdave.

we probably don't need loggerhead, which may simplify the deployment.
Assignee: website → justdave
bzr.bugzilla.org now exists.   http://bzr.bugzilla.org/

Going to need the git-to-bzr mirror set up on this box before we can cut it over.
The script is here: https://github.com/markrcote/bugzilla-git-migration/blob/master/git-to-bzr.pl

You can get fubar to send over the current crontab on bzr.mozilla.org.
*/5 * * * * root /usr/bin/perl /root/bugzilla-git-migration/git-to-bzr.pl --from-repo=https://git.mozilla.org/bugzilla/bugzilla.git --from-branch=4.0 --to-repo=/var/www/html/bzr.mozilla.org/bugzilla/ --to-branch=4.0
*/5 * * * * root /usr/bin/perl /root/bugzilla-git-migration/git-to-bzr.pl --from-repo=https://git.mozilla.org/bugzilla/bugzilla.git --from-branch=4.2 --to-repo=/var/www/html/bzr.mozilla.org/bugzilla/ --to-branch=4.2
*/5 * * * * root /usr/bin/perl /root/bugzilla-git-migration/git-to-bzr.pl --from-repo=https://git.mozilla.org/bugzilla/bugzilla.git --from-branch=4.4 --to-repo=/var/www/html/bzr.mozilla.org/bugzilla/ --to-branch=4.4
*/5 * * * * root /usr/bin/perl /root/bugzilla-git-migration/git-to-bzr.pl --from-repo=https://git.mozilla.org/bugzilla/qa.git --from-branch=4.0 --to-repo=/var/www/html/bzr.mozilla.org/bugzilla/qa/ --to-branch=4.0
*/5 * * * * root /usr/bin/perl /root/bugzilla-git-migration/git-to-bzr.pl --from-repo=https://git.mozilla.org/bugzilla/qa.git --from-branch=4.2 --to-repo=/var/www/html/bzr.mozilla.org/bugzilla/qa/ --to-branch=4.2
*/5 * * * * root /usr/bin/perl /root/bugzilla-git-migration/git-to-bzr.pl --from-repo=https://git.mozilla.org/bugzilla/qa.git --from-branch=4.4 --to-repo=/var/www/html/bzr.mozilla.org/bugzilla/qa/ --to-branch=4.4
*/5 * * * * root /usr/bin/perl /root/bugzilla-git-migration/git-to-bzr.pl --from-repo=https://git.mozilla.org/bugzilla/bugzilla.git --from-branch=master --to-repo=/var/www/html/bzr.mozilla.org/bugzilla/ --to-branch=trunk
*/5 * * * * root /usr/bin/perl /root/bugzilla-git-migration/git-to-bzr.pl --from-repo=https://git.mozilla.org/bugzilla/extensions/Testopia.git --from-branch=master --to-repo=/var/www/html/bzr.mozilla.org/bugzilla/extensions/testopia --to-branch=trunk
Note that I think you can drop trunk.  I really don't see the point in mirroring it anymore, since we have testing set up on Travis CI from GitHub, and we aren't mirroring any new releases over.
OK, the mirroring script and cron jobs are in place, but I'm not sure how to verify if it's working or not...
You could add trunk temporarily and verify that new commits show up.
Mirroring was set up for trunk and appeared to be working for a while (after git was upgraded to support a log option used by the mirroring script), but bzr.bugzilla.org/bugzilla/trunk/ appears to have recently diverged from bzr.mozilla.org/bugzilla/trunk/ and git.mozilla.org/bugzilla/bugzilla.git master.  bzr.b.o has 9352 revisions, whereas bzr.m.o has 9363 revisions: revisions 9351-9353, 9355-9361, and 9363 appear to be missing from bzr.b.o.  These commits are confirmed on git.mozilla.org, so it appears to be a failure of the mirroring script on bzr.b.o.  justdave, did you see any errors?
Flags: needinfo?(justdave)
bzr: ERROR: Unrecognized bug mozilla:123278. Commit refused.
bzr: ERROR: Unrecognized bug mozilla:1093868. Commit refused.
bzr: ERROR: Unrecognized bug mozilla:1061226. Commit refused.
bzr: ERROR: Unrecognized bug mozilla:1143005. Commit refused.

etc.
Flags: needinfo?(justdave)
Those bugs do match up with missing commits on bzr.b.o.  Those bugs exist and are public.  Maybe there are network failures and bzr is giving an unhelpful error message?  What version of bzr is on there, and what version is on bzr.m.o?

In any case, the script shouldn't continue to mirror new revisions when one has failed.  I'll fix that in a separate bug.
Flags: needinfo?(justdave)
Would it make sense to just remove the commit check from the bzr commit hook? Since that would need to be getting done on the git side and nobody can directly commit to bzr anymore anyway?
Flags: needinfo?(justdave)
(In reply to Dave Miller [:justdave] (justdave@bugzilla.org) from comment #12)
> Would it make sense to just remove the commit check from the bzr commit
> hook? Since that would need to be getting done on the git side and nobody
> can directly commit to bzr anymore anyway?

+1
Next question is how do I do that? :)
Wait, this is due to the bug tracker not being defined at all in the bzr configuration on bzr.b.o.  The error is not particularly clear, but I found where it's being raised in the bzr code, and it's in response to a UnknownBugTrackerAbbreviation exception.  Glad they obfuscated that when the error is actually printed... anyway, we need the config file set up as it is on bzr.m.o (also see http://doc.bazaar.canonical.com/bzr.2.6/en/user-reference/bugs-help.html).
what config does that go in?  The only bazaar.conf on bzr2.scl3 appears to be specific to loggerheadd, I can't find a locations.conf, and every repo has a branch.conf in it already which was included in the original data rsync.
I have no idea, having not set it up or ever used the server directly.  See link in comment 15 if you just want to set it up anew on bzr.b.o.
err, is it /root/.bazaar.conf:
[DEFAULT]
email = Sync <sync@git.mozilla.org>
bugzilla_mozilla_url = https://bugzilla.mozilla.org
That would be it.
Oh, sorry, I didn't know there was a bug about this or I would have commented sooner. I actually added /root/.bazaar/bazaar.conf file on bzr.b.o server with the lines from comment 18 (except email one) about three days ago when I grew tired of deleting the webmaster spam. :) So the new server should now support "--fixes mozilla:xx" switch used in the mirror script.

BTW, this configuration information was in https://wiki.mozilla.org/Bugzilla:Committing_Patches page. Only in history now that it got switched to talk about git but luckily wiki has history diffing so I could dig it up from there. :)
justdave is now rsyncing the bzr repo back over from bzr.m.o, since we lost some earlier commits.  We should be good to go then, although at some point I want to fix the mirroring script so that it properly aborts if a commit fails.
I'm not sure there's anything left to do here unless Mark is using this as his reminder to fix that script so it aborts correctly when a commit fails.
There are blockers still open; I wouldn't say this is fully complete until we get a proper cert and redirect in place.
... although, for the purposes of decommissioning bzr.m.o, I think we can call this fixed.  I announced the switchover last week anyway.  I'm unblocking and resolving this; we can track the cert and the dependent redirect separately.
Status: NEW → RESOLVED
Closed: 5 years ago
No longer depends on: 1155525
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.