Closed Bug 758003 Opened 9 years ago Closed 8 years ago

Releng needs for testing production git

Categories

(Developer Services :: General, task)

x86
macOS
task
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: hwine, Assigned: bkero)

References

Details

Attachments

(1 file)

From email thread with cshields & joduinn. Most needed items are at top.

From: Hal Wine <hwine@mozilla.com>
Subject: Re: Testing access for git.m.o requirements
Date: May 23, 2012 13:49:07 PDT
To: Corey Shields <cshields@mozilla.com>
Cc: John O'Duinn <joduinn@mozilla.com>

Corey,

I may have obfuscated my needs at this point, so wanted to clarify. To start testing, I need:
 - url to new host, and any relevant connection info (vpn, etc.)
 - 4 repos to test against: 1 each r/o & r/w on hg.m.o and the new git server

The the non-technical issues can be worked out in parallel.

Please let me know who will be working on this, and where the work is tracked. My interest is in having a smooth handoff, so I'll be following the old "over communicate" rule.

--Hal

P.S. I know the dc move was the main focus for quite a while - here's an earlier email on this topic (subject: Touching base on git.m.o; date: 2012-04-20)

On May 22, 2012, at 13:49 , Hal Wine wrote:

Corey,

Before I get into the technical questions, let me ask how we're going to handle the policy questions. 

My understanding at the time I did the original git on allizom eval was that the server would be somehow partitioned into "release managed" and "IT managed" parts. Releng would be responsible for all hooks, etc. installed for the "release managed repos". My current understanding is that IT will be managing all parts of git.m.o, just as they do for hg.m.o.

If IT is managing all of git.m.o, then I mostly need to understand how the requirements John has previously relayed will be implemented, so I can verify they match our expectations. As I understand the requirements:
release owned repositories are all those under hg.m.o/{releases,build,projects,integration,l10n,l10n-central} + mozilla-central + try
a similar "name space" structure will be available on git.m.o for (possible) future git versions of these repositories
any "equivalent" repo will be set up to have only one repo (hg or git) be committable (except by an internal release user to maintain the mirroring) (i.e. one will be the "repository of record" and the other a read-only version)
for release owned repositories, the present rule is hg is the repository of record

At the technical level, to verify the above, I'll need 4 repos created to fill the matrix of permissions (2 on each server, one r/o, the other r/w). I can execute various workflows against those test repositories to validate expectations about commit levels, etc. It would be helpful to be able to read the hook code, just to ensure we don't go planning in an incompatible direction without knowing it's going to be a major headache.

Once the validation work is complete, then work can begin adding repos to git.m.o and make them live (in r/o mode) and see how things go from a load perspective.

--Hal
Depends on: 528360
The host running these is not currently routed to the internet, so a VPN connection to our SCL3 datacentre is required.  The hostname is git1.dmz.scl3.mozilla.com, and the gitweb interface is http://git1.dmz.scl3.mozilla.com/git/

git cloning can be done by doing 'git clone gitolite@git1.dmz.scl3.mozilla.com:test_hwine.git'

These repos were created for hg:

drwxr-sr-x 3 hwine@mozilla.com scm_level_2 4.0K May 29 14:12 test_hwine
drwxrwsr-x 3 hg                scm_level_2 4.0K May 29 14:13 test_scm2

and these repos were created for git.  Please note that the presence of the owner/desc params means that the repositories are public and will be listed on gitweb:

gitolite::repo {
        "test_hwine":
            rwplus => "hwine@mozilla.com",
            owner  => "Hal Wine",
            desc   => "Hal's git/hg test plaything";
 
        "test_scm2":
            rwplus => "@scm_level_2",
            owner  => "Hal Wine",
            desc   => "Hal's git/hg test plaything";
}
Hal, please let us know if you're blocked on anything else at this point.
Shyam - will do - I have a release to get started this morning, then I'll be looking at the setup for the first time, since bug 759517 was fixed last night.
confirmed I have write access to the repos, so can begin full testing Friday.

Note: something isn't quite right with the web access (not needed for my initial testing):
Thu May 31 16:54:58 PDT 2012
0 [Hal@Hals-MacBook-Air test_1]
$ curl http://git1.dmz.scl3.mozilla.com/git/
<h1>Software error:</h1>
<pre>syntax error at /etc/gitweb.conf line 35, near &quot;$feature&quot;
</pre>
<p>
For help, please send mail to the webmaster (<a href="mailto:root@localhost">root@localhost</a>), giving this error message 
and the time and date of the error.

</p>
0 [Hal@Hals-MacBook-Air test_1]
Yes, the web access is a known issue and Ben is looking into it.
(In reply to Ben Kero [:bkero] from comment #1)
> These repos were created for hg:
> 
> drwxr-sr-x 3 hwine@mozilla.com scm_level_2 4.0K May 29 14:12 test_hwine
> drwxrwsr-x 3 hg                scm_level_2 4.0K May 29 14:13 test_scm2
> 
> and these repos were created for git.  Please note that the presence of the
> owner/desc params means that the repositories are public and will be listed
> on gitweb:
> 
> gitolite::repo {
>         "test_hwine":
>             rwplus => "hwine@mozilla.com",
>             owner  => "Hal Wine",
>             desc   => "Hal's git/hg test plaything";
>  
>         "test_scm2":
>             rwplus => "@scm_level_2",
>             owner  => "Hal Wine",
>             desc   => "Hal's git/hg test plaything";
> }

Confirmed that access works as desired w.r.t. restricting write access to named user in test_hwine on both git & hg.
Assignee: server-ops-devservices → bkero
Per email exchange, please create a new account with no SCM level permissions, and attached ssh key as auth.

Then change ownership on the two test repos (test_hwine on hg & git) to be writable only by this key.

Thanks!
What username for this account?
Assignee: bkero → jdow
Shoot - thought a username suggestion was in there, but not.

How about: mirror-repo or repo-sync or hgsyncgit. Ping me in channel if none of those work.
(In reply to Justin Dow [:jabba] from comment #8)
> What username for this account?

Let's go with github-sync-test for now.
I've created:

uid=github-sync-test,ou=logins,dc=mozilla
mail=github-sync-test@mozilla.com

with the attached SSH key and added the hgAccount objectClass to it, but did not put the user in any SCM_level groups. I believe this is all that is required on my end. Punting back to bkero for the repo work.
Assignee: jdow → bkero
I've created a repo for you with the following permissions:

repo    test_github-sync
    RW+     = github-sync-test@mozilla.com
    R       = @all gitweb
    test_github-sync "Hal Wine" = "Github sync testing"

In English that means the name of the repo is 'test_github-sync', which is writable only by the github-sync-test@mozilla.com user, and is readable by @all and gitweb.
Blocks: 768730
What is still required to close this bug out?
I believe this one is ready to close.
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Depends on: 769148
reopening due to bug 769148 - that will be part of the test case now
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
769148 is fixed
Status: REOPENED → RESOLVED
Closed: 9 years ago9 years ago
Resolution: --- → FIXED
Hal,

I'd like a new account or this account name changed before we go "live". It should just be github-sync@mozilla.com. Thoughts? Let's discuss on IRC before we go live.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
(In reply to Shyam Mani [:fox2mike] from comment #17)
> Hal,
> 
> I'd like a new account or this account name changed before we go "live". It
> should just be github-sync@mozilla.com. Thoughts? Let's discuss on IRC
> before we go live.

discussed - not a go live blocker, so moved to bug 770576

rest good
Status: REOPENED → RESOLVED
Closed: 9 years ago8 years ago
Resolution: --- → FIXED
Component: Server Operations: Developer Services → General
Product: mozilla.org → Developer Services
You need to log in before you can comment on or make changes to this bug.