Closed Bug 778717 Opened 12 years ago Closed 12 years ago

investigate why active svn committers got their account deactivated lately

Categories

(Developer Services :: General, task)

Product:
Developer Services
Component:
General
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: pascalc, Assigned: fox2mike)

References

Details

I am noticing that many people, mostly localizers but not only, who are active committers to svn got their svn commit rights deactivated in the last weeks, here is a few recent examples:
bug 774538
bug 778713
bug 778710
byg 778254
Bug 776893
Bug 772543
Bug 771171
Bug 764749

Can you investigate if there is not a problem with the way accounts are marked as dormant? It should be after 6 months of inactivity, but it looks more like 6 days recently.

Can you also provide the list of localizers accounts that are currently deactivated for inactivity so as that I can review it and, if needed, ask for some accounts to be reactivated?

Thanks
I should have a log of all the deactivations. I'll re-enable all of them for the past month and then investigate what is broken.
Assignee: server-ops → jdow
Summary: invesyigate why active svn committers got their account deactivated lately → investigate why active svn committers got their account deactivated lately
I've reenabled the following accounts:

dn: mail=edu@eduard-gamonal.net,o=net,dc=mozilla
dn: mail=kkemenczy@gmail.com,o=net,dc=mozilla
dn: mail=francesco.lodolo@mozillaitalia.org,o=net,dc=mozilla
dn: mail=ibai@oihanguren.com,o=net,dc=mozilla
dn: mail=fernandopsilveira@yahoo.com.br,o=net,dc=mozilla
dn: mail=kristjanbjarni@gmail.com,o=net,dc=mozilla
dn: mail=valastiak@mozilla.sk,o=net,dc=mozilla
dn: mail=smolejv@gmx.net,o=net,dc=mozilla
dn: mail=petercpg@mail.moztw.org,o=net,dc=mozilla
dn: mail=malditoastur@gmail.com,o=net,dc=mozilla
dn: mail=fwenzel@mozilla.com,o=com,dc=mozilla
dn: mail=steven@silverorange.com,o=net,dc=mozilla
dn: mail=jasnapaka@jasnapaka.com,o=net,dc=mozilla
dn: mail=odvarko@gmail.com,o=net,dc=mozilla
dn: mail=aking@mozilla.com,o=com,dc=mozilla
dn: mail=mdinic@mozilla.com,o=com,dc=mozilla
dn: mail=raul.malea@gmail.com,o=net,dc=mozilla
dn: mail=mautematico@mozilla-mexico.org,o=net,dc=mozilla
dn: mail=clochix@clochix.net,o=net,dc=mozilla
dn: mail=inmitabarrios@gmail.com,o=net,dc=mozilla
dn: mail=rrosario@mozilla.com,o=com,dc=mozilla
dn: mail=ross@mozillafoundation.org,o=org,dc=mozilla
dn: mail=dboswell@mozilla.com,o=com,dc=mozilla
dn: mail=jdennis@redhat.com,o=net,dc=mozilla
dn: mail=lcrouch@mozilla.com,o=com,dc=mozilla
dn: mail=vannak.eng@gmail.com,o=net,dc=mozilla
dn: mail=witia@aviary.pl,o=net,dc=mozilla
dn: mail=ankit@redhat.com,o=net,dc=mozilla
dn: mail=besnik@programeshqip.org,o=net,dc=mozilla
dn: mail=kerim@mozilla.ba,o=net,dc=mozilla
dn: mail=fiotakis@otenet.gr,o=net,dc=mozilla
dn: mail=joergenr@stofanet.dk,o=net,dc=mozilla
dn: mail=eu@keko.me,o=net,dc=mozilla
dn: mail=leszekz@gmail.com,o=net,dc=mozilla
dn: mail=armenzg@mozilla.com,o=com,dc=mozilla
dn: mail=bogomil@efb.bg,o=net,dc=mozilla
dn: mail=aricaud@mozilla.com,o=com,dc=mozilla
dn: mail=ibrahima.sarr@pulaagu.com,o=net,dc=mozilla
dn: mail=sebastianzartner@gmx.de,o=net,dc=mozilla
dn: mail=eduardo@esperanto.org.uy,o=net,dc=mozilla
dn: mail=mozilla_by@poczta.fm,o=net,dc=mozilla
dn: mail=lebedel.delphine@gmail.com,o=net,dc=mozilla
dn: mail=mozilla_by@poczta.fm,o=net,dc=mozilla
dn: mail=lebedel.delphine@gmail.com,o=net,dc=mozilla
dn: mail=dtownsend@mozilla.com,o=com,dc=mozilla
dn: mail=tim.babych@gmail.com,o=net,dc=mozilla
dn: mail=sergiy.bogdancev@gmail.com,o=net,dc=mozilla
dn: mail=clegnitto@mozilla.com,o=com,dc=mozilla
dn: mail=joanna.niklas@gmail.com,o=net,dc=mozilla
dn: mail=w.benes@gmail.com,o=net,dc=mozilla
dn: mail=wreynolds@mozilla.com,o=com,dc=mozilla
dn: mail=jurk@linuxcsb.org,o=net,dc=mozilla
dn: mail=tomer@gmx.net,o=net,dc=mozilla
dn: mail=m@owca.info,o=net,dc=mozilla
dn: mail=linux.anas@gmail.com,o=net,dc=mozilla
dn: mail=stephen@silverorange.com,o=net,dc=mozilla
dn: mail=matthew.gertner@gmail.com,o=net,dc=mozilla
dn: mail=prasad.mvs@gmail.com,o=net,dc=mozilla

and I disabled the cronjob that disables accounts until we can determine the problem.
Thanks Justin! I ca
Thanks Justin! I can confirm that all the localizers in this list are active people.
I have no idea what the script does so this is just a wild guess.

How long is the inactivity period? Maybe since the switch to SVN 1.7, the files that were previously touched to determine activity are not touched anymore?
(In reply to Anthony Ricaud (:rik) from comment #6)
> How long is the inactivity period? 

6 months
http://www.mozilla.org/hacking/committer/#dormant-accounts
So, for technical details, the scope of the problem is likely limited to commits of https. SSH committers don't seem to be affected. Here is how the system works:

when you commit over ssh (svn+ssh), the sshd on the svn servers pushes a custom shell that only allows svn commands. As part of the authentication process, it also sends an update to the LDAP master that updates the user's svnAccessDate attribute with a current time stamp.

when you commit over https, it is a WebDAV service fronted by apache that does the svn functionality. In this case, the update to LDAP happens via a commit hook.

Every day there is a cron job on the ldap master that reads each user's svnAccessDate timestamp and if it is older than 6 months, it will change svnAccountEnabled from TRUE to FALSE.

I think all the users that were disabled are users generally using https://svn.mozilla.org/ rather than svn+ssh://svn.mozilla.org. So this leads me to think that the issue is only with the commit hook, but not the custom ssh shell.
Also, in bug 778723 comment 1, I see similar behaviour for hg, at least for one account.
Assignee: jdow → server-ops-devservices
Component: Server Operations → Server Operations: Developer Services
QA Contact: phong → shyam
Severity: major → normal
Assignee: server-ops-devservices → shyam
Depends on: 799608
So after poking at this quite a bit and doing some debugging, the issue here was that the date formats on LDAP were different for the svnAccessDate attribute. I've filed bug 799608 to fix that.

There was also an in-explicable problem with the svn pre-commit hooks, for some reason the hook updating the svnAccessDate wasn't running and I don't really have a lead as to why. I changed the hook ordering when I was debugging and it just started working, out of the blue. It's working fine now.

We'll add some monitoring to the system to keep tabs and make sure this doesn't happen again.
Sounds good, thanks Shyam!
Shyam, we are good to re-enable the script now. Please at your leisure, apply this patch in our puppet:

jabba@jabbamini:~/svn/puppet/trunk/modules/openldap/files/scripts> svn diff vcs_accounts_disable.py 
Index: vcs_accounts_disable.py
===================================================================
--- vcs_accounts_disable.py	(revision 50680)
+++ vcs_accounts_disable.py	(working copy)
@@ -1,7 +1,7 @@
 #! /bin/env python
-import sys
+#import sys
 
-sys.exit(0)
+#sys.exit(0)
 
 import ldap, datetime
I re-enabled the disable script. Things should be working correctly again and only disabling people after 6 months of inactivity.
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
Component: Server Operations: Developer Services → General
Product: mozilla.org → Developer Services
You need to log in before you can comment on or make changes to this bug.