802727 - Pushlog2.db files created by webheads

Status: RESOLVED FIXED

(Developer Services :: Mercurial: hg.mozilla.org, defect)

Description

[Ben Kero [:bkero]]

The pushlog2 hook is being run by our webheads.  It's being triggered as part of the changegroup hooks.  mpm (the lead hg dev) said that code is called exactly once in the mercurial core code.

The problem this is creating is that it's creating these pushlog files with incorrect permissions for them to operate (644 instead of 664).

http://www.selenic.com/hg/file/9837cafc25b1/mercurial/localrepo.py

We need to figure out exactly why it's being called on our webheads.

Ted, Shyam indicated that since you wrote the pushlog2 script you'll have some insight into why this might be happening.

Comment 1

[(not currently active) Ted Mielczarek]

I suspect this isn't the hook being run at all, but the /pushlog[html] web command being run before anything is pushed to the repo. The web command opens the db to try to query it:
http://hg.mozilla.org/hgcustom/pushlog/file/e99a36d3fd4a/pushlog-feed.py#l221

It looks like it tries to handle "the database doesn't exist", but in my local testing doing sqlite3.connect("nonexisting.file") succeeds and creates the file. Perhaps this behavior changed between pysqlite2 and the sqlite3 module that's included by default in Python 2.5+? Alternately, did the .hg dir used to be read-only on our webheads? (The changeset that added that code hints that that might have been the case: http://hg.mozilla.org/hgcustom/pushlog/rev/6db4ef76c839 )

Regardless, we could probably put a simple "if os.path.exists(pushdb):" around that connect, and fall back to the "conn = None" state to avoid this.

Comment 2

[Shyam Mani [:fox2mike]]

Ted, 

Woo. Thanks for that info. The webheads are temporarily rw to allow hg to leverage some of it's own internal caching to prevent things like try from melting down. We've since run into a nice set of issues which are unique to having the webheads rw :D 

So if there was a way to prevent that, awesome. puhslog2.db should only be created by ssh pushes (since that does it correctly with the right perms etc) and anything created by a webhead messes up commiting into the repo and we have to reset to fix.

Thanks!

Comment 3

[(not currently active) Ted Mielczarek]

I made that change and pushed it:
http://hg.mozilla.org/hgcustom/pushlog/rev/1c8f18f08846

If you update the pushlog webcommands to that changeset this should hopefully fix the issue. (I did write a test and it passed, so that's something.)

Comment 4

[Shyam Mani [:fox2mike]]

It still creates the pushlog2.db file, I deployed your changes to extensions/ and created a new repo projects/testing-802727 and if I hit http://hg.mozilla.org/projects/testing-802727/pushloghtml/ it creates pushlog2.db

Comment 5

[Shyam Mani [:fox2mike]]

Ted, what are the next steps here? This is really becoming a pain :| And we don't see the hg webheads going back ro at any time in the near future, so we need to prevent this from happening.

<3

Comment 6

[(not currently active) Ted Mielczarek]

I'm kind of stumped. That's the only place in the code where we open that database, so I don't see how that could happen anymore.

Comment 7

[(not currently active) Ted Mielczarek]

I also can't reproduce this in local testing. I made an empty test repo:
$ cd /tmp; mkdir hgrepo; cd hgrepo
$ cat > .hg/hgrc
[extensions]
hgwebjson = /Users/luser/build/pushlog/hgwebjson.py
pushlog-feed = /Users/luser/build/pushlog/pushlog-feed.py
buglink = /Users/luser/build/pushlog/buglink.py

[web]
templates=/Users/luser/build/hg_templates
style=gitweb_mozilla

$ hg serve
listening at http://fancy:8000/ (bound to *:8000)

Then I load http://localhost:8000/pushloghtml in my browser, it displays an empty pushlog, and:
$ ls -l /tmp/hgrepo/.hg/
total 24
-rw-r--r--  1 luser  wheel   57 Oct 24 08:14 00changelog.i
-rw-r--r--  1 luser  wheel  241 Oct 24 08:22 hgrc
-rw-r--r--  1 luser  wheel   33 Oct 24 08:14 requires
drwxr-xr-x  2 luser  wheel   68 Oct 24 08:14 store

Am I missing something about the hg.mo setup?

Comment 8

[Shyam Mani [:fox2mike]]

Ted, so the other day, I did update the pushlog-feed.py and clear out the pyc before I tested. It failed. Now it works flawlessly. Oh well, I'm going to mark this fixed then :)

Thanks a lot for your help!

Comment 9

[(not currently active) Ted Mielczarek]

Great!