568941 - 500 Internal Server Error for http://hg.mozilla.org/labs/weave/

Status: RESOLVED FIXED

(mozilla.org Graveyard :: Server Operations, task)

Description

[Ed Lee :Mardak]

The web page seems to be broken but the repository is still good for clone/push/pull/etc.

The main page gives 500:

http://hg.mozilla.org/labs/weave/

The subpages kindof work but there's no content:

http://hg.mozilla.org/labs/weave/shortlog

Comment 1

[Aravind Gottipati [:aravind]]

The repo has some invalid tags on it.

.hgtags@bf99e5413493, line 89: tag '1.3rc3' refers to unknown node

This causes hgweb to break.  Let me know if you can fix this using ssh or if you want me to look at other options.

Comment 2

[Ed Lee :Mardak]

I removed the entry for 1.3rc3, but it still seems broken. I pushed a

http://hg.mozilla.org/labs/weave/rev/ec1c60b17b34

hgweb seems to work locally if I "hg serve"

Comment 3

[Aravind Gottipati [:aravind]]

hg serve doesn't choke on things.. but mod_wsgi which we use to serve hg.mozilla.org is very sensitive to these things.

Comment 4

[Ed Lee :Mardak]

Is it the same error breaking hgweb now that I removed the entry from .hgtags?

Comment 5

[Aravind Gottipati [:aravind]]

yes, I still see the tag.

Comment 6

[Ed Lee :Mardak]

I tried removing the bad revision id

http://hg.mozilla.org/labs/weave/rev/ec1c60b17b34
-69a1e61f5b893638380441eef2b55b629da95f86 1.3rc3


Then I tried adding a valid revision id
http://hg.mozilla.org/labs/weave/rev/1989192c107f
+c2cbb14ee03e9ec15eb026350d3f4ab173beb509 1.3rc3


But that doesn't seem to help either. The original revision id is stored in .hg/store/data/.hgtags.i

Comment 7

[Ed Lee :Mardak]

Well, it seems like adding revisions won't help the issue. We could remove revisions from the server repository but this runs into issues with mozilla-custom scripts for pushloghtml. And this wouldn't really prevent future issues of incorrect tag revisions.

Comment 8

[matthew zeier [:mrz]]

The fix here is to manually fix the changesets, something that's not directly an IT action and something anyone can do.

mardak, can you handle that?

Comment 9

[Ed Lee :Mardak]

How am I supposed to fix the changesets? hg client only provides ways to add more changesets but not edit any that already exist I believe.. ?

Comment 10

[Aravind Gottipati [:aravind]]

The last time I did it, I cloned it locally, cleaned out the repo on the server..

then I manually added back every single changeset with a hg export from the old repo, and a hg import into the new one (I didn't add the one, that created the bogus tag).

It was a painful laborious process..

Comment 11

[Ed Lee :Mardak]

I've got a clean version of the repository on my machine ready to push. Just let me know when the server copy will be wiped.

Comment 12

[Ed Lee :Mardak]

Zandr got rid of the repository and I pushed the fixed one.