718492 - Camino nightly builds failing with constant hg "premature EOF reading chunk" errors since 12am 16 Jan

Status: VERIFIED FIXED

(mozilla.org Graveyard :: Server Operations, task)

Description

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

Since this morning's attempt to generate a (clobber) nightly, Camino's nightly builder has been red due to constant EOF failures from hg.mozilla.org.

The first build timed-out trying to clone mozilla-1.9.2 and gave a (different) failure trying to clone camino:

Begin: Mon Jan 16 00:03:01 2012
hg clone  http://hg.mozilla.org/releases/mozilla-1.9.2/ mozilla 
Process killed. Took 2 seconds to die.
End:   Mon Jan 16 00:16:22 2012
Begin: Mon Jan 16 00:16:22 2012
hg clone  http://hg.mozilla.org/camino/ mozilla/camino 
requesting all changes
abort: No such file or directory: /Volumes/tinder/Cm2.1-M1.9.2/Darwin_9.8.0_Depend/mozilla/camino/.hg/store/journal
Exception AttributeError: "'transaction' object has no attribute 'file'" in <bound method transaction.__del__ of <mercurial.transaction.transaction object at 0x1191630>> ignored
End:   Mon Jan 16 00:17:28 2012
Error: Hg checkout failed.

http://tinderbox.mozilla.org/showlog.cgi?log=Camino/1326700920.1326701849.14540.gz

All subsequent builds have failed with the EOF error cloning mozilla-1.9.2:

Begin: Mon Jan 16 11:39:31 2012
hg clone  http://hg.mozilla.org/releases/mozilla-1.9.2/ mozilla 
requesting all changes
adding changesets
adding manifests
adding file changes
transaction abort!
rollback completed
abort: premature EOF reading chunk (got 729 bytes, expected 24124)
End:   Mon Jan 16 11:46:38 2012
Error: Hg checkout failed.

http://tinderbox.mozilla.org/showlog.cgi?log=Camino/1326742740.1326743199.13977.gz

This host is an Xserve running Mac OS X Server 10.5.8 with Mercurial 1.5.2.

I suspect this is related to the Mercurial upgrade, as philor told me the upgrade triggered multiple types of hg clone failures across the ecosystem, but I have no idea why it didn't manifest for Camino until this morning.

This may also be related to bug 718186 or bug 718251, but I'm filing this separately because 1) the error messages from hg are different and 2) the timing of the appearance is different.

Comment 1

[Corey Shields [:cshields]]

Please try again.  the cache that sat in front of hgweb has been replaced as of an hour ago - and these problems should go away.

(In reply to Smokey Ardisson (not following bugs - do not email) from comment #0)
> This may also be related to bug 718186 or bug 718251, but I'm filing this
> separately because 1) the error messages from hg are different and 2) the
> timing of the appearance is different.

Never hurts to file a new bug.  This was a new symptom of the same problem that caused the other 2 bugs you pointed out.

Comment 2

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

(In reply to Corey Shields [:cshields] from comment #1)
> Please try again.  the cache that sat in front of hgweb has been replaced as
> of an hour ago - and these problems should go away.

Looks like the build went green right about that time.  Thanks!