1759235 - Try closed: pushlog of Try repository repeatedly stops updating, Treeherder doesn't show recent pushes

Status: RESOLVED WORKSFORME

(Developer Services :: Mercurial: hg.mozilla.org, defect, P1)

Description

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Observed 4+ times in the last 2 days, Zeid resolved the previous issues and the pushlog stopped updating again which causes Treeherder to not ingest the pushes and once the issue is fixed, the tasks have to be scheduled with the fxci replay-hg-push command.

Comment 2

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Try stopped updating again.

Comment 3

[Connor Sheehan [:sheehan]]

I gave try a replicatesync and it seems to have recovered.

Leaving NI for me as I'm going to spend some time debugging this issue in the next few days.

Comment 4

[Atila Butkovits]

Try stopped updating and autoland seems to be slow.

Comment 5

[Connor Sheehan [:sheehan]]

Try pushes are appearing again, investigating the cause now.

Comment 6

[Dan Mosedale (:dmosedale, :dmose)]

:sheehan does that mean it's ok to re-open the try tree so that "CLOSED TREE" checkin isn't required?

Comment 7

[Connor Sheehan [:sheehan]]

(In reply to Dan Mosedale (:dmosedale, :dmose) from comment #6)

:sheehan does that mean it's ok to re-open the try tree so that "CLOSED TREE" checkin isn't required?

Yes, we can re-open trees.

Comment 8

[Jan de Mooij [:jandem]]

This is happening again.

Comment 9

[Timothy Nikkel (:tnikkel)]

Filed bug 1767138 as a new instance of this. (I mistakenly thought this bug was closed so I filed new).

Comment 10

[Atila Butkovits]

We are observing this bug again.
Connor, can you take a look?

Comment 11

[Jens Stutte [:jstutte]]

Happens to me, too.

Comment 12

[Zeid [:zeid]]

Ran hg replicatesync again and the pushlog appears to be up to date now.