1135762 - index: Investigate indexing of index.buildbot.revisions.d37655f24cb0.try.linux

Status: RESOLVED INCOMPLETE

(Taskcluster :: General, defect)

Description

[Jonas Finnemann Jensen (:jonasfj)]

From task:
https://tools.taskcluster.net/task-inspector/#7JjESdEERkGA9NzGy6xcuw/

Can seem to find this in the index... Something is wrong.
We need to look up logs.

Comment 1

[Michael Shal [:mshal]]

From https://bugzilla.mozilla.org/show_bug.cgi?id=1135748#c1 -

> @mshal, I notice that another task has now been indexed under d37655f24cb0.

I was retriggering the Linux build a few times to test some mozharness changes. One of these retriggers successfully created a task (zY4fQ_L2S8eTVgKnuZWyuA), which was indexed properly. The first set of builds all created tasks (Linux, OSX, Windows), but none were indexed. I'll try to see if it happens again...

Comment 2

[Jonas Finnemann Jensen (:jonasfj)]

I could find any references to "7JjESdEERkGA9NzGy6xcuw" apart from requests to get task status.
So presumably logging was down when we handled this task, or something bad happened.
I search on papertrail and downloaded the log archives, but there is no references other than from
events.taskcluster.net, queue.status and queue.getTask all of which are used by the inspector. 

We probably have a bug somewhere. Could be lack of retries in mozharness when calling reportCompleted.
But that would make this a freak accident and the likelihood that the queue managed to update the
database but died before sending the pulse message is very small. In this case the request would not return 200, so client would retry the request. And since retry was broken recently that could have been it.

But it's a slim chance, other than that it has to be a bug somewhere...

Anyways, we have no logs or anything to trace this. I searched logs back to the 20'th and didn't find anything about 7JjESdEERkGA9NzGy6xcuw other that queue.status calls from the inspector.
So I'm closing this as incomplete, we have no way of figuring out what happened.

Please, reopen or refile, if this happens again. A possible workaround is to use index.insertTask
directly from mozharness instead of adding index.<namespace> routes to the tasks.
See: http://docs.taskcluster.net/services/index/#insertTask

(Feel free to reopen, if anyone has ideas how to investigate this further)

Comment 3

[Pete Moore [:pmoore][:pete]]

Resetting Version and Target Milestone that accidentally got changed...