Verify VCS Perfherder data for Windows tasks is being ingested

RESOLVED FIXED

Status

defect
P1
normal
RESOLVED FIXED
a year ago
2 months ago

People

(Reporter: gps, Assigned: ekyle)

Tracking

Version 3
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

Reporter

Description

a year ago
I'm able to run ActiveData queries against the VCS Perfherder metrics being recorded by `hg robustcheckout`.

However, I'm selecting the raw data and am not seeing any data from Windows workers. There's a chance I'm doing it wrong. But some of the aggregate data seems to reflect no data for Windows.

https://taskcluster-artifacts.net/Yl5tnpaBQTGjduaj76r2PQ/0/public/logs/live_backing.log is an example of a Windows task whose data is not ingested. Search for "PERFHERDER_DATA". The fact it is at the beginning of a line may confuse the log parser?
Reporter

Comment 1

a year ago
FWIW, I have other patches in flight that attempt to make Firefox CI tasks use run-task. At that point, logs will be prefixed with the typical time prefix and this may cause ActiveData to start recognizing the PERFHERDER_DATA lines.

So if this is a non-trivial amount of work, ping me before you invest much time and I may be able to save you the work!
Assignee

Updated

11 months ago
Flags: needinfo?(klahnakoski)
Assignee

Comment 2

11 months ago
Sorry, I never even saw this bug. There is a dup somewhere, which I also did nothing on. needinfo myself so I remember to act on this.
Flags: needinfo?(klahnakoski)
Assignee

Comment 3

5 months ago
p1, just to verify before closing
Priority: -- → P1
Assignee

Updated

2 months ago
Flags: needinfo?(klahnakoski)
Assignee

Comment 5

2 months ago

Turns out it was two problems. The multiline json (with taskcluster timestamps)

[taskcluster 2019-03-28T17:35:29.768Z]

and the PERFHERDER record has no timestamp

Here is the file

https://taskcluster-artifacts.net/Qu-_x508RWys08fcgMtVgQ/0/public/logs/live_backing.log

Here is he task record

{
    "from":"task",
    "select":"_id",
    "where":{"eq":{"task.id":"Qu-_x508RWys08fcgMtVgQ"}}
}

Here are the (new) perf records

{
    "from":"perf",
    "select":["_id","run.key","result.framework.name"],
    "where":{"and":[{"eq":{"task.id":"Qu-_x508RWys08fcgMtVgQ"}}]},
    "limit":1000
}

Processing is proceeding atarting a few hours back.

Assignee

Updated

2 months ago
Flags: needinfo?(klahnakoski)
Assignee

Updated

2 months ago
Status: NEW → RESOLVED
Last Resolved: 2 months ago
Flags: needinfo?(klahnakoski)
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.