1296686 - Find tasks missing from TH UI

Status: RESOLVED FIXED

(Testing Graveyard :: ActiveData, defect)

Description

[Kyle Lahnakoski [:ekyle]]

We can quickly identify tasks that do not show up in Treeherder by using ActiveData:

    http://activedata.allizom.org/tools/query.html#query_id=2+QsUe7K
    {
        "from": "task",
        "where": {"and": [
            {"missing": "treeherder.job.id"},
            {"gt": {"task.run.start_time": "{{today-2day}}"}},
            {"lt": {"task.run.start_time": "{{today-day}}"}}
        ]}
    }

Now, there are probably many things happening in task cluster that legitimately do not show in Treeherder, so this query should be modified to exclude those categories.  But we do find anomalies:

For example, why is this task not showing?

https://tools.taskcluster.net/task-inspector/#Zfg1D_XkQOKdwbvQ5gClvQ/

This only works if ActiveData can ingest Treeherder data.

Comment 1

[Kyle Lahnakoski [:ekyle]]

Treeherder tasks are not ingested fast enough to ensure the taskcluster tasks are marked with TH data.  On the other hand TH data is now available, and it has `job.taskcluster.id`.  ActiveData does not perfomr cross-references (or joins of any type), but we can write a script that can show the difference between these two sources.