1162604 - Index nothing but mozilla-inbound

Status: RESOLVED FIXED

(Testing Graveyard :: ActiveData, defect)

Description

[Kyle Lahnakoski [:ekyle]]

The weak ES cluster can not handle the number of records.  Update the indexing routines to only insert `mozilla-inbound` and 10% of the other branches.

The 10% is to remind query writers to filter by branch.

Comment 1

[Kyle Lahnakoski [:ekyle]]

Other branches set to 1%

https://github.com/klahnakoski/TestLog-ETL/commit/64106a1634735aec3779c4b67e258af87e2d5a04

Comment 2

[Andrew Halberstadt [:ahal]]

I guess it depends on what type of problems we're trying to solve, but if we had to choose a single branch to index, I'd prefer mozilla-central. Because:

1) Inbound is only one of 3+ integration branches. So it will miss changes incoming from fx-team or b2g-inbound until central is merged back in.

2) Inbound has no guarantee of stability, every push could potentially be backed out and cause anomalies in the data.

Alternatively, would it be possible to index both inbound and central? Central has <10 pushes per day, so shouldn't be *too* much extra load.

Comment 3

[Kyle Lahnakoski [:ekyle]]

I will include mozilla-central.  It will be days before it is backfilled.