Closed Bug 1440637 Opened 6 years ago Closed 5 years ago

ADI for nightly are weird after 2018-01-23

Categories

(Cloud Services :: Operations: Kinto, task)

Product:
Cloud Services
Component:
Operations: Kinto
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: calixte, Unassigned)

References

Details

ADI are computed in using blocklist pings.
As you can see on [1], before the 01-23 we've weekly bumps and after no more (see [2] for the full graph).
The ADI used to build the graph are retrieved in using Socorro API (see [3]). 
:jason, do you have any idea to explain these changes ? 

[1] https://screenshots.firefox.com/t18eITZlEUAEpicN/www.arewestableyet.com
[2] https://www.arewestableyet.com/longtermgraph/index.html?fxnightly-blp
[3] https://crash-stats.mozilla.com/api/#ADI
Flags: needinfo?(jthomas)
See Also: → 1440314
Flags: needinfo?(gijskruitbosch+bugs)
This looks like a dupe of bug 1439526 to me. Do all these dashboards use ADI processing? Are the scripts accessible somewhere? I'm happy to try to help get a patch so that the 304s do not disturb log processing, but I'd need access to what's generating these things (and even 'these' things, in some cases...).
Flags: needinfo?(gijskruitbosch+bugs) → needinfo?(pascalc)
(In reply to :Gijs from comment #1)
> Do all these dashboards use ADI processing?

My brain is on post-lunch hiatus, apparently. I meant blocklist http access log processing.
I am a consumer of these dashboards, not the producer, I think I got access via Sheeri Cabral years ago, I don't know who built the scripts but maybe somebody in the #metrics channel on slack knows?
Flags: needinfo?(pascalc)
After few searches, it seems that data are retrieved from peach-gw.peach.metrics.scl3.mozilla.com.
The ADI processing pipeline only uses log records that contain HTTP Code 200. I mentioned in Bug 1399864 that this should be discussed with IT Data team since they manage the processing today. cc'ing Vyas
Flags: needinfo?(jthomas)
See Also: → 1399864
https://mana.mozilla.org/wiki/display/BIDW/ADI+Process+Flow has more details on the processing pipeline. The 'Aggregate processing Code', specifically the HiveQL query needs to be updated to include 304 status code.
I wrote up bug #1440745 to fix the script Socorro uses to fetch ADI data from hive. I'll start working on that now.
See Also: → 1439526

I believe this is fixed and thus closing it for now.

Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.