Add the capability to manage several nightlies per day in the Aggregates dataset

NEW
Unassigned

Status

Data Platform and Tools
Datasets: Telemetry Aggregates
P3
normal
a year ago
7 days ago

People

(Reporter: sylvestre, Unassigned)

Tracking

(Depends on: 1 bug)

Details

(Reporter)

Description

a year ago
We would like to move to two nightlies per day (mostly to accommodate Europeans)
As discussed here:
https://mail.mozilla.org/pipermail/fhr-dev/2017-March/001224.html
this will require some changes in the pipeline to get that done.
Blocks: 1255755
Summary: Please implement the capability to manage several nightlies per day → Please implement the capability to manage several nightlies per day in the Aggreagates dataset
Summary: Please implement the capability to manage several nightlies per day in the Aggreagates dataset → Add the capability to manage several nightlies per day in the Aggreagates dataset
Is this "We'd like to do this because it'd be nice to have" or "This is needed to support Dawn" or something else? Trying to get a sense of priority here relative to the growing amount of work our team needs to do to support Quantum.
Flags: needinfo?(sledru)
(Reporter)

Comment 2

a year ago
In the dawn project, we believe that we should improve the quality of nightly to be able to keep the same level of quality.
Because nightly are live around 2pm Europe time, we cannot test the changes in our morning. Having a nightly ready for Europe in the morning would help in verify the bugs and having a shorter feedback loop.
Flags: needinfo?(sledru)
Summary: Add the capability to manage several nightlies per day in the Aggreagates dataset → Add the capability to manage several nightlies per day in the Aggregates dataset
As mentioned in the fhr-dev thread, we're already dealing with respins in the current world, so this doesn't need to block moving to two nightlies a day. While aggregating by date isn't optimal, as far as I can tell, the main downside to this is lost granularity. Balancing that against the work currently underway to lower the latency of Telemetry data and the non-trivial amount of work this'll take to do properly, I'm putting it in the backlog for now. If we find ourselves in a situation where day-level resolution on Telemetry results isn't good enough, we can revisit the priority.
Priority: -- → P3
No longer blocks: 1255755
Component: Metrics: Pipeline → Datasets: Telemetry Aggregates
Product: Cloud Services → Data Platform and Tools
Now that we are really getting multiple nightlies a day, this seems to be relevant; but we are holding off on implementation until we have build-hub [0] integration, which should give push-log ranges for builds.

[0] https://github.com/mozilla-services/buildhub

Updated

7 months ago
Depends on: 1339616

Updated

14 days ago
Duplicate of this bug: 1460662
Duplicate of this bug: 1462391
Bug 1462391 comment 0 talks about a specific instance where Telemetry Evolution essentially gave the wrong regression date because of this build merging.
You need to log in before you can comment on or make changes to this bug.