1533444 - Help modeling stability data such that it can be interpreted and understood by release management

Status: RESOLVED INACTIVE

(Data Science :: Investigation, task)

Description

[Marcia Knous [:marcia]]

Brief description of the request: The Release management team provides regular stability reports and analysis to the wider Mozilla population, including weekly reports at the Channel and Cross Functional Meetings. We have been using Mission Control, but have found that there are still a series of coverage areas that we need more granularity (see Google spreadsheet below). We are also in the midst of writing some stability related release criteria, and having this information will improve our ability to be able to set thresholds and manage stability issues both pre and post release.

Link to any assets: https://docs.google.com/spreadsheets/d/1TL0OLRvfFAhi_oHkTeAJu0Bz6IRog-yUJApuww07GVk/edit#gid=0

Is there a specific data scientist you would like or someone who has helped to triage this request: No

Comment 1

[William Lachance (:wlach)]

Please keep me updated on any work/analysis done, as I'd like to feed it into the project development plans for Mission Control in 2019:

https://docs.google.com/document/d/1Q3MpW5Z11zrslYPUTdu75bqt3aXJ0W850UK3rSjhbfM/edit#heading=h.a9h5gnp3jfi0

Comment 2

[Jess Mokrzecki [:jmok]]

Hi Marcia,

Could you please let us know the desired timeline for this request. Also does this impact any roadmaps or critical decisions?

Comment 3

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

We use MissionControl dashboard to manage Firefox Nightly, Beta and Release channel stability. I would be happy if we got someone to look into this in the next week or two. Thanks!

Comment 4

[Rob Miller [:rmiller]]

Hey Mark,

Not sure how to prioritize Mission Control work against the rest of wlach's workload, can you take a look at this one to let us know whether/how this might fit?

Comment 5

[Mark Reid [:mreid]]

As I understand it, this bug is about investigating improvements in how we measure stability in Firefox releases. Will is a stakeholder here in terms of building support for any improved measures in Mission Control, but the crux of the problem is in getting a fresh perspective on how we measure things today and figuring out ways we might make things better in the linked set of use cases.

As such I think it makes sense to work on this as a data science investigation first, with a next step of implementation in Mission Control.

Comment 6

[Marcia Knous [:marcia]]

Mark has summarized it correctly in Comment 5. We would like to meet with someone in data science soon to review the items in the document I have in Comment 0. Thanks.

Comment 7

[Emily Thompson]

Hi all,
There's quite a bit to unpack here, and unfortunately, we don't have the bandwidth to commit to an investigation at the moment (unless this gets bumped much higher up in priority), so it will have to go in the backlog. That being said, we're happy to review the various coverage areas and help scope out future projects. If that's still valuable to you, I'd suggest switching the component to "Review" from "Investigation" and we can probably get to it sooner.

Comment 8

[Mark Reid [:mreid]]

I just wanted to add an update here that I've had some conversations with management about the relative priority of this investigation, and will update again next week.

Comment 9

[Marcia Knous [:marcia]]

(In reply to Mark Reid [:mreid] from comment #8)

I just wanted to add an update here that I've had some conversations with management about the relative priority of this investigation, and will update again next week.

Hi Mark - Do you have an update for us? Thanks.

Comment 10

[Mark Reid [:mreid]]

Yes, :kparlante is going to be presenting this to :dcamp to help establish the prioritization.

Comment 11

[Mark Reid [:mreid]]

Quick update:

• Javaun has met with Ritu to help clarify the asks here, and they will continue to meet to address this set of work against the other priorities on Data Science from the Product organization.
• Due to various PTO, there will be no actions undertaken until the week of April 22nd at the earliest.

Comment 12

[Jess Mokrzecki [:jmok]]

Work for the DS team is now tracked in Jira. You can search with the Data Science Jira project for the corresponding ticket.