1439746 - Create Blocker Flags List for Shield-Studies component

Status: RESOLVED FIXED

(bugzilla.mozilla.org :: Administration, task, P3)

Description

[Gregg Lind (Fx Strategy and Insights - Shield - Heartbeat )]

Hello!  We would like to add blocking flags for the "Shield Studies" component.  

Known flag list (incomplete).  Use whatever words / existing vocabulary exist for these things.

- legal
- data steward
- probe correctness
- functional qa
- peer review
- product integrity
- UX
- brand risk
- security
- privacy

What else do you need to make this happen?  Can see it on a dev server first, so we can practice with it a bit?

Comment 1

[Emma Humphries ☕️🎸🧞‍♀️✨ (she/they) [:emceeaich] (Pacific Time) use needinfo]

Gregg, 

I've created three of these flags on the dev-server for you to try out. These have been limited to Shield bugs.

https://mzl.la/2obJWyg is a list of shield bugs you can use these flags on.

For each flag you can specify:

* which group can request a flag
* which group can grant or deny a flag 
* a particular user to request review by
* a list of addresses to notify when the flag is requested

Let me know what groups you would need to construct or use for requests and approvals of flags.

Comment 2

[Gregg Lind (Fx Strategy and Insights - Shield - Heartbeat )]

Thanks for the work on this!  I will play with it a bit!

Comment 3

[Gregg Lind (Fx Strategy and Insights - Shield - Heartbeat )]

# Goal of project:

By looking at the FLAG LIST, have a sense for where a study is blocked / ready to ship status.

# Proposed Workflow:
- this is for project that are expected to take a "Shield or Pioneer Deployed Code" path.  If you project is Prior Research / Survey, you probably don't need all this.
- "Shield: Shield-Studies" bugs should mostly be started by Firefox PMs.

- All "active" flags are required as R+ before shipping
- Once Relman sets the shield-relman flag, ALLOWED to enable Normandy Enrollment Recipe
- Needinfo as appropriate for other specialist reviews

# Proposed Flags list for Shield Studies

Notes:
0.  Apply to "Shield:Shield Study" component ONLY
1.  All are 'editbugs-team', I think.
2.  This is the proposed order they should appear.  
3. Use your experience and judgment to fill in cc
4. As usual, "Needinfo" is a solution for tagging people / concerns that don't fit into these categories such as:

    -  L10n
    - Security, 
    - UX
    - web services
    - data pipeline

We expect that you will deal with UX, Security, Localization, Web Services as part of product dev.  **Needinfo** people as needed that don't fit into these.

** "editflagtypes" format**:  Name | Description | Cc list.

## Active (will show on all bugs)

ALL of these must be signed off before we can launch an Extension study on normandy

* shield-science |  "Science Review, including Escalation Matrix" | experiments-review@m.c
* shield-data-steward | "Data Steward" | (no email)
* shield-peer    | "Engineering / platform: performance, tryserver, security. Engineering is up to Firefox Release standards, does not break updater, or destroy performance."
* shield-qa      |  "Functional QA, live testing, look and feel, perception, Firefox-ness." | pi-request@m.c
* shield-relman  |  "If R+, ready to launch in proposed channel." | ()

Note: "shield-relman" is the 'last' in the list "32767".


## Requestable (can be asked for in Edit flags)
* shield-legal   | "Legal Review, for when you have possible type 3 or 4, contract or regulatory concerns." | 
* shield-vp      | "Escalation Risk matrix result.  Large populations, brand risk, large performance degradation." | 
* shield-bd    | "Flag if impacts partner contracts, existing or potential."
* shield-firefox-product | "Questions around brand alignment, product fit."
* shield-ux | "use existing Ux paradigms, OR have no user visible aspects at all, OR you gotta flag UX."
* shield-demonstration    | "Demonstrate the feature to ensure it looks right."


I hope this looks right, and suggestions are welcome!

Comment 4

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

Lgtm. I'd suggest renaming shield-bd to shield-partner. I read "bd" to mean business division.

Some studies go from Nightly to Beta to release. How do we remember and store the values of each of these per channel?

Comment 5

[Gregg Lind (Fx Strategy and Insights - Shield - Heartbeat )]

(BD is biz-dev.  You are right that "Partner" is a great name!

Channel tracking... In general, would suggest "child bugs" when we hop channels / do major new rollouts, but I welcome ideas and suggestions.  The 'how much modification is a new recipe' question is... contentious :)

Thanks!

Comment 6

[Emma Humphries ☕️🎸🧞‍♀️✨ (she/they) [:emceeaich] (Pacific Time) use needinfo]

We could make these flags release flags, which have multiple-values, so you'd have:

flag     | values                         
shield-* | nightly, beta, release, esr(?) 

See attachment.

You'd not be able to cc people on flag request/creation, however so you'd need to have standing searches/flags for these.

Or, depending on the workflow, you could have one release flag indicating which train this study is for, as long as you don't have a situation where the other flags could have different values depending on the release.

As long as all the flags in comment three have one value only (and not something like shield-legal+ in nightly and shield-legal? in beta.

Otherwise we'd have to have sets of flags for each release train and that will get ungainly.

Comment 7

[Gregg Lind (Fx Strategy and Insights - Shield - Heartbeat )]

:emceeaich  I claim we punt on it for now about channels.

Comment 8

[Jeff Griffiths (:canuckistani) (:⚡︎)]

I think you want sign-off from MattG, he has my Axe.

Comment 9

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

(In reply to Gregg Lind (Fx Strategy and Insights - Shield - Heartbeat ) from comment #5)
> (BD is biz-dev.  You are right that "Partner" is a great name!
> 
> Channel tracking... In general, would suggest "child bugs" when we hop
> channels / do major new rollouts, but I welcome ideas and suggestions.  The
> 'how much modification is a new recipe' question is... contentious :)
> 
> Thanks!

Do we track them as child bugs today? I am not sure if I've seen that happen.

Comment 10

[Matt Grimes [:Matt_G]]

Today we use the same bug. We send a new intent to ship email that outlines the previous iteration(s) of the study and the requested changes. Those changes are then captured in the original bug as well to keep everything in one place. If there's a more preferable method please let us know.

Comment 11

[Emma Humphries ☕️🎸🧞‍♀️✨ (she/they) [:emceeaich] (Pacific Time) use needinfo]

Attachment: Flags

I've set these up as flags for Shield::Shield Studies bugs.

Comment 12

[David Lawrence [:dkl]]

(In reply to Emma Humphries, Bugmaster ☕️ (she/her) [:emceeaich] (UTC-8) needinfo? me from comment #11)
> Created attachment 8964413 [details]
> Flags
> 
> I've set these up as flags for Shield::Shield Studies bugs.

Gregg, are these satisfactory and can we close this out?

dkl