684088 - Return the QA Contact field to QA people and migrate current qa watchers to component watching

Status: RESOLVED FIXED

(bugzilla.mozilla.org :: Administration, task)

Description

[Marcia Knous [:marcia]]

As discussed during the 20110901 triage meeting, it was decided that there is a significant need for this field to be created. At this time there is no clear way for someone to own the verification part of a bug, and in the discussion during the triage meeting it was felt this would help the QA process quite a bit.

Note that this is not meant to replace the existing "QA Contact" field - we would like that field to be renamed to "Watch" or something that actually reflects what the field is currently used for.

The QA Assignee field would have the same properties as the "Assignee" field (edit, take)

The triage team believes this should be a P1 request.

I think there should probably be a post on dev-planning/QMO as well to explain why this change is needed.

Comment 1

[David Lawrence [:dkl]]

I am not against the idea of adding another custom field if it is critical to do so. But would it be possible to utilize one or more of the existing fields to provide the same function? For example maybe a flag called 'qa-verified' that can have a requestee (qa assignee) attached to it when the flag is set to '?'. Then the qa assignee can set the flag to '+' when verification is complete and the requestee value is cleared. That is just one way, there may possibly be others as well. Reason is we are trying to keep the amount of custom fields down to a manageable level if possible only adding them where absolutely necessary. If no other way is agreed on, I will add the new QA Assignee field to BMO.

Another caveat that I must mention is that Bugzilla does not currently have a custom field of type 'user'. There is only free form text fields or a drop down selection field. So if a free form field is used, there will not be any way to verify the user is valid that is entered without adding code to the backend. Also auto-complete will not work. If a drop down is used, it could be possible to just out the list of people who will be doing the verification in it but that could be difficult to maintain over time. Given more time it could be possible to do the work to add the support for this. Just isn't there right now.

glob, any other ideas?

dkl

Comment 2

[:glob ✱]

adding qa as a custom field isn't a good idea as it would require a lot of changes to the security code within bugzilla.

it would be significantly less invasive to bugzilla to add a new field to act as the 'watcher' field, and revert the qa field back to tracking a qa person.  as the watcher field doesn't need to be changeable by non-admins, and could be managed with a simple extension.

another option is to drop the idea of (ab)using users as watchers, and migrate everyone across to the component watching facility.  that will free up the qa field for its intended use.


my preferred option is to migrate to the component watcher; this will bring us parity with how the qa field is designed to be used in bugzilla, and reflects what users are actually doing -- they are watching the qa contact in order to watch a component.

Comment 3

[Asa Dotzler [:asa]]

(In reply to Byron Jones ‹:glob› from comment #2) 
> my preferred option is to migrate to the component watcher; this will bring
> us parity with how the qa field is designed to be used in bugzilla, and
> reflects what users are actually doing -- they are watching the qa contact
> in order to watch a component.

I agree that we should just take the QA Contact field back. That will be controversial because lots of people are using it for watching but it's the right thing do so we should do that and try to mitigate the pain of moving watching to another facility as best we can.

What is "the component watcher" and how can we transition current QA watchers to Component watchers?

Comment 4

[Asa Dotzler [:asa]]

do we need a separate bug for adding "(take)" link to the QA Contact field?

Comment 5

[:glob ✱]

(In reply to Asa Dotzler [:asa] from comment #3)
> What is "the component watcher"

it's an extension to bugzilla; https://bugzilla.mozilla.org/userprefs.cgi?tab=component_watch

> and how can we transition current QA watchers to Component watchers?

1. communicate that we're going to do it
2. insert rows into the component-watching table for everyone who is watching a @bmo.bugs address
3. drop the user watching rows
4. communicate that we've done it

Comment 6

[Asa Dotzler [:asa]]

(In reply to Byron Jones ‹:glob› from comment #5)
> 1. communicate that we're going to do it
> 2. insert rows into the component-watching table for everyone who is
> watching a @bmo.bugs address
> 3. drop the user watching rows
> 4. communicate that we've done it

Beautiful. 

Marcia, if I drafted a post for .planning, would you edit it and post it? I'm a lightning rod right now and don't want to bring any asa-baggage to this otherwise easy discussion :-)

Comment 7

[David Lawrence [:dkl]]

(In reply to Asa Dotzler [:asa] from comment #4)
> do we need a separate bug for adding "(take)" link to the QA Contact field?

yes please.  i did the take for assignee originally so i will do qa as well.

dkl

Comment 8

[David Lawrence [:dkl]]

(In reply to Byron Jones ‹:glob› from comment #5)
> 1. communicate that we're going to do it
> 2. insert rows into the component-watching table for everyone who is
> watching a @bmo.bugs address
> 3. drop the user watching rows
> 4. communicate that we've done it

Just food for thought. What if we just moved all of the *@*.bugs addresses from the default qa to the initial CC list instead? This would also reopen the qa field. We would of course have to move the accounts on all of the affected bugs.

Long term  I definitely like the component watching solution is better. We would also need to drop the watcher component address from the qa field in current bugs. You will need to add that in your announcement as well I think. That would cause spam unless we run it as part of the other script and update lastdiffed without sending email.

Dkl

Comment 9

[Asa Dotzler [:asa]]

(In reply to David Lawrence [:dkl] from comment #7)
> (In reply to Asa Dotzler [:asa] from comment #4)
> > do we need a separate bug for adding "(take)" link to the QA Contact field?
> 
> yes please.  i did the take for assignee originally so i will do qa as well.
> 
> dkl

bug 684171 :D

Comment 10

[:glob ✱]

(In reply to David Lawrence [:dkl] from comment #8)
> Just food for thought. What if we just moved all of the *@*.bugs addresses
> from the default qa to the initial CC list instead?

while that would also work, i think using the component watching facility is a cleaner solution.

> We would also need to drop the watcher component address from the qa field in current bugs

ah, true.  and update all components to reset the current default qa -- for the initial migration we should reset them all to nobody.

> That would cause spam unless we run it as part of the other script and update
> lastdiffed without sending email

for the scale of change we're talking about, we definitely need to avoid sending bugmail :)

Comment 11

[Matt Evans [:mevans]]

I'm concerned with the magnitude of this change. If it is not properly socialized I fear there will be quite some backlash. 

What is proposed here; taking back the qa assigned field for it's designed purpose here is the best solution. 

However it isn't clear to me that the solution will preserve the current state of watched components through the qa assigned field. Will users will see no affect once this is done, the other than changed fields? 

We will also have to communicate to users how they will implement watching components once this change is done. 

Asa do you really think a post in planning will be sufficient?

Comment 12

[:glob ✱]

(In reply to Matt Evans [:mevans] from comment #11)
> However it isn't clear to me that the solution will preserve the current
> state of watched components through the qa assigned field. Will users will
> see no affect once this is done, the other than changed fields? 

they will continue to receive emails for the components they are watching.  some headers in the email will change to reflect the change in trigger (from "watching a user" to "watching a component").

obviously how people set up new components to watch will change; we could add a note on the user watching page pointing them in the right direction.


one thing that is currently possible, but is not possible with component watching, is you can currently CC a component's qa user to bugs.  this can be useful to make people who are watching a component aware of a related issue on a bug in a different component.  i've played around with a few ideas to address this, however i haven't been able to come up with anything that didn't result in unsatisfactory complexity.

there are only 5445 bugs where this has happened, largely on the server-operations bugs.  the reason is this is one of the few components on bmo using the qa field as intended; so there may be a decent use case here.


some users may need to reconfigured there email preferences (https://bugzilla.mozilla.org/userprefs.cgi?tab=email).  for users who are not watching any components currently, we can copy their current qa prefs to the component prefs.  for users who are both watching qa-users and components, they will no longer have separate preferences, and we shouldn't copy their perfs across.  this is an edge case and i think we shouldn't worry about it. 


there are other benefits aside from the qa field reclaiming:

currently it's possible for the qa on a bug to be changed away from the user which people are watching, which will result in emails not being sent.  this issue will be resolved.

with the current system it's painful to watch all components in a product, and there's no way to automatically watch new components; you have to somehow discover that a component has been added and manually watch the new user.  the component watching ext allows you to watch all components in a product easily, and this will automatically include any new components.
 
> We will also have to communicate to users how they will implement watching
> components once this change is done. 
> 
> Asa do you really think a post in planning will be sufficient?

i plan on blogging about this, which will get us planet.m.o coverage.

Comment 13

[Marcia Knous [:marcia]]

Yes, if you draft something I will post it. I am sure if will generate some angst.

We really need to be careful about sending any needless bugmail as that will cause end users lots of aggravation.

(In reply to Asa Dotzler [:asa] from comment #6)
> (In reply to Byron Jones ‹:glob› from comment #5)
> > 1. communicate that we're going to do it
> > 2. insert rows into the component-watching table for everyone who is
> > watching a @bmo.bugs address
> > 3. drop the user watching rows
> > 4. communicate that we've done it
> 
> Beautiful. 
> 
> Marcia, if I drafted a post for .planning, would you edit it and post it?
> I'm a lightning rod right now and don't want to bring any asa-baggage to
> this otherwise easy discussion :-)

Comment 14

[:glob ✱]

i was thinking about this over the weekend, with particular attention to the "cc people watching this component" issue i raised in comment 12..


it would be little work to extend the component-watching extension to add a new field to components: "watch user".  the username that we current put into the QA field will instead be put into this field.

when a change triggers a email to component watchers, we can use the bugmail_recipients hook to inject the watch-user account into the recipient list, and then anyone watching that user will also receive emails.  likewise if a watch-user is already on the recipient list (eg. via a CC), we inject all users watching the associated component(s).

the only upstream change required for this would be a hook to the "browse" report so the watch-users are discoverable.


the migration steps would be:

1. update the component-watching extension as described
2. update all components with a .bugs QA
   a. copy QA to watch-user
   b. set QA to nobody
3. update all bugs (!) with a .bugs QA
   a. set QA to nobody
   b. update lastdiffed to skip triggering bugmail
4. migrate all user's settings from watching .bugs users to watching components
5. add a note to the user watching prefs pointing to component watching prefs


step 3 is the scary one; i'm not sure if we should be updating all bugs, or just open bugs, or maybe open bugs + recently closed bugs.  thoughts?

Comment 15

[Gervase Markham [:gerv]]

Seems like a good transition strategy. I'd suggest "in for a penny, in for a pound" - update all bugs, unless there's some technical reason that this is hard. Given that you are not sending bugmail, it doesn't matter how many bugs you change. (We should record the change in history, though.)
 
Gerv

Comment 16

[:glob ✱]

(In reply to Gervase Markham [:gerv] from comment #15)
> Seems like a good transition strategy. 

raised bug 684701 to track the enhancement to component-watching

Comment 17

[Matt Evans [:mevans]]

All, what is the timeframe/ETA for getting this complete? Is there anything QA can do to help this move along? Thanks Matt.

Comment 18

[:glob ✱]

(In reply to Matt Evans [:mevans] from comment #17)
> All, what is the timeframe/ETA for getting this complete? 

hopefully within a couple of weeks; there's work happening on bug 684701 which is pretty well complete.  next step is to write the migration code and other minor patches to satisfy this bug.

> Is there anything QA can do to help this move along?

the biggest non-code chunk of work here is formulating the announcements. asa has indicated that he's snowed under, so if you want to get the ball rolling on that it would be a great help.


it is possible to make the migration slightly easier with regards to both code and the end-user impact; we could skip migration from user watching to component watching (step 4 from comment 14).

Comment 19

[:glob ✱]

i've started off the announcement email on http://etherpad.mozilla.org:9000/Bxrz8UMlZg


note to self: the migration script will also need to create component-watching email prefs for all accounts which are watching users but have never set up component watching, by copying the current qa-contact prefs.

Comment 20

[Marcia Knous [:marcia]]

glob: Just curious when this will be ready for prime time since we are now approaching the end part of October. I think this could helpful in terms of QA Q4 goals. Thanks.

Comment 21

[:glob ✱]

sorry marcia, i wrote an update comment yesterday, but forgot to hit save :(

an update: the migration script is almost complete, i'm currently performing a load of testing against it, and sorting out the edge cases.  we'll be updating just over 400,000 bugs, which takes 11 minutes on my development system.

gerv provided some excellent feedback on the announcement email (now at https://bmo.etherpad.mozilla.org/1), thanks!

i'm expecting the code to be completed early next week, with reviews, testing and announcements expected to take a few days on top of that.

Comment 22

[Gervase Markham [:gerv]]

I still have one fairly big question about the announcement email, which I've now added to the etherpad.

Speaking of which, the SSL config for the etherpad server is broken - it uses a *.mozilla.org cert which doesn't work for e.g. bmo.etherpad.mozilla.org.

Gerv

Comment 23

[:glob ✱]

(In reply to Gervase Markham [:gerv] from comment #22)
> I still have one fairly big question about the announcement email, which
> I've now added to the etherpad.

i've answered via the pad :)

> Speaking of which, the SSL config for the etherpad server is broken - it
> uses a *.mozilla.org cert which doesn't work for e.g.
> bmo.etherpad.mozilla.org.

i believe that's that's bug 693006

Comment 24

[:glob ✱]

Attachment: migration script v1

here's the migration script, which performs the following tasks:
  - for all bugs where the qa contact ends in .bugs:
    - the qa contact is changed to '[e-mail]'
    - an entry is inserted into the bug's history
    - an email is *not* triggered
  - for all users watching users that end in .bugs:
    - the users are mapped to components
    - if the user is watching all components in a product, they will watch
      the product itself (instead of individual components)
    - if the user is already using component watching, settings are merged
    - the .bugs user is removed from the watch list
  - for all components where the initial-qa-contact ends in .bugs:
    - the initial-qa-contact is changed to '[e-mail]'

Comment 25

[David Lawrence [:dkl]]

(In reply to Byron Jones ‹:glob› from comment #24)
> Created attachment 569297 [details]
> migration script v1
> 
> here's the migration script, which performs the following tasks:
>   - for all bugs where the qa contact ends in .bugs:
>     - the qa contact is changed to '[e-mail]'

Quick question as I start to review. Why (sorry if this was answered already somewhere else) are we setting the qa contact to [e-mail] as opposed to just clearing it completely? Unlike assignee, the qa contact can be empty.

dkl

Comment 26

[David Lawrence [:dkl]]

Comment on attachment 569297 [details]
migration script v1


>        # insert into bug activity
>        $dbh->do("
>            INSERT INTO bugs_activity(bug_id, who, bug_when, fieldid, removed, added)
>                 VALUES (?, ?, ?, ?, ?, ?)",
>            undef,
>            $bug_id, $nobody_user_id, $now, $qa_field_id, $old_qa, $nobody_user_id

removed and added in bugs_activity is a text field so you need to use the login names of the 
old qa and nobody users.

could do: $bug_id, $nobody_user_id, $now, $qa_field_id, _login($old_qa), _login($nobody_user_id)

Everything from here down makes my head hurt and to go grab a few bottles of wine :)

>    while (my($user_id, $product_id, $component_id) = $sth->fetchrow_array) {
>        $current_watch{$user_id}{$product_id} = [] unless $current_watch{$user_id}{$product_id};
>        push @{$current_watch{$user_id}{$product_id}}, $component_id if $component_id;
>    }
>
>    # remove existing entries from component-watch
>    foreach my $user_id (keys %current_watch) {
>        next unless exists $component_watch{$user_id};
>        foreach my $product_id (keys %{$current_watch{$user_id}}) {
>            next unless exists $component_watch{$user_id}{$product_id};
>            if (scalar @{$current_watch{$user_id}{$product_id}}) {
>                # if the user now wants to watch the whole product, nothing to do
>                if (!scalar @{$component_watch{$user_id}{$product_id}}) {
>                    next;
>                }
>                # remove duplicates
>                my @components;
>                foreach my $component_id (@{$component_watch{$user_id}{$product_id}}) {
>                    next unless exists $current_watch{$user_id}{$product_id};
>                    if (grep { $_ == $component_id } @{$current_watch{$user_id}{$product_id}}) {
>                    } else {
>                        push @components, $component_id;
>                    }
>                }
>                if (scalar @components) {
>                    # save the filtered list
>                    $component_watch{$user_id}{$product_id} = \@components;
>                } else {
>                    # nothing new to watch
>                    delete $component_watch{$user_id}{$product_id};
>                }
>            } else {
>                # currently watching the whole product
>                delete $component_watch{$user_id}{$product_id};
>            }
>        }
>    }
[snip]

Running now on my test instance to see how it works out...

dkl

Comment 27

[David Lawrence [:dkl]]

One thing I noticed on my test migration (only took about 10-15 mins) was for my personal preferences, something looks weird.

On BMO my current watch list is:
[e-mail]
[e-mail]
[e-mail]
[e-mail]
[e-mail]
[e-mail]
[e-mail]
[e-mail]
[e-mail]
[e-mail]
[e-mail]
[e-mail]
[e-mail]
[e-mail]
[e-mail]
[e-mail]
[e-mail]
[e-mail]
[e-mail]
[e-mail]

After migration, the only thing that looks to have been migrated properly was:

bugzilla.mozilla.org/General

I do not see any of the others in my component watching list and they are still in the watcher list under email preferences. Somehow they were skipped over. I will look through the code and see what may have caused this. Also I will do a fresh import of the test database again so I can run it once more if I make any changes.

dkl

Comment 28

[:glob ✱]

(In reply to David Lawrence [:dkl] from comment #25)
> Quick question as I start to review. Why (sorry if this was answered already
> somewhere else) are we setting the qa contact to [e-mail] as
> opposed to just clearing it completely? Unlike assignee, the qa contact can
> be empty.

oh, because i forgot that you could clear it.  setting it to empty is better :)

> Everything from here down makes my head hurt and to go grab a few bottles of wine :)

agreed; it hurt my head writing it :)

> After migration, the only thing that looks to have been migrated properly was:
> bugzilla.mozilla.org/General

that's because the bugzilla product is configured differently... it uses the @bugzilla.bugs as the default assignee (instead of nobody), and [e-mail] as the qa for all components.  we're only migrating qa watchers, and you're not watching the bugzilla qa user.

Comment 29

[Dave Miller [:justdave]]

Bugzilla is overriding the assignee in that manner, too.  Essentially, if you're watching one of the assignee users, you're watching that component, and if you're watching the QA users, you're watching the entire product.  Do we have "product watching" as well?  Might be worth migrating those ownerships out in Bugzilla and reassigning those bugs to nobody as well.

Comment 30

[:glob ✱]

(In reply to Dave Miller [:justdave] from comment #29)
> Do we have "product watching" as well?

yes, select "__Any__" as the component to watch.

if the migration script detects that you're currently watching all components in a product, you will be set up to watch the whole product instead.

> Might be worth migrating those ownerships out in Bugzilla and reassigning those bugs
> to nobody as well.

sure, but i'd prefer to tackle that as a separate project, rather than complicate this one any further.

Comment 31

[:glob ✱]

Attachment: script patch v1

patch against the migration script which clears the qa-contact instead of setting it to nobody.

Comment 32

[Marcia Knous [:marcia]]

glob and dkl: Just checking in to see how this work is going and whether we have an ETA. Also does this have to happen before Bug 684171?

Can this be tested in landfill?

Comment 33

[:glob ✱]

(In reply to Marcia Knous [:marcia] from comment #32)
> glob and dkl: Just checking in to see how this work is going and whether we
> have an ETA.

waiting on the review from dkl.

> Also does this have to happen before Bug 684171?

no; the bugs are related but they are independent.

> Can this be tested in landfill?

we have a staging instance of bmo which this will be tested on, once dlk has completed his review.  (landfill is for upstream bugzilla development, and it isn't large enough to hold bmo).

Comment 34

[David Lawrence [:dkl]]

Comment on attachment 569297 [details]
migration script v1

r=dkl as it seems to work for my testing. This is with the _1, _2, and _3 patches 
applied that fix the issues with clearing qa contact values. 

One thing I noticed is that you are not migrating email preferences for those who are not currently utilizing component watching as described in comment 12. Will that be done as a different script or was it decided it was not important?

dkl

Comment 35

[:glob ✱]

(In reply to David Lawrence [:dkl] from comment #34)
> Comment on attachment 569297 [details]
> migration script v1
> 
> r=dkl as it seems to work for my testing.

thanks for the review :)

> One thing I noticed is that you are not migrating email preferences for
> those who are not currently utilizing component watching as described in
> comment 12. Will that be done as a different script or was it decided it was
> not important?

nuts, i forgot about that; thanks for picking that up.  i'll write an updated script, but it should be easy to test as it's pretty well a stand-alone migration step.

Comment 36

[:glob ✱]

Attachment: migration script v3

updated script which copies the qa email settings to component watching.

unfortunately it wasn't possible to make it easy to test without running the full script.

Comment 37

[David Lawrence [:dkl]]

Comment on attachment 582175 [details] [diff] [review]
migration script v3

Looks good. Refreshed my bmo test db, set up my tests, and reran the migration script. Watched qa contacts were properly moved to component watchers and custom qa contact email preferences were migrated over to component watcher email preferences. r=dkl

Comment 38

[timeless]

So. I don't see anyone here asking why the QA field was changed to not be a live person.

There seems to be an assumption that the reason the QA field was dummy accounts was to enable watching the components. This is in a way true. It isn't really the reason.

The reason has to do with people and churn. There are fewer than 10 people involved in bmo today who have been involved since day 1. Of those, perhaps 1 may be doing precisely the same things that they were doing on day 1 and all days since. I think the actual count is precisely 0 fwiw.

Why does this matter?

If someone is given a role ([e-mail] or [e-mail] or [e-mail]) and they for whatever reason cease to have that role, then someone needs to perform bookkeeping to update all instances where that person used to have that role to the new person for that role. In bugzilla, a role is mapped to a field (for this bug, that's the QA field). Updating a role involves updating some set of bugs where that person is set to the QA field value. Lastly, an update of a bug involves potentially sending N people mail indicating the change to the bug.

If a QA contact exists in a role for a non trivial length of time, and a component is trafficked, the number of bugs attached to that QA contact will be sizeable. Thus if you update this field, you trigger a sizeable amount of bugmail. Some people will want this mail, but many will think you're just being annoying. And note that while the field may change for many bugs as described above, it can change for individual bugs (or sets) for whhich people can reasonably want to see such changes.

Lastly, what does one do to bugs which are verified by one QA contact who then retires and aee later reopened? Historically, the QA contact wasn't reset by reopening a bug, so such bugs ended up helpless.

These were the primary problems that led to using watchable role accounts. There were convenient side effects of watchable qas: cc'ing potentially interested parties to bugs in other components, letting people other than a QA watch a component w/o watching the QA's other bugs, lazy queries by mail strings instead of components, fitting other logic tables for bugmail control (watching is treated the same as watched so "when I am added/removed to the role" makes sense).

When the person assigned to QA a component changes, no bugmail is required. Maintaining a table of responsibility for components can be done in a wiki or in a shared query in combination with whines. 

Those who forget the past are bound to repeat it. If you have questions about the past, or if you think you know why something was done but weren't around when it was done, please consider asking for info about people who were around and then asking them for an explanation, you may be surprised.

Comment 39

[:glob ✱]

(In reply to timeless from comment #38)
> If someone is given a role ([e-mail] or [e-mail] or
> [e-mail]) and they for whatever reason cease to have that role,
> then someone needs to perform bookkeeping to update all instances where that
> person used to have that role to the new person for that role.

we need to have a real person assigned to the qa to track who will be actually responsible for the qa tasks.  yes, if someone leaves, then we'll have to touch the qa field on all bugs that person was responsible for, however the same issue also applies to the assignee field.

we can minimise the impact by preferring to have no default qa set on components, instead have qa actively set the qa-contact field to themselves as required.

> Lastly, what does one do to bugs which are verified by one QA contact who
> then retires and aee later reopened? Historically, the QA contact wasn't
> reset by reopening a bug, so such bugs ended up helpless.

i see no reason why the qa-contact would need to be reset when you reopen a bug.
the assignee isn't reset when a bug is reopened.
 
> Those who forget the past are bound to repeat it. If you have questions
> about the past, or if you think you know why something was done but weren't
> around when it was done, please consider asking for info about people who
> were around and then asking them for an explanation, you may be surprised.

gerv has been involved in this decision.

Comment 40

[Gervase Markham [:gerv]]

timeless: surely your argument about people changing roles applies just as well to default assignees as default QA contacts? Yet we don't say that bugs should be assigned to generic addresses.a

I think that there is large social value in having a group of bugs "assigned" to a particular person for QA responsibility. And Marcia, as a representative of the QA group, wants it to be possible. So we are making it possible.

Gerv

Comment 41

[timeless]

Indeed it does! Which is why most components have nobody@ as default assignees. We also ask people not to take ownership until they're actively working on a bug.

There are a few exceptions, but they tend to be off the beaten path. They also are people who didn't experience the pain which led to this process.

On social value, we have a module owners list which was moved to a wiki to make it easier for people to keep it current. Having QAs listed in the same place would make sense.

Comment 42

[timeless]

Sorry, I missed a point:

I'm sure that the original design of the QA contact field was selected at the behest of whomever at Netscape ran the QA team or certainly in collaboration with that person. The people who suffer the consequences of organizational change are people *outside* the QA team. If a person leaves, they can rename their account or turn off mail. If a person is new or new to a role, they can create a new account. It's everyone else who gets the mail for changes. Getting hundreds of messages because of a change in a Corporation doesn't make sense to a community.

That was Netscape 10+ years ago. The same issues apply today with Mozilla Corporation. Mozilla has some churn just as Netscape did. In fact, it happens rarely for both companies which means that the change when it happens is larger as more bugs accumulate for a certain person.

Comment 43

[Gervase Markham [:gerv]]

The plan AIUI is to have the QA contact field empty by default - that is to say, all products and components which currently have generic default QA contacts will now have no default QA contact (comment 39). Is that correct?

If so, I'm not sure what timeless is arguing for, unless he is arguing that people should never have bugs assigned to themselves for either fixing or QA, in case they leave the project.

Gerv

Comment 44

[:glob ✱]

(In reply to Gervase Markham [:gerv] from comment #43)
> The plan AIUI is to have the QA contact field empty by default - that is to
> say, all products and components which currently have generic default QA
> contacts will now have no default QA contact (comment 39). Is that correct?

yes; i would expect QA to 'take' the qa-contact field when they are responsible for QA for a bug; very much along the same lines as the assignee.

Comment 45

[Gervase Markham [:gerv]]

Having said that, if the QA team want to start allocating default-QA roles to people (particularly non-employees) as a way of saying "you own this patch", I would be heartily in favour. They will all start out blank, but there's no reason for them to remain that way.

Gerv

Comment 46

[David Lawrence [:dkl]]

Also fwiw, people could start using the qa contact field now if they desired as globs migration script will not touch the qa contact field if it doesn't have one of the old watcher addresses.

dkl

Comment 47

[:Gavin Sharp [email: [e-mail]]]

(In reply to David Lawrence [:dkl] from comment #46)
> Also fwiw, people could start using the qa contact field now if they desired
> as globs migration script will not touch the qa contact field if it doesn't
> have one of the old watcher addresses.

That will break watch-mail for people watching those addresses, until the migration script runs.

This plan scares me a lot, because it's not clear that we've ensured that component watching is a suitable alternative to the current system (that many people rely on), and it's very difficult to revert if we somehow find issues later on.

Comment 48

[Marcia Knous [:marcia]]

This bug came up at our last community meeting. As QA starts working more fervently to build new community we really would like a way for people to own bugs from a QA perspective. Just like there are "good first bugs" for developers, there are good first bugs for QA and we want the community to feel empowered to take a bug from the QA perspective to try to find STR, regression ranges, verification, etc.

Is there anything we can do to mitigate the risk that Gavin calls out in Comment 47?

Comment 49

[u279076]

Can we rename "QA Contact" to something like "Watchlist" and add a new field called "QA Contact"? 

As a newcomer to a bug, I would assume "QA Contact" to mean, "this is who I contact to ask about testing for this bug".

Comment 50

[Gervase Markham [:gerv]]

glob: what needs to happen for us to execute on this plan?

gavin: can you be more specific about how you think component watching may not be a sufficient substitute for the current system, and what we might do to address that?

ashughes: that suggestion sounds even more confusing to me! :-)

Gerv

Comment 51

[:glob ✱]

sorry this bug has been dragging on, as gavin points out it's a scary change, and i've been sidetracked by several other projects :(

(In reply to Anthony Hughes, Mozilla QA (irc: ashughes) from comment #49)
> Can we rename "QA Contact" to something like "Watchlist" and add a new field
> called "QA Contact"? 

the qa-contact field has special meaning within bugzilla, so renaming it and adding a new field which replicates its function would be a reasonable amount of work, and imho wouldn't move us forward much.  if we're (ab)using the qa-contact field to watch components, it makes more sense to watch components directly.

(In reply to Marcia Knous [:marcia] from comment #48)
> Is there anything we can do to mitigate the risk that Gavin calls out in
> Comment 47?

gavin: can you give more details about where you feel the gaps are?

note the qa-contact watcher email addresses will not be going away -- they will continue to be configured when we add a component, and you'll still be able to watch those users to get all emails relating to that component.  the difference is the linkage between the bug and the email address is being shifted from "is this user the qa-contact" to the actual component itself.


what are we waiting on:
  - the code is ready to go
  - signoff on https://bmo.etherpad.mozilla.org/1
  - setting a date
  - community announcements (planet, everyone@mozilla, dev.planning, ...?)



perhaps what we should do now is post to dev.planning to gather feedback, and hopefully address concerns such as gavin's (either with communication or code).  once things settle down there, pick an update date and make rocket go.

Comment 52

[Gervase Markham [:gerv]]

Sounds like a plan.

Gerv

Comment 53

[:Gavin Sharp [email: [e-mail]]]

(In reply to Byron Jones ‹:glob› from comment #51)
> gavin: can you give more details about where you feel the gaps are?

No, I don't have any details, because I personally haven't tried switching to using component watching. I'm just suggesting that it's quite possible that there are issues, given previous history of this feature (granted, many of the initial issues have since been fixed). It seems like it would be a good idea to encourage more people to switch voluntarily ahead of time to shake out any potential issues, before forcing it on everyone in a difficult-to-reverse way.

> note the qa-contact watcher email addresses will not be going away -- they
> will continue to be configured when we add a component, and you'll still be
> able to watch those users to get all emails relating to that component.  the
> difference is the linkage between the bug and the email address is being
> shifted from "is this user the qa-contact" to the actual component itself.

I don't really understand this. Whether the concept of a "QA contact" continues to exist or not isn't really relevant - they're not useful for watching if they don't stay consistent and people regularly modify them, so they're effectively "going away" for the purpose of watching.

Comment 54

[:glob ✱]

(In reply to Gavin Sharp (use [e-mail] for email) from comment #53)

> > gavin: can you give more details about where you feel the gaps are?
> 
> No, I don't have any details, because I personally haven't tried switching
> to using component watching. [..] It seems like it would be a
> good idea to encourage more people to switch voluntarily ahead of time to
> shake out any potential issues

i encourage you to switch to using component watching :)

i'll make that part of my dev.planning post, and will post an associated item to my blog for planet coverage.

> > note the qa-contact watcher email addresses will not be going away -- they
>
> I don't really understand this.

sorry i wasn't clear enough :(

i'm not suggesting people watch the qa-contact post-migration.

the current email addresses that we're entering as qa-contacts (eg. [e-mail]) will continue to exist after this migration.  they will continue to be bound to a component, and watching one of these @*.bugs accounts will continue to function as per the current system.

Comment 55

[:Gavin Sharp [email: [e-mail]]]

(In reply to Byron Jones ‹:glob› from comment #54)
> the current email addresses that we're entering as qa-contacts (eg.
> [e-mail]) will continue to exist after this migration.  they
> will continue to be bound to a component, and watching one of these @*.bugs
> accounts will continue to function as per the current system.

So despite the addresses no longer being used reliably as the QA contact, they will continue to get mail for all of the bugs in the associated component? Is this some kind of bmo-specific hack? Is there any way to see the component/address association? And if this is true, why do we need to migrate people to component watching?

Comment 56

[:glob ✱]

(In reply to Gavin Sharp (use [e-mail] for email) from comment #55)
> So despite the addresses no longer being used reliably as the QA contact,
> they will continue to get mail for all of the bugs in the associated
> component?

yes; instead of using the qa-contact on a bug as the linkage, a field is added to components where you specify the 'watch user' (attachment 598151 [details] which shows the new "watch user" field).

> Is this some kind of bmo-specific hack?

it's part of the component watching extension.
you can't watch components in upstream bugzilla (see bug 76794).

> Is there any way to see the component/address association?

it'll be visible on the component watch preferences tab (attachment 598150 [details]).

> And if this is true, why do we need to migrate people to component watching?

good question; it's one i've wrestled with for a while.  the main reason for the migration is to reflect the user's intentions within the ui - they are watching a user in order to watch a component.  as we have the ability to watch components directly, i think it makes sense to migrate these preferences across.

if this is your primary point off contention, i'm not against taking the more reserved approach of not migrating prefs from user to component watching.

Comment 57

[:Gavin Sharp [email: [e-mail]]]

(In reply to Byron Jones ‹:glob› from comment #56)
> good question; it's one i've wrestled with for a while.  the main reason for
> the migration is to reflect the user's intentions within the ui - they are
> watching a user in order to watch a component.  as we have the ability to
> watch components directly, i think it makes sense to migrate these
> preferences across.

OK, this makes sense.

> if this is your primary point off contention, i'm not against taking the
> more reserved approach of not migrating prefs from user to component
> watching.

This also makes sense :) Or at least, I think it makes sense to decouple the two steps - migrating users can happen at some later point, without impacting QA's ability to reclaim the QA contact field.

Thanks for the explanation!

Comment 58

[:glob ✱]

ok, we'll defer doing the preferences migration until later.

http://globau.wordpress.com/2012/02/21/76/

Comment 59

[:glob ✱]

summary of feedback so far:

- some confusion around if people will have to change anything - hopefully clarified, but
  we'll have to revise the announcement when we make the switch
- we'll have to migration users's email selection preferences, but not if they are watching
  users or components (the current code already does this)
- searching for bugs with an empty QA contact is non-obvious and slow
- some products may want to be excluded from this (eg. bugzilla)

Comment 60

[:glob ✱]

Attachment: migration script v4

this revision:
  - excludes bugzilla
  - no longer migrates users from watching users to component watching
  - copies qa email settings to comp-watch settings (only when comp-watch
    settings do not already exist)

Comment 61

[David Lawrence [:dkl]]

Comment on attachment 604067 [details]
migration script v4

Still reviewing other parts as I am reimporting my BMO snapshot.

Nothing major so far.

># debugging methods

>sub _prod {
>    my $id = shift;
>    return $cache{prod}{$id} if exists $cache{prod}{$id};
>    return $dbh->selectrow_array("SELECT name FROM products WHERE id=?", undef, $id);
>    return $cache{prod}{$id};
>}

Not storing the value from the SQL in cache before returning.

>sub _comp {
>    my $id = shift;
>    return $cache{comp}{$id} if exists $cache{comp}{$id};
>    return $dbh->selectrow_array("SELECT name FROM components WHERE id=?", undef, $id);
>    return $cache{comp}{$id};
>}

Same here.

Comment 62

[:glob ✱]

(In reply to Byron Jones ‹:glob› from comment #59)
> - searching for bugs with an empty QA contact is non-obvious and slow

bug 673012 has been backported to bmo and is now live.  you can search for an empty qa contact by looking for "doesn't contain" "@".

Comment 63

[David Lawrence [:dkl]]

Comment on attachment 604067 [details]
migration script v4

Looks good to me. Much less complicated that earlier versions ;) Works for my testing as well on fresh BMO snapshot. r=dkl

Comment 64

[Jason Smith [:jsmith]]

Could we get a status update on the progress of this bug? I'd like to add use of the QA contact field to some of processes on the teams I work with.

Comment 65

[:glob ✱]

ok, i've revisited the correspondence and i believe that we're code complete and good to go!  more more outstanding issues with component watching can be dealt with after the switch.

i haven't revised the announcement text as i can't think of any way to make it clearer (http://globau.wordpress.com/2012/02/21/76/).

i'll chat with IT about scheduling downtime for this, i'm eyeballing next week or the week after that (i'm not sure who needs to be around for this outage, and a lot of things IT-wise have recently shifting with regards to BMO).

once we've set the date, we should re-publish the announcement in the lead up.

Comment 66

[Marcia Knous [:marcia]]

Bryon: Do we have any update on the remaining pieces that need to be done regarding scheduling?

Comment 67

[:glob ✱]

(In reply to Marcia Knous [:marcia] from comment #66)
> Bryon: Do we have any update on the remaining pieces that need to be done
> regarding scheduling?

we're waiting on IT to get things ready to perform a trail run of this on our staging environment (bug 755180).  once that's happened we can schedule the go-live, fire off announcements, etc.

Comment 68

[Marcia Knous [:marcia]]

Thanks for the update. Do we have a firmer ETA on when this would happen? Also do you need help with QA for the staging instance? There are some QA team members that are very interested in seeing this happen ASAP.

(In reply to Byron Jones ‹:glob› from comment #67)
> (In reply to Marcia Knous [:marcia] from comment #66)
> > Bryon: Do we have any update on the remaining pieces that need to be done
> > regarding scheduling?
> 
> we're waiting on IT to get things ready to perform a trail run of this on
> our staging environment (bug 755180).  once that's happened we can schedule
> the go-live, fire off announcements, etc.

Comment 69

[:glob ✱]

(In reply to Marcia Knous [:marcia] from comment #68)
> Thanks for the update. Do we have a firmer ETA on when this would happen?
> Also do you need help with QA for the staging instance? There are some QA
> team members that are very interested in seeing this happen ASAP.

sorry about the delays; we're looking at mid tomorrow (in about 12 hours) for the run on staging.

Comment 70

[:glob ✱]

the trial run was successful \o/

the entire process to update about 450,000 bugs took approx 15 minutes, which is quicker than i expected.


i'll chat with IT tomorrow morning about an appropriate window for deploying this to production; i have about two weeks away in mind to give us time to finalize and publish the announcement.

Comment 71

[:glob ✱]

we're still sorting out a suitable time for this migration, but we're looking at somewhere in the period of jul 4 to jul 6.  as soon as we have a fixed time i'll post it here so we can get the announcements out.

Comment 72

[:glob ✱]

i've scheduled a two hour outage on wed jul 4 2012 at 8pm us/pacific to perform this migration.

Comment 73

[:glob ✱]

deployed, 453,407 bugs updated.