Closed Bug 791758 Opened 7 years ago Closed 7 years ago

Mass change from whiteboard '[orange]' to keyword 'intermittent-failure'

Categories

(Developer Services :: General, task)

task
Not set

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: emorley, Assigned: fox2mike)

References

Details

Attachments

(2 files, 2 obsolete files)

[I'm filing this bug now so I can get all the dependencies straight, but please hold off any work for a couple of days, until feedback on the dev.platform post (linked in bug 790571) has been obtained.]

In bug 790571, we would like to switch from using whiteboard '[orange]' to using a new keyword 'intermitent-failure', for our intermittent failure bugs.

As such we would like:
1) A new 'intermitent-failure' keyword added to b.m.o
2) A script run to add this new keyword to all bugs on b.m.o whose whiteboard contain '[orange]', to allow testing to begin in bug 790571. (Without causing bugmail).
3) Once the patches in bug 790571 have been staged/tested/deployed:
  a) the script in step #2 run again to catch any stragglers.
  b) a script run to mass remove '[orange]' from the whiteboard of any bug that has the keyword intermitent-failure - again without causing bugmail.
The other bugs are shaping up, so I don't suppose we can now:

1) Add a new "intermittent-failure" keyword to b.m.o
2) Create the script mentioned in comment 0, so we have it ready to be run on production.
Blocks: 807114, 807113, 807120
(In reply to Ed Morley [:edmorley UTC+0] from comment #1)
> The other bugs are shaping up, so I don't suppose we can now:
> 
> 1) Add a new "intermittent-failure" keyword to b.m.o

Sure. Can I get a description for the new keyword and I will go ahead and get that created.

> 2) Create the script mentioned in comment 0, so we have it ready to be run
> on production.

Will start working on a script.

dkl
Assignee: nobody → dkl
Status: NEW → ASSIGNED
Flags: needinfo?(bmo)
(In reply to David Lawrence [:dkl] from comment #2)
> Sure. Can I get a description for the new keyword and I will go ahead and
> get that created.

"For the tracking of intermittent test failures within the test-suites of Gecko related products."

> Will start working on a script.

Many thanks! :-)
Flags: needinfo?(bmo)
keyword created
these two scripts are almost identical .. any reason why they aren't combined into a single script?
(In reply to Byron Jones ‹:glob› from comment #7)
> these two scripts are almost identical .. any reason why they aren't
> combined into a single script.

I could do it as a single script but I was trying to keep them both as simple as possible. According to Ed's instructions, he wanted to do the migration as a two step process. 1) Just add the new keyword but yet leave the whiteboard alone. 2) Later on once everything seems to be going ok, then run another script to go back and remove the orange whiteboard. No reason technically that they can't be in one script but figured it would be easier to review this way.

dkl
Since comment 0 I've realised we need to match against both [orange] and [tb-orange], sorry!

(In reply to David Lawrence [:dkl] from comment #8)
> (In reply to Byron Jones ‹:glob› from comment #7)
> > these two scripts are almost identical .. any reason why they aren't
> > combined into a single script.
> 
> I could do it as a single script but I was trying to keep them both as
> simple as possible. According to Ed's instructions, he wanted to do the
> migration as a two step process. 1) Just add the new keyword but yet leave
> the whiteboard alone. 2) Later on once everything seems to be going ok, then
> run another script to go back and remove the orange whiteboard. No reason
> technically that they can't be in one script but figured it would be easier
> to review this way.

I guess the only other consideration, is that for (2) above, we'll have to re-run the part 1 script anyway, to catch bugs filed/changed since. Whilst unlikely, someone could then still file additional bugs in-between the first and second script being run, unless we add the keyword at the same time as removing the whiteboard annotation.

As such, how about a unified script along the lines of:

* Search for all bugs with [orange] or [tb-orange] in the whiteboard
* For each bug:
  If "--remove" parameter specified:
    ** Remove [orange] / [tb-orange] from whiteboard
    ** Add keyword intermittent-failure
  else
    ** Just add keyword intermittent failure
* Print vs print+updateDB would depend on an additional parameter, similar to the current script.
(In reply to Ed Morley [:edmorley UTC+0] from comment #9)
> I guess the only other consideration, is that for (2) above, we'll have to
> re-run the part 1 script anyway, to catch bugs filed/changed since. Whilst
> unlikely, someone could then still file additional bugs in-between the first
> and second script being run, unless we add the keyword at the same time as
> removing the whiteboard annotation.
> 
> As such, how about a unified script along the lines of:
> 
> * Search for all bugs with [orange] or [tb-orange] in the whiteboard
> * For each bug:
>   If "--remove" parameter specified:
>     ** Remove [orange] / [tb-orange] from whiteboard
>     ** Add keyword intermittent-failure
>   else
>     ** Just add keyword intermittent failure
> * Print vs print+updateDB would depend on an additional parameter, similar
> to the current script.

Sure I could do that. Will work on another version of the script here shortly.

dkl
Comment on attachment 676902 [details] [diff] [review]
Script to add intermittent-failure keyword to orange bugs (v1)

Review of attachment 676902 [details] [diff] [review]:
-----------------------------------------------------------------

since the script will be updated, i'll dump two minor nits...

::: contrib/reorg-tools/add_orange_keyword.pl
@@ +51,5 @@
> +
> +# intermittent-keyword id (assumes already created)
> +my $keyword_id = $dbh->selectrow_array(
> +    "SELECT id FROM keyworddefs WHERE name = 'intermittent-failure'");
> +$keyword_id or die "Can't find keyword id for 'intermittent-failure'\n";

nit: get_field_id() may be clearer for both of these.

@@ +61,5 @@
> +my $bugs = $dbh->selectall_arrayref("
> +    SELECT DISTINCT bugs.bug_id
> +      FROM bugs LEFT JOIN keywords ON bugs.bug_id = keywords.bug_id
> +     WHERE keywords.keywordid != ?
> +           AND bugs.status_whiteboard REGEXP '\\\\[orange\\\\]'

there's no reason for this to be a regex; use LIKE '%[orange]%'
Attachment #676902 - Flags: review?(glob)
Attachment #676903 - Flags: review?(glob)
New single script that handles both adding the keyword and optionally cleaning the whiteboard.

dkl
Attachment #676902 - Attachment is obsolete: true
Attachment #676903 - Attachment is obsolete: true
Attachment #679748 - Flags: review?(glob)
Comment on attachment 679748 [details] [diff] [review]
Script to add intermittent-failure to orange bugs and optionally update whiteboard (v1)

Review of attachment 679748 [details] [diff] [review]:
-----------------------------------------------------------------

r=glob

::: contrib/reorg-tools/migrate_orange_bugs.pl
@@ +73,5 @@
> +$field_id or usage("Can't find field ID for 'keywords' field\n");
> +
> +# intermittent-keyword id (assumes already created)
> +my $keyword_id = Bugzilla::Keyword->new({ name => 'intermittent-failure' })->id;
> +$keyword_id or usage("Can't find keyword id for 'intermittent-failure'\n");

this throws:
Can't call method "id" on an undefined value at ./contrib/reorg-tools/migrate_orange_bugs.pl line 76.
when the keyword doesn't exist.

@@ +135,5 @@
> +
> +    print "done.\n";
> +}
> +
> +$dbh->bz_commit_transaction() if $doit;

i'd like to see a message at the end of the run if $doit is false so the user knows that nothing actually happened.
Attachment #679748 - Flags: review?(glob) → review+
Comment on attachment 679748 [details] [diff] [review]
Script to add intermittent-failure to orange bugs and optionally update whiteboard (v1)

>+    FROM bugs WHERE bugs.status_whiteboard LIKE '%[orange]%'",
...
>+            $whiteboard =~ s/\[orange\]//ig;

Sorry to be a pain, but we need to match against [orange] and also [tb-orange] bugs (comment 9).

Thank you so much for doing this! :-D
Thanks glob. Committed.

Committing to: bzr+ssh://dlawrence%40mozilla.com@bzr.mozilla.org/bmo/4.0
added contrib/reorg-tools/migrate_orange_bugs.pl
Committed revision 8377.

Committing to: bzr+ssh://dlawrence%40mozilla.com@bzr.mozilla.org/bmo/4.2             
added contrib/reorg-tools/migrate_orange_bugs.pl
Committed revision 8400.

Ed, please advise when we should schedule to have the 1st phase of the script ran against production.

dkl
Flags: needinfo?(bmo)
(In reply to David Lawrence [:dkl] from comment #15)
> Ed, please advise when we should schedule to have the 1st phase of the
> script ran against production.

At whatever point you have a spare moment would be great (no immediate rush), thank you :-)
(1st phase, so running without the --remove option)
Flags: needinfo?(bmo)
(In reply to Ed Morley [:edmorley UTC+0] from comment #16)
> At whatever point you have a spare moment would be great (no immediate
> rush), thank you :-)
> (1st phase, so running without the --remove option)

Thanks. Will do now then :)

Admin, please execute the following script on the production BMO web head of your choice:

# cd /path/to/bugzilla/code
# perl -wT -I. contrib/reorg-tools/migrate_orange_bugs.pl --doit

Thanks
dkl
Assignee: dkl → server-ops-devservices
Status: ASSIGNED → NEW
Component: Administration → Server Operations: Developer Services
Product: bugzilla.mozilla.org → mozilla.org
QA Contact: shyam
Version: Production → other
Ben/Shyam, let me know if you need any more info for this :-)
(In reply to Ed Morley [:edmorley UTC+0] from comment #18)
> Ben/Shyam, let me know if you need any more info for this :-)

I know fox2mike is traveling at the moment but there are other IT admins that can run scripts for us on production. Maybe Ashish?

dkl
I think we haven't pushed bugzilla since the script was committed to the tree and therefore the script isn't available to be run from the webheads...
Assignee: server-ops-devservices → shyam
Attached file Command output
And as soon as I said that... heh.

All good!
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
(In reply to Shyam Mani [:fox2mike] from comment #21)
> Created attachment 683320 [details]
> Command output
> 
> And as soon as I said that... heh.
> 
> All good!

Thanks Shyam. 

Ed, when you are ready to remove the whiteboard entries with --remove, please clone this bug and create a new one so we can have IT run the script again then.

Thanks
dkl
Will do; thank you Shyam! :-)
Blocks: 814083
Depends on: 815112
Component: Server Operations: Developer Services → General
Product: mozilla.org → Developer Services
You need to log in before you can comment on or make changes to this bug.