771045 - bug 684088 caused the 'change date' of bugs touched to be updated

Status: RESOLVED FIXED

(bugzilla.mozilla.org :: General, defect)

Description

[:glob ✱]

bug 684088 caused the 'change date' of bugs touched to be updated, which breaks 'sort by change date'.

we should look at resetting the last change date for bugs that were touched during this update.

Comment 1

[:glob ✱]

Attachment: qa_migration_fix_1.pl

looks like we'll have to delete the entry from the bugs_activity table and reset the bug's last-change and last-emailed data to the prior entry's timestamp (but only for bugs that haven't been updated after the qa-migration code was run).

the attached script does that.  to avoid a massive lock on 400k+ bugs, each bug is run within its own transaction, which means this script takes a very long time to run.

i'm running it on my development environment right now, will get this kicked off on bugzilla-stage and then production if it looks good.

Comment 2

[:glob ✱]

Attachment: qa_migration_fix_2.pl

updated script

Comment 3

[:glob ✱]

Attachment: qa_migration_fix_3.pl

addresses a race condition between checking the last update on the bug and rolling it back.

Comment 4

[Simon Green]

(In reply to Byron Jones ‹:glob› from comment #3)
> Created attachment 639237 [details]
> qa_migration_fix_3.pl
> 
> addresses a race condition between checking the last update on the bug and
> rolling it back.

What is the last change was a comment been added or a new attachment (without a comment)? These events aren't in the bugs_activity table, but would update the last changed timestamp.

Also what if a bug has had no changes (other than this one), and therefore no second last entry in bugs_activity. This that case handled correctly?

Comment 5

[:glob ✱]

Attachment: qa_migration_fix_4.pl

thanks simon!  this revision checks the bug's delta_ts, and will use the bug's creation_ts if there are no other bugs_activity entries.

Comment 6

[:glob ✱]

Attachment: qa_migration_fix_5.pl

adds logging

Comment 7

[Simon Green]

From #bugzilla, times are AEST (UTC+1000)
19:03 < simon> attachment 639257 [details] still doesn't cover the case when the last change is a comment or new attachment.
...
19:09 <@glob> i only touch delta_ts if the bug _hasn't_ been updated, so it's safe to use the second-last entry in this case
19:11 <@glob> it's running on production now (has been for a while), but the updates are being logged so we can fix up issues later
...
19:31 < simon> glob: I'm thinking of a different case.
19:31 < simon> For example,
19:32 < simon> June 20 - set a flag (creates bugs_activity record)
19:32 < simon> June 25 - make a comment (no bugs_activity record)
19:33 < simon> June 31 - change qa (creates record we want to remove)
19:33 < simon> No further changes are made.
19:33 < simon> Your script is going to come along and reset the date to June 20th (the second last bugs_activity record).
19:33 <@glob> ah, rap
19:33 <@glob> crap
19:33 <@glob> i see
19:34 < simon> Tomorrow I will do something to the bug. The date of that bugmail is going to be June 25th, and it will have the comment as well as the change I make tomorrow.
19:34 < simon> #failmail
19:34 < simon> Now it's not the end of the world stuff, but it might confuse some people.
19:36 < simon> and it is easy enough to modify the script to handle this situation (but much harder if you have already run it). You'll probably need to use the log from the run you are doing now, since the bugs_activity table doesn't have the June 30th change anymore :(
19:37 < simon> If it's in a single db transaction Ctrl-C is your friend. :)
19:37 <@glob> yeah, it's done about 200k bugs
19:37 <@glob> it isn't in a single transaction, i didn't want to hold a lock for that long :(
19:37 <@glob> first things first, fix the script
19:37 <@glob> then we can deal with the mistakes from the log
19:39 < simon> No point fixing the script. You won't be able to run it again. I'd go with a second script that takes the log from the first script, sees if their is a later comment creation or attachment creation, and then update the bug creation date (if it hasn't changed since the check)
19:39 < simon> YMMV :)
19:39 <@glob> that sounds sane
19:47 < simon> glob: Do you want me to put the above as a bug comment?
19:47 <@glob> simon, yeah, let's track that
19:48 <@glob> on the plus side, i'm having difficulty finding a bug which matches that scenario
19:48 <@glob> oo, found one

Comment 8

[:glob ✱]

the script in attachment 639257 [details] has completed its run over production; i'm now working on another script to fix the error which simon pointed out.

Comment 9

[:glob ✱]

the fixup script has been run, 43,720 bugs were updated.