1475801 - (bmo-emoji-restore-attachments) Restore old attachments that were moved as part of emoji migration

Status: VERIFIED FIXED

(bugzilla.mozilla.org :: Infrastructure, enhancement)

Description

[Dylan Hardison [:dylan] (he/him)]

Roughly half of the eight million attachments were moved out of storage during the utf8mb4 conversion in order to speed up that process. They need to be restored.

Comment 1

[:Paolo Amadini]

Thanks. For the record, I was just looking at a comment from 2003 for the XUL removal project, and the associated attachment would've been useful to make sense of the comment and confirm that the reference we have to it in the codebase is now outdated.

This is just to say that while we can live without older Bugzilla attachments, they do speed up some parts of our work!

Comment 2

[Jorg K (CEST = GMT+2)]

I'm wondering in general whether some attachments can't be deleted for good.

Like people attach 1MB+ screenshots or even videos. I think that those could go after - say - five years if the bug is resolved.

Comment 3

[Thomas D. (:thomas8)]

(In reply to Jorg K (GMT+2) from comment #2)
> I'm wondering in general whether some attachments can't be deleted for good.
> 
> Like people attach 1MB+ screenshots or even videos. I think that those could
> go after - say - five years if the bug is resolved.

That sounds like a truly bad idea for future reference, and would be pretty hard and time-wasting for anyone to make the call between dispensable and potentially relevant attachments. This would quite randomly eliminate part of bug history, and also part of contributors' activity track record - we may not like the file size, but someone made an effort to produce and publish that screenshot or video to help improve Mozilla products, or to propose some improvement which might have been rejected at the time but potentially still of interest later. Wrongly wontfixed RFE's also come to mind. And I thought the times of worrying over data storage sizes are over... (and fortunately, it's not everyday that we're doing utf8mb4 conversions...)

Comment 4

[Dylan Hardison [:dylan] (he/him)]

(In reply to Jorg K (GMT+2) from comment #2)
> I'm wondering in general whether some attachments can't be deleted for good.
> 
> Like people attach 1MB+ screenshots or even videos. I think that those could
> go after - say - five years if the bug is resolved.

The size of all the attachments isn't that big of a deal, most of the time.
However in this case we knew that halving the number would speed up the row format conversion, so we decided to trade inconvenience (missing attachments) for speed. Considering it still took 14 hours, I'm pretty happy with this decision.

Deciding to do something with the number of attachments because of this would be a premature optimization.
This is an anomalous event in the 20 year history of this database. :-)

Comment 5

[Darkspirit]

(In reply to Jorg K (GMT+2) from comment #2)
> or even videos.

Sorry 😅

Comment 6

[Dylan Hardison [:dylan] (he/him)]

(In reply to Jan Andre Ikenmeyer [:darkspirit] from comment #5)
> (In reply to Jorg K (GMT+2) from comment #2)
> > or even videos.
> 
> Sorry 😅

Never apologize for using a feature in roughly the way it is intended. :-)

Comment 7

[Dylan Hardison [:dylan] (he/him)]

Bob will be starting this soon, and attachments will be restored in reverse chronological order. 
Timing is a bit rough, and after having 8 hours turn into 14, I'm hesitant to say but "many hours, probably less than a day" is reasonable.

Comment 8

[Bob Micheletto [:bobm]]

(In reply to Dylan Hardison [:dylan] (he/him) from comment #7)
The process has begun.

Comment 9

[Bob Micheletto [:bobm]]

(In reply to Bob Micheletto [:bobm] from comment #8)
> The process has begun.
Backup complete.  Beginning copy / import.

Comment 10

[Bob Micheletto [:bobm]]

(In reply to Bob Micheletto [:bobm] from comment #9)
> (In reply to Bob Micheletto [:bobm] from comment #8)
Import stalled roughly at a third of the way through completion.  Working through an issue with the import.

Comment 11

[Bob Micheletto [:bobm]]

Import finished at 18:37 US/Eastern.  Closing bug.

Comment 12

[Bob Micheletto [:bobm]]

(In reply to Bob Micheletto [:bobm] from comment #11)
> Import finished at 18:37 US/Eastern.  Closing bug.

Note: That was 18:37 US/Pacific