Closed Bug 1135269 Opened 11 years ago Closed 11 years ago

UX blog posts since 2013 are missing :(

Categories

(Data & BI Services Team :: DB: MySQL, task)

Product:
Data & BI Services Team
Component:
DB: MySQL
Version:
Operational Work
Type:
task
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: madhava, Unassigned)

Details

(Whiteboard: [kanban:https://webops.kanbanize.com/ctrl_board/2/613] )

Hi there - All the posts since some time in 2013 on blog.mozilla.org/ux/ seem to be gone. There's a lot in there that we'd really like to have online. Are there backups? Help!
Assignee: nobody → server-ops-webops
Component: blog.mozilla.org → WebOps: IT-Managed Tools
OS: Mac OS X → All
Product: Websites → Infrastructure & Operations
QA Contact: nmaul
Hardware: x86 → All
Version: unspecified → other
Whiteboard: [kanban:https://webops.kanbanize.com/ctrl_board/2/613]
Moved to WebOps component. I have no idea what happened but have some vague recollection that something like this happened once before?
DB folks: Do y'all have a backup of the wp_150-related tables of the blog_mozilla_org database that might contain the missing 2013-2015 UX blog entries? It looks like the last time the blog was updated was in November of this year. [1] The site id is 150; the tables containing the elements of the blog posts should be prefaced by "wp_150_". [2] There is a very odd gap, where the last post visible in the blog is "A Browser for Teens by Teens": the database shows a post entitled "The Internet and Browsing in Indonesia: Five Findings from the Field" that doesn't appear and there's no other posts until auto-saved draft posts with a date from earlier this week (2015-02-20). [3] [1] mysql> select blog_id, site_id, domain, path, registered, last_updated from wp_blogs where path like '%ux%'; +---------+---------+------------------+------+---------------------+---------------------+ | blog_id | site_id | domain | path | registered | last_updated | +---------+---------+------------------+------+---------------------+---------------------+ | 150 | 1 | blog.mozilla.org | /ux/ | 2011-07-19 14:05:39 | 2014-11-25 17:38:01 | +---------+---------+------------------+------+---------------------+---------------------+ [2] These tables are: | wp_150_comments | | wp_150_ewwwio_images | | wp_150_links | | wp_150_options | | wp_150_postmeta | | wp_150_posts | | wp_150_term_relationships | | wp_150_term_taxonomy | | wp_150_terms | [3] mysql> select ID, post_author, post_date, post_date_gmt, post_title, post_status from wp_150_posts where post_date > '2013-11-07'; +------+-------------+---------------------+---------------------+----------------------------------------------------------------------+-------------+ | ID | post_author | post_date | post_date_gmt | post_title | post_status | +------+-------------+---------------------+---------------------+----------------------------------------------------------------------+-------------+ [...] | 2945 | 477 | 2013-11-07 17:04:55 | 2013-11-08 01:04:55 | A Browser for Teens by Teens | inherit | | 2946 | 477 | 2013-11-07 17:08:04 | 2013-11-08 01:08:04 | A Browser for Teens by Teens | inherit | | 2948 | 507 | 2013-11-08 13:03:02 | 2013-11-08 21:03:02 | The Internet and Browsing in Indonesia: Five Findings from the Field | inherit | | 2950 | 507 | 2015-02-20 14:31:31 | 0000-00-00 00:00:00 | Auto Draft | auto-draft | | 2951 | 59 | 2015-02-20 14:36:37 | 0000-00-00 00:00:00 | Auto Draft | auto-draft | | 2952 | 437 | 2015-02-20 14:43:05 | 0000-00-00 00:00:00 | Auto Draft | auto-draft | +------+-------------+---------------------+---------------------+----------------------------------------------------------------------+-------------++-------------------------------------------+-------------+
Assignee: server-ops-webops → server-ops-database
Component: WebOps: IT-Managed Tools → Database Operations
Product: Infrastructure & Operations → Data & BI Services Team
QA Contact: nmaul → scabral
Version: other → Operational Work
Which db cluster does this live on?
Found it (generic db cluster). When was this found to be an issue? Just trying to figure out which backup to get data from. We have backups from the last 30 days, and also from the first of the month for the past 8 months.
Hi - I just noticed it on Friday, but I don't know how long it's been the case :( Definitely it was all there around when we posted for the search UI improvements which were in ... December. BTW - we just had a team member post just now, here, so there is one new 2015 post: https://blog.mozilla.org/ux/2015/02/save-share-revisit/
Looks like the blog_mozilla_org database had a bunch of stuff deleted between Jan 26-27 - the logical backups go from 2.2G of sql text to 1.8G (uncompressed). wp_150_ tables
wp_150_ tables went from 81M to 34M. Did anything happen Jan 26-ish?
I have added back in all the data from the wp_150 tables: - using the backup - INSERT lines only - changing to INSERT IGNORE in case the values are already there Data was restored to the following tables: `wp_150_commentmeta` `wp_150_comments` `wp_150_ewwwio_images` `wp_150_links` `wp_150_options` `wp_150_postmeta` `wp_150_posts` `wp_150_term_relationships` `wp_150_terms` `wp_150_term_taxonomy` `wp_usermeta` https://blog.mozilla.org/ux/ seems to have posts from 2014 again (I only checked the first few posts). I have checked and there was nothing we did on Jan 26th (a Monday) to generic in phx. :-\ We only have changelogs for a week so we cannot go back and see what happened on the db side - and even if we could, it would only show us the commands that were run, not who did them. I'm concerned that other tables may be missing data as well. It'd be awesome to know what happened so we can target the proper tables for restoration.
Looks like there are also missing posts from: 3 INSERT INTO `wp_27_posts` 9 INSERT INTO `wp_88_posts` 1 INSERT INTO `wp_100_posts` 27 INSERT INTO `wp_193_posts` 1 INSERT INTO `wp_237_posts` 2508 INSERT INTO `wp_240_posts` 9 INSERT INTO `wp_242_posts` 5 INSERT INTO `wp_244_posts` For reference, wp_150 had 601 entries that were removed from Jan 26 to Jan 27th: 601 INSERT INTO `wp_150_posts` Most concerning is wp_240 - what is that?
blog_id 240 leads to path /openstandard-en/ Here's a list: | 27 | /webdev/ | 3 posts/revisions removed | 88 | /about_mozilla/ | 9 posts/revisions removed | 100 | /addons/ | 1 post/revision removed | 193 | /webqa/ | 27 posts/revisions removed | 237 | /getfirebug/ | 1 post/revision removed | 240 | /openstandard-en/ | 2508 posts/revisions removed | 242 | /hacks/ | 9 posts/revisions removed | 244 | /quality/ | 5 posts/revisions removed
I'll open a separate bug to track talking to each of the blog owners to see if they are missing posts. @madhava, @craigcook: Does it look like all of the missing posts have been recovered?
(In reply to C. Liang [:cyliang] from comment #11) > I'll open a separate bug to track talking to each of the blog owners to see > if they are missing posts. > > @madhava, @craigcook: Does it look like all of the missing posts have been > recovered? Thanks for helping us fix this and get the posts back. It looks like the posts have been recovered. I checked a bunch from last year and they were recovered, but it's hard to be 100% sure without having a list of posts to compare them to. Any idea why this happened or if there's a chance it will happen again? I only ask since I'm about to share a few posts with a mailing list. I can hold off if you don't think the blog is stable for some reason.
It's hard to say what might have happened given that we no longer have logs going that far back. =\ As far as I can tell, the blog is stable so I don't think you need to hold off on sharing posts.
(In reply to C. Liang [:cyliang] from comment #13) > It's hard to say what might have happened given that we no longer have logs > going that far back. =\ > > As far as I can tell, the blog is stable so I don't think you need to hold > off on sharing posts. Ok. Thanks! I'll share away.
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.