906156 - Rebuild the rendered page database so we start fresh (on Sept 3)

Status: RESOLVED FIXED

(developer.mozilla.org Graveyard :: General, defect)

Description

[Eric Shepherd [:sheppy]]

We've accumulated a lot of out-of-date renderings of templates over the last year+ of time, and it's time to get a fresh start. We've made a lot of really nifty changes to templates in that time that many pages don't reflect yet. Let's fix that by just dumping out all the pages that have been cached and starting over.

We should also keep in mind that there are some pages that use a *lot* of templates that we should make sure we get refreshed before users have to run into them.

Those include pages like:

https://developer.mozilla.org/<locale>/docs/Web/API
https://developer.mozilla.org/en-US/docs/Web/CSS/Reference

The writing team will add all of those links we can think of here, so we can either automatically rebuild them during this process, or just make sure the writers hit them before any users do afterward.

We'd like to do this on Monday, August 26 if at all possible.

Comment 1

[Eric Shepherd [:sheppy]]

Our plan is to use the time between now and August 26 to finish up a round of template improvements to pick up, fwiw.

Comment 2

[Luke Crouch [:groovecoder]]

Should we start this today?

Comment 3

[Jean-Yves Perrier [:teoli]]

We should delay it by one week: the off-site just prevented us to prepare the few templates that we want to update before it.

Comment 4

[Eric Shepherd [:sheppy]]

Agreed; we decided to wait a week, then I forgot to update the bug.

Comment 5

[John Karahalis [:openjck]]

September 2 is Labor Day. How about September 3?

Comment 6

[Eric Shepherd [:sheppy]]

September 3 it is.

Comment 7

[Les Orchard [:lorchard]]

I've started this running as of 2 min ago. The script doesn't have an ETA calculation built in, but I can check on it in a few hours and see what the progress looks like to make a guestimate

Comment 8

[Les Orchard [:lorchard]]

Looking at a quick sample of progress, it seems like rebuilding all 61053 docs will take at least 24 hours at current pace.

Comment 9

[Les Orchard [:lorchard]]

Going AFK for the evening. Looks like we're at 16214 of 61053 documents done. My hasty math says that puts us at an ETA of ~18 hours from now.

Comment 10

[John Karahalis [:openjck]]

After this work is complete, will a template-using page load slowly the first time it is viewed? If so, can we automate this re-caching process so that users and, consequently, our SEO are not affected?

Comment 11

[Eric Shepherd [:sheppy]]

(In reply to John Karahalis [:openjck] from comment #10)
> After this work is complete, will a template-using page load slowly the
> first time it is viewed? If so, can we automate this re-caching process so
> that users and, consequently, our SEO are not affected?

No, the pages are actually in the process of all being regenerated, so there won't be a delay like that. I had thought there would be but I was wrong.

Comment 12

[Les Orchard [:lorchard]]

(In reply to John Karahalis [:openjck] from comment #10)
> After this work is complete, will a template-using page load slowly the
> first time it is viewed? 

No. Again, this is *NOT* a cache purge. Causing every page to be rendered on demand at view time would probably bring the site down with the load.

The wiki is built such that we try to only render pages & run scripts when there's a new revision, or in response to a shift-reload by a logged-in user.

So, the rebuild script is taking the hit of the slowness by doing the equivalent of a shift-reload for each document in series. That's why it's going to take ~24 hours.

> If so, can we automate this re-caching process so
> that users and, consequently, our SEO are not affected?

See bug 772704 about automating this. We need a) a way to inventory what documents are affected by a template change, and b) to trigger background rebuilds of those pages when a change is detected.

Comment 13

[Les Orchard [:lorchard]]

The rebuild is finished, a bit earlier than I expected.