Make it possible to esreindex specific content types

VERIFIED FIXED in 2012-01-24

Status

support.mozilla.org
Search
VERIFIED FIXED
6 years ago
6 years ago

People

(Reporter: jsocol, Assigned: willkg)

Tracking

unspecified
2012-01-24

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: u=dev c=search s=2012.2 p=1 post-sprint [qa-])

(Reporter)

Description

6 years ago
I keep getting timeouts trying to build an Elastic index. Since it always starts in Questions, it always times out in Questions, and I can't get it to index the part I need (the KB) without editing the management command.

It would be great to be able to say

esreindex [app[, app[,...]]

like

esreindex wiki

or

esreindex wiki forum

In its current form, I think that means losing all the data from any unnamed app. Obviously that kind of sucks. Maybe we should consider splitting these into separate indexes? (Maybe further separated by locale, I know we were discussing that at one point.) If so, that should be a blocker bug on this.
Two things:

1. Does changing the timeout in your settings_local.py help? e.g. something like:

    ES_TIMEOUT = 10

2. You can index just a percentage of your db:

    ./manage.py esreindex --percent 5

I don't know if that helps or not, but figured it was worth mentioning.

3. This should be easy to fix. I can bang it out in an hour if you want. These sorts of fixes are helpful since they speed up development, so it's worth it.
Er, let me clarify.

* It's easy to add the arguments and not drop and rebuild the single index.
* I'm pretty sure it's set up to split these into separate indexes on document type without doing much re-plumbing.
* By "an hour", I really mean "half-day".
(Reporter)

Comment 3

6 years ago
Ah, I missed ES_TIMEOUT in the docs. Will do that for now and reorder the management command to do KB first.

This is a case where the percentage thing would be useless to me. I need to have the KB indexed. I don't much care about questions and I don't care at all about forums, but I need the KB.

Hopefully it's a nice one-pointer we can do in the next couple sprints.
Making this depend on bug #709080 since that refactors ES indexing.
Depends on: 709080
Grabbing this one.

I fixed it in https://github.com/mozilla/kitsune/commit/57f4faf507f9e995b2b8f533530ad678171ce483

Sticking it in the 2012.2 sprint as a 1 pter.

Marking it as [qa-] because it's a command-line thing and developer-oriented.
Assignee: nobody → willkg
Status: NEW → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → FIXED
Whiteboard: u=dev c=search s=2012.2 p=1 [qa-]
Target Milestone: --- → 2012-01-24
Closed as [qa-]
Status: RESOLVED → VERIFIED
I did this work along with other work for the 2012.1 sprint, but technically added this post-sprint, so I'm marking it as such.
Whiteboard: u=dev c=search s=2012.2 p=1 [qa-] → u=dev c=search s=2012.2 p=1 post-sprint [qa-]
You need to log in before you can comment on or make changes to this bug.