729319 - Hide threads that have no answers from /questions list after 90 days

Status: RESOLVED FIXED

(support.mozilla.org :: Forum, task, P1)

Description

[:ibai]

Threads with no answers should be removed from the view and search indexes 3 months after their creation.

After one month, we should add a "noindex" meta tag to make sure that is removed from Google on time.

The threads should be left in the DB for reference.

Comment 1

[James Socol [:jsocol, :james]]

This is going to make the current "worst query on the site" (which counts the number of questions to show) exponentially worse. (Caching this query for 15 minutes dropped DB load by approximately 80%.)

If we're going to do this, I'd propose a small UI change to questions pagination that would let us drop this query, and it should be a blocker for this bug.

We should drop the full paginator and just go with Previous, Next links. c.f. Reddit.

That means we'd never have to count how many visible questions there are, which means we could drop that query completely. The downside is that users couldn't jump to page X via the UI, and they wouldn't know they were on the last page until we said something like "there's nothing here". But there are so many pages right now that I doubt anyone ever goes that far, anyway.

Comment 2

[Will Kahn-Greene [:willkg] ET needinfo? me]

(In reply to James Socol [:jsocol, :james] from comment #1)
> This is going to make the current "worst query on the site" (which counts
> the number of questions to show) exponentially worse. (Caching this query
> for 15 minutes dropped DB load by approximately 80%.)
> 
> If we're going to do this, I'd propose a small UI change to questions
> pagination that would let us drop this query, and it should be a blocker for
> this bug.

If I understand what we're talking about here, this query will go away when we split the ES and Sphinx code because because we'll be able to get search results and everything we need to show them entirely from ES without touching the DB.

I'm hoping to work on bug #728422 in the 2012.5 sprint (i.e. the next one) which will fix this on the ES side.

Does that help?

Comment 3

[Will Kahn-Greene [:willkg] ET needinfo? me]

(In reply to Will Kahn-Greene [:willkg] from comment #2)
> (In reply to James Socol [:jsocol, :james] from comment #1)
> > This is going to make the current "worst query on the site" (which counts
> > the number of questions to show) exponentially worse. (Caching this query
> > for 15 minutes dropped DB load by approximately 80%.)
> > 
> > If we're going to do this, I'd propose a small UI change to questions
> > pagination that would let us drop this query, and it should be a blocker for
> > this bug.
> 
> If I understand what we're talking about here, this query will go away when
> we split the ES and Sphinx code because because we'll be able to get search
> results and everything we need to show them entirely from ES without
> touching the DB.
> 
> I'm hoping to work on bug #728422 in the 2012.5 sprint (i.e. the next one)
> which will fix this on the ES side.
> 
> Does that help?

Nevermind--my head is in the search gutter.

Comment 4

[James Socol [:jsocol, :james]]

(In reply to Will Kahn-Greene [:willkg] from comment #3)
> (In reply to Will Kahn-Greene [:willkg] from comment #2)
> > If I understand what we're talking about here, this query will go away when
> > we split the ES and Sphinx code because because we'll be able to get search
> > results and everything we need to show them entirely from ES without
> > touching the DB.

> Nevermind--my head is in the search gutter.

That may be true, but you raise a good point: there are at least 2 bugs here. One to exclude these from search results, another to exclude them from the /questions view.

Comment 5

[:ibai]

Should I create another bug and rename this to refer to only one of the parts?

Comment 6

[Ricky Rosario [:rrosario, :r1cky]]

Filed Bug 731183 and and Bug 731185.

Comment 7

[Ricky Rosario [:rrosario, :r1cky]]

Filed Bug 731194 for the UI proposal from Comment 1.

Either way we have to be careful here not to make performance worse than it already is.

Comment 8

[Will Kahn-Greene [:willkg] ET needinfo? me]

Tossing this into the 2012.6 sprint to look at dropping questions from the /questions page after 90 days. If I can get to it, I can get to it.

Comment 9

[Will Kahn-Greene [:willkg] ET needinfo? me]

Oops--I meant to grab this one.

Comment 10

[Will Kahn-Greene [:willkg] ET needinfo? me]

I think this is actually a user-facing thing and not a contributor-facing thing.

Comment 11

[Will Kahn-Greene [:willkg] ET needinfo? me]

When I look at the /questions page, there are a bunch of different views of the questions. It's not obvious to me which views "hide threads that have no answers after 90 days" should apply to.

List of views:

* All
* Unsolved
* Solved
* No Replies
* My Contributions

Should it apply to all the views for everyone?

Comment 12

[:ibai]

Hi Will, they should be hidden from all the views. The effect should be like removing them from the DB without actually removing them (for reporting purposes).

Thanks!

Comment 13

[Will Kahn-Greene [:willkg] ET needinfo? me]

Passing this back to Ricky because he implemented it already. It should land soon.

Comment 14

[Ricky Rosario [:rrosario, :r1cky]]

Landed on prod:

https://github.com/mozilla/kitsune/commit/639a92965516e0890a0eeea14e39778e1f6934f2