If you think a bug might affect users in the 57 release, please set the correct tracking and status flags for Release Management.

Unified Search: Limit age of forum posts in results

RESOLVED FIXED in 2012Q4

Status

support.mozilla.org
Search
P3
normal
RESOLVED FIXED
5 years ago
5 years ago

People

(Reporter: Matt_G, Assigned: rrosario)

Tracking

unspecified
2012Q4

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: u=user c=search p=1 s=2012.21)

(Reporter)

Description

5 years ago
There should be an upward threshold for the age of forum posts returned in unified search. We assume that newer content will be more accurate and cut down on users "reviving" ancient threads. This threshold should apply to unified search only and should not impact advanced search.
How do you determine the age of a forum post?

What does "upward threshold" mean?
(Reporter)

Comment 2

5 years ago
Some forums posts are more than a year old and unsolved. They are showing up fairly high in the search results in some instances. We should set a limit on how old a post can be if we are going to display it. Kadir, Michael, and I are working out what that limit should be. I'll add that information in the next day or so.
No longer blocks: 765768
Baseline: Don't show any question older than 180 days (creation date). We should probably be more aggressive though. We'll work that out later.

Updated

5 years ago
Whiteboard: u=user c=search p= s=2012.21
(Assignee)

Comment 4

5 years ago
See bug 688463 for a bunch of implementation ideas.
(Assignee)

Comment 5

5 years ago
Making this a 3pter as there is some research/prototyping that needs to happen to figure out how to implement this.
Whiteboard: u=user c=search p= s=2012.21 → u=user c=search p=3 s=2012.21
(Assignee)

Updated

5 years ago
Priority: -- → P3
Hardware: x86 → All
Target Milestone: --- → 2012Q4
(Assignee)

Comment 6

5 years ago
I talked with willkg again and this is easier than all the comments in bug 688463. We need to add something like `.filter(created__gt=convert_to_timestamp(now()-timedelta(days=180)))`

Also add tests.

I don't think that takes over half day => 1pt
Whiteboard: u=user c=search p=3 s=2012.21 → u=user c=search p=1 s=2012.21
(Assignee)

Comment 7

5 years ago
working on this today.
Assignee: nobody → rrosario
(Assignee)

Comment 8

5 years ago
in a pull request:
https://github.com/mozilla/kitsune/pull/938
It's actually: https://github.com/mozilla/kitsune/pull/939
(Assignee)

Comment 10

5 years ago
Landed and deployed:

https://github.com/mozilla/kitsune/commit/c170683211985896a7b30cee0c88c1e634b53729

I'm not sure what the CTR will do, but helpfulness in the Support Forum will hopefully go up.
Status: NEW → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED
First day impact is in: http://www.evernote.com/shard/s4/sh/830733d7-6a82-4227-8be3-d3a6e1d958fd/c601a8d599fa47a5b6ac7f11532262be

CTR for KB is up by 7 percentage points, but CTR for forum is down 13 percentage points. Overall CTR is down 6 percentage points. 

The helpfulness in the KB from search is flat, while the helpfulness in the forums from search is still not measurable and the overall helpfulness in the forums is flat. We only have one day of CTR data, so let's wait a little longer before calling anything.
The first full day is in, and the CTR is down 10 percentage points. At this point one thing is clear to me: We have to find out better ways to judge the impact of this. For once, we'd need to know, are we serving people better or worse now that we eliminate threads that are meant for way older releases?

As far as I can tell, we have currently no automated way to measure that. Here are some ideas on what do do, and please jump in if you have any other ideas:

1. Add helpful vote meta data for forum votes (might help, but will take several weeks to produce the confidence level we need)

2. Add SurveyGizmo survey to those threads that are visited from our internal search (we'd need to figure out how and when to show the survey so it makes sense)

3. Measure how many people click through to a search result and go back to the search results again (I don't think we can get this from the logs, but maybe GA can do that?)

4. Compare results manually. Look at queries that would have produced an old forum thread at the first position before and produces a KB thread now.

5. Experiment with more weight to forum threads now that we have fewer of them.

Any thoughts on this, or other ideas?
You need to log in before you can comment on or make changes to this bug.