878453 - Sort articles and subtopics on topic landing page

Status: RESOLVED WONTFIX

(support.mozilla.org :: Knowledge Base Software, task, P3)

Description

[Kadir Topal [:atopal]]

One thing to resolve still is the issue of how to order subtopics amongst the articles. As I see it we have several options. 

Principles: 
* Users don't care whether something is a subtopic or article
* Users don't care how involved the method of ranking is as long as they see the right results
* We want the algorithm to do the right thing in most cases, not rank things manually all the time.
* A simple method that produces a good result is better than a complicated method that produces slightly better results.

Proposal: (keep in mind that I have no idea about the effort need to implement this)
Show items ranked by their page view and ratings minus the page views coming from in-product links (counting in-product links puts on top articles no-one wants to navigate to, like how to send flash crash data). Subtopics would be ranked by the combined views and ratings the articles subsumed under them generate.

1. Article 1 (ranking of article 1)
2. Topic 1 (ranking of article x + ranking of article y + ranking of article z)
3. Article 2 (ranking of article 2)
etc.

As far as I can tell this satisfies all the principles, but I don't know if there are things that would make this hard to implement. Any other suggestions? Maybe based on different principles?

Comment 1

[Ibai Garcia [:ibai]]

I like the idea. I'm just not sure if the efforts is worth the handful of articles that only suffer from this problem. I.e. the proposal conflicts with the 4th principle.

I will go with the same proposal without the exception. I recall Ricky having a concern about giving too much weight to subtopics, but I think that's a desirable behavior: subtopics have more content on them so more users should be interested on them.

My only concern is about those subtopics with particularly large amount of articles that are not necessarily popular.

Could it be possible to manually curate the list for the last month for the "Customize..." topic (it's the most diverse one in terms of amount of articles per subtopics, it has articles with in product links and it has some of the most popular articles) and use it to better understand what would it happen? If this is helpful I can do it myself this week.

Comment 2

[Ricky Rosario [:rrosario, :r1cky]]

Moving this to the backlog. Let's first implement the new pages and new product/topic/subtopic structure. Then we can look at sorting.

Comment 3

[Ricky Rosario [:rrosario, :r1cky]]

Unblocking. We can work on sorting after the page is implemented. behing the flag or not.

Comment 4

[Ibai Garcia [:ibai]]

For Reference, if we aggregate visits of articles on each subtopic and then we organize but visits, Customize... has the following order:

Customize Firefox with add-ons, plugins, and extensions (17)	3311068
Firefox options, preferences and settings (12)	1199863
Firefox Sync settings (10)	584577
Firefox controls and buttons (6)	185248
Tab settings (5)	148327
Customize Firefox controls, buttons and toolbars 	132448
How to make web links open in Firefox by default 	91204
Bookmark options (2)	54557
Block and unblock websites with parental controls 	39205
Viewing HTML5 audio and video in Firefox 	10731

All the subtopics but one on top and one subtopic 2 lines under that.

And probably it's ok to just sort them by subtopics first and articles after. The subtopics have more content on it so they are better chances that the answer that you are looking for is there.

Ideas?

Comment 5

[Verdi [:verdi] Best to slack me]

(In reply to Ibai Garcia [:ibai] from comment #4) 
> And probably it's ok to just sort them by subtopics first and articles
> after. The subtopics have more content on it so they are better chances that
> the answer that you are looking for is there.
> 
> Ideas?

I think we still have to count visits so we know how to sort the subtopics. If that's the case, we can just sort the whole list rather than sort subtopics at the top and then sort articles below. Or am I misunderstanding this?

Comment 6

[Ricky Rosario [:rrosario, :r1cky]]

First off... We currently are sorting articles by number of votes (as a proxy to number of views). Is that not working? Do we want to change it?

The least amount of work solution is to keep subtopics separate (at the top?) sorted by the display order which can be changed in the admin. And keep articles sorted as they are today.

Comment 7

[Ibai Garcia [:ibai]]

Ricky...it was easier for me to do it this way.. the point is that reads are highly correlated to votes...isn't it?

What I want to say is that these data indicates that it wouldn't be that different to show subtopics on top and then articles.. and if that's easier/cleaner...we should go that way. IMHO.

Comment 8

[Kadir Topal [:atopal]]

So, we moved to the new topic landing pages. Ricky, can you explain what model we ended up using?

Comment 9

[Ricky Rosario [:rrosario, :r1cky]]

Same sort as before for articles (by votes as a proxy of views). Subtopics at the top sorted by display order from the database.

Comment 10

[Kadir Topal [:atopal]]

Hmm, there is some overlap between the votes and the views. Currently the top 20 upvoted articles have 13 articles in common with the top 20 most viewed articles, meaning that 35% of the most viewed articles are not getting as many votes. The same goes for the top 50, the overlap is 65%.

Also, there are some oddities. In the top 20 of votes for example we have the following:

* How can I help by submitting mobile performance data?
* JavaScript settings and preferences for interactive web pages
* Flash 11.3 doesn't load video in Firefox
* Firefox release notes - What's New and Known Issues

In any case, both approaches suffer from the feedback loop: The more votes an article has, the higher up it will be shown. The higher up it's shown, the more people will click through to it. The more people click through to it, the more people will vote the article up, and the cycle continues.

The upside of using votes is that it only includes articles where we can be reasonably sure that people read them, unlike views, where people might land by accident. Still, more views, also from in-product, means more votes, but this is the hierarchy of our support pages, it doesn't need to accommodate searchers or people who click in-product links.

In the end it might be that Google searches and people starting from /products cancel each other out in views, and there is no feedback loop. In any case, we should see if we are still favoring the first 3 articles and test with tree jack if that's good enough to cover our most important tasks on the page.

Leaving this open for. Let me know if you have a good idea about sorting or testing this.

Comment 11

[Ricky Rosario [:rrosario, :r1cky]]

2013Q2 -> Future