Suspicious "me too" numbers

RESOLVED WORKSFORME

Status

support.mozilla.org
Questions
RESOLVED WORKSFORME
6 years ago
4 years ago

People

(Reporter: atopal, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

(Reporter)

Description

6 years ago
Created attachment 583747 [details]
webtrends data for specific article

David points out that some of the obscure questions get a high number of "me too" votes, which seems strange, and Ibai showed data from Webtrends that doesn't seem to back up the number of votes displayed. 

Specifically, we are talking about this article:
https://support.mozilla.com/questions/901985

Question filed 2 weeks ago
47 people have this problem 15 new this week. 

But Webtrends shows that that question only had 27 views in total. So it seems there is something wrong, either in the way we calculate the "me too" votes or the way that webtrends collects view data.

Can you suggest a way to test the correctness of the me too votes?
(In reply to Kadir Topal [:atopal] from comment #0)

> Can you suggest a way to test the correctness of the me too votes?

Our votes data is more reliable than webtrends data. The only way to add a vote is to actually post to the form url. I will investigate this specific case to see where the votes came from and verify they make sense. I'll comment my findings.
Assignee: nobody → rrosario
Target Milestone: --- → 2011Q4
I didn't find anything suspicious other than your webtrends numbers ;). Don't forget that users can end up on that page in any locale (for example, https://support.mozilla.com/es/questions/901985) and I am not sure if webtrends is aggregating that, probably not. So either webtrends is way off or you are missing visits to the page in other locales.


Below is the timestamp + user agent for each vote:

2011-12-04 13:28:49 Mozilla/5.0 (Ubuntu; X11; Linux i686; rv:9.0) Gecko/20100101 Firefox/9.0
2011-12-05 00:10:57 Mozilla/5.0 (X11; Linux i686; rv:7.0.1) Gecko/20100101 Firefox/7.0.1
2011-12-05 20:46:52 Mozilla/5.0 (X11; Linux i686; rv:7.0.1) Gecko/20100101 Firefox/7.0.1
2011-12-06 01:32:16 Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/534.26+ (KHTML, like Gecko) Version/5.0 Safari/534.26+ Ubuntu/11.04 (3.0.4-1ubuntu1) Epiphany/3.0.4
2011-12-06 04:39:21 Mozilla/5.0 (X11; Linux i686; rv:7.0.1) Gecko/20100101 Firefox/7.0.1
2011-12-06 13:13:23 Mozilla/5.0 (Ubuntu; X11; Linux i686; rv:8.0) Gecko/20100101 Firefox/8.0
2011-12-06 17:22:01 Mozilla/5.0 (Windows NT 6.1; rv:8.0.1) Gecko/20100101 Firefox/8.0.1
2011-12-06 18:48:19 Mozilla/5.0 (X11; Linux x86_64; rv:7.0.1) Gecko/20100101 Firefox/7.0.1
2011-12-06 18:58:10 Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.121 Safari/535.2
2011-12-06 23:37:00 Mozilla/5.0 (Ubuntu; X11; Linux x86_64; rv:8.0) Gecko/20100101 Firefox/8.0
2011-12-07 06:52:18 Opera/9.80 (X11; Linux i686; U; en) Presto/2.10.229 Version/11.60
2011-12-09 09:14:11 Mozilla/5.0 (X11; Linux i686; rv:7.0.1) Gecko/20100101 Firefox/7.0.1
2011-12-09 12:38:18 Mozilla/5.0 (Windows NT 6.0; rv:8.0) Gecko/20100101 Firefox/8.0
2011-12-09 16:12:12 Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.23) Gecko/20110921 Ubuntu/10.10 (maverick) Firefox/3.6.23
2011-12-10 00:23:37 Mozilla/5.0 (X11; Linux i686; rv:7.0.1) Gecko/20100101 Firefox/7.0.1
2011-12-10 04:47:08 Mozilla/5.0 (Ubuntu; X11; Linux i686; rv:8.0) Gecko/20100101 Firefox/8.0
2011-12-10 13:10:30 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0) Gecko/20100101 Firefox/8.0
2011-12-11 05:50:03 Mozilla/5.0 (Ubuntu; X11; Linux x86_64; rv:8.0) Gecko/20100101 Firefox/8.0
2011-12-11 08:55:53 Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.63 Safari/535.7
2011-12-11 12:49:42 Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0.1) Gecko/20100101 Firefox/8.0.1
2011-12-11 14:10:10 Mozilla/5.0 (Ubuntu; X11; Linux i686; rv:8.0) Gecko/20100101 Firefox/8.0
2011-12-11 18:29:25 Mozilla/5.0 (Ubuntu; X11; Linux x86_64; rv:8.0) Gecko/20100101 Firefox/8.0
2011-12-11 21:03:35 Mozilla/5.0 (X11; Linux i686) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.120 Safari/535.2
2011-12-12 02:35:16 Mozilla/5.0 (Ubuntu; X11; Linux x86_64; rv:8.0) Gecko/20100101 Firefox/8.0
2011-12-12 05:37:11 Mozilla/5.0 (X11; Linux i686; rv:7.0.1) Gecko/20100101 Firefox/7.0.1
2011-12-12 12:43:33 Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.121 Safari/535.2
2011-12-13 03:53:36 Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.2 (KHTML, like Gecko) Ubuntu/11.10 Chromium/15.0.874.106 Chrome/15.0.874.106 Safari/535.2
2011-12-13 10:08:06 Mozilla/5.0 (Ubuntu; X11; Linux i686; rv:8.0) Gecko/20100101 Firefox/8.0
2011-12-13 14:36:40 Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.121 Safari/535.2
2011-12-13 20:31:59 Opera/9.80 (Windows NT 6.0; U; ru) Presto/2.10.229 Version/11.60
2011-12-14 13:57:37 Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.1 (KHTML, like Gecko) Ubuntu/11.10 Chromium/14.0.835.202 Chrome/14.0.835.202 Safari/535.1
2011-12-14 22:21:43 Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.2 (KHTML, like Gecko) Ubuntu/11.10 Chromium/15.0.874.106 Chrome/15.0.874.106 Safari/535.2
2011-12-15 07:04:05 Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.63 Safari/535.7
2011-12-15 08:04:40 Mozilla/5.0 (Ubuntu; X11; Linux i686; rv:8.0) Gecko/20100101 Firefox/8.0
2011-12-15 09:29:25 Mozilla/5.0 (X11; Linux i686) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.63 Safari/535.7
2011-12-15 11:54:50 Mozilla/5.0 (X11; Linux i686; rv:7.0.1) Gecko/20100101 Firefox/7.0.1
2011-12-17 00:24:35 Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0)
2011-12-17 07:02:43 Mozilla/5.0 (Ubuntu; X11; Linux x86_64; rv:8.0) Gecko/20100101 Firefox/8.0
2011-12-17 11:25:29 Mozilla/5.0 (Ubuntu; X11; Linux x86_64; rv:8.0) Gecko/20100101 Firefox/8.0
2011-12-18 17:10:57 Mozilla/5.0 (X11; Linux x86_64; rv:8.0) Gecko/20100101 Firefox/8.0
2011-12-19 00:19:15 Mozilla/5.0 (Ubuntu; X11; Linux i686; rv:8.0) Gecko/20100101 Firefox/8.0
2011-12-19 10:17:52 Opera/9.80 (X11; Linux i686; U; en) Presto/2.10.229 Version/11.60
2011-12-19 12:28:32 Mozilla/5.0 (Ubuntu; X11; Linux x86_64; rv:8.0) Gecko/20100101 Firefox/8.0
2011-12-19 17:00:01 Mozilla/5.0 (Ubuntu; X11; Linux i686; rv:8.0) Gecko/20100101 Firefox/8.0
2011-12-20 06:05:15 Mozilla/5.0 (X11; Linux i686; rv:7.0.1) Gecko/20100101 Firefox/7.0.1
Status: NEW → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → WORKSFORME
(Reporter)

Comment 3

6 years ago
(In reply to Ricky Rosario [:rrosario, :r1cky] from comment #2)
> I didn't find anything suspicious other than your webtrends numbers ;).
> Don't forget that users can end up on that page in any locale (for example,
> https://support.mozilla.com/es/questions/901985) and I am not sure if
> webtrends is aggregating that, probably not. So either webtrends is way off
> or you are missing visits to the page in other locales.

The locales were accounted for, if I'm reading that correctly. The search was only for "questions/901985", so it should cover all locales.

Ricky, is there a possibility of the votes coming from spiders and Webtrends filtering out spiders? We need to figure out the root cause for this, since it means that we can either not trust our Webtrends data or our votes data. 

Cheng, Ibai, do you have any idea, how get the raw data for Webtrends?
(Reporter)

Updated

6 years ago
Status: RESOLVED → REOPENED
Resolution: WORKSFORME → ---
(In reply to Kadir Topal [:atopal] from comment #3)
> Ricky, is there a possibility of the votes coming from spiders and Webtrends
> filtering out spiders?

I posted all the user agent strings. Do you see any spiders? :-)
(In reply to Kadir Topal [:atopal] from comment #3)
> The locales were accounted for, if I'm reading that correctly. The search
> was only for "questions/901985", so it should cover all locales.

Are you sure about that? Also https vs http? I am not sure how webtrends aggregates the visits. Maybe I can get access to play with it a little?

Comment 6

6 years ago
Yeah...spiders don't vote...AFAIK.

Cheng should be able to get raw data about the visits/pageviews from our servers. That's more trustworthy than Webtrends. In this case, Webtrends' metric is Visits that could reflect several pageviews...that's a potential source of abuse: A visitor voting several times. Ricky's data doesn't seem to group much (i.e. not many datapoints share all the same info).

So I don't know.
We set a cookie so that anonymous users can't vote multiple times. Sure they can delete the cookie, but WHY?

Comment 8

6 years ago
A user may think that over-voting a thread will give him better chances to get an answer. It's reasonable. But, if I user is capable of finding the cookie... isn't just better to post the question again and again? (you need multiple addresses..but it's obviously more noisy).

My take is that we are not fully understanding the meaning of the webtrends metric or it's flawed. Aside from that, we may have an issue of how people uses the "me too"...but that's not related to this bug IMHO.
(In reply to Kadir Topal [:atopal] from comment #3)
> The locales were accounted for, if I'm reading that correctly. The search
> was only for "questions/901985", so it should cover all locales.

Does this search actually include all traffic? I thought we could only look at the top 2k URLs, which would exclude many questions, anyway, and certain fragmented traffic over several locales to a given question.


(In reply to Ibai from comment #8)
> (you need
> multiple addresses..but it's obviously more noisy).

There's no duplicate question detection. I.e. you could use the same account and ask the same question 4 times.

Comment 10

6 years ago
Yeah, but we manually close them when we find a user asking the same question over and over.
My work is done here.
Assignee: rrosario → nobody
Target Milestone: 2011Q4 → ---

Comment 12

6 years ago
Pulled access log data, 467 hits since the question was created.  Webtrends is on drugs.
(Reporter)

Comment 13

6 years ago
Cww, was that for a specific language version of the URL or to en-US only? Does Webtrends count URLs with different parameters as different URLs? We add parameters to article URLs opened through search AFAIK.

467 hits could mean that Webtrends is off by an order of magnitude. Hmm...
I suspect this is resolved by now with the changes we made to the cron job that updates votes. Anything left to do?
(Reporter)

Comment 15

4 years ago
We don't even have webtrends anymore, and with the latest "view" rates we know that even very obscure threads get a lot of views.
Status: REOPENED → RESOLVED
Last Resolved: 6 years ago4 years ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.