Status

support.mozilla.org
Army of Awesome
RESOLVED FIXED
7 years ago
7 years ago

People

(Reporter: atopal, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

7 years ago
Based on this account https://support.mozilla.com/en-US/forums/contributors/704891
it looks like the listing of top contributors doesn't measure what it says it does. Could you please check what happened to smo's contributions?

Comment 1

7 years ago
follow my traffic under awsmo and let me know, if you need any help.

regards

smo
Fred: is this data from metrics or us?
Metrics!

CCing Pedro, who can probably help.

Comment 4

7 years ago
Nelson worked on this, cc'ing him

Comment 5

7 years ago
We'll investigate it as soon as possible and try to determine what happened.

Comment 6

7 years ago
I'm terribly sorry about this situation, but unfortunately it seems to be twitter's fault. I don't know why it happens, but the fact is that searching for #fxhelp (http://twitter.com/#search?q=%23fxhelp) no tweets show up from awsmo. I looked up tweets starting at noon on Saturday January 1st, there are a lot of tweets, but none from your username although they show up on your account. I'm sure there are many other users in the same situation, as almost all tweets are from the same 4 or 5 users, and the army of volunteers is much bigger than that.

The API we use returns the same results as a twitter search (I don't know any other way of fetching the data), so we have no way (for the time being) of sorting this out. We'll look into alternatives.

Comment 7

7 years ago
I found a thread that discussed the missing tweets problem of the Twitter API: http://code.google.com/p/twitter-api/issues/detail?id=1154. The thread started over 1 year ago, but the latest post, dated December 16th, mentions it started again and the tweets from awsmo are missing from December 18th onwards, so it could be related.
Have we checked the elastic search datastore to see if it is affected in the same way?  That API is reading twitter data from the "firehose" Twitter Streaming API so it might not have the same problem.
Okay, I just checked with the query below and the problem shows up there too.  This indicates a significant data problem on twitter's end and it is likely something we won't be able to recover from.  The ElasticSearch engine is continuously listening to Twitter's data stream for Firefox related tweets.  Those tweets were never broadcast through the API for us to store.

I think we are going to have to look at wording the Army website to let people know of this sort of possibility.  We can also look into ways that we might be able to detect it and proactively notify people, but I'm not sure what that solution would look like.


http://cm-metricsetl03.mozilla.org:9200/twitter_river/status/_search?size=0
{
    "query": {
        "constant_score": {
            "filter": {
                "and": {
                    "filters" : [
                        { "term": { "hashtag":"fxhelp" } },
                        { "term": { "user.screen_name":"awsmo" } },
                        { "range": { "created_at": { "from": "2010-12-01", "to": "2011-01-04" } } }
                    ]
                }
            }
        }
    },
    "facets": {
        "tweets_per_day": {
            "histogram": {
                "key_script": "doc['created_at'].date.year * 10000 + doc['created_at'].date.monthOfYear * 100 + doc['created_at'].date.dayOfMonth",
                "value_script": "1"
            }
        }
    }
}

And it returns these results:
{
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 165,
    "max_score" : 1.0,
    "hits" : [ ]
  },
  "facets" : {
    "tweets_per_day" : {
      "_type" : "histogram",
      "_key_field" : "_na",
      "_value_field" : "_na",
      "_comparator" : "key",
      "_interval" : -1,
      "entries" : [ {
        "key" : 20101210,
        "count" : 3,
        "total" : 3.0,
        "mean" : 1.0
      }, {
        "key" : 20101211,
        "count" : 14,
        "total" : 14.0,
        "mean" : 1.0
      }, {
        "key" : 20101212,
        "count" : 34,
        "total" : 34.0,
        "mean" : 1.0
      }, {
        "key" : 20101215,
        "count" : 22,
        "total" : 22.0,
        "mean" : 1.0
      }, {
        "key" : 20101216,
        "count" : 24,
        "total" : 24.0,
        "mean" : 1.0
      }, {
        "key" : 20101217,
        "count" : 24,
        "total" : 24.0,
        "mean" : 1.0
      }, {
        "key" : 20101218,
        "count" : 44,
        "total" : 44.0,
        "mean" : 1.0
      } ]
    }
  }
}
Do we know if this problem is still occurring or has been resolved? Seems like this occurs periodically and would only be an issue for a short period of time.

Comment 11

7 years ago
It's still occurring. I queried twitter earlier today (around 3 or 4 pm GMT) and no tweets from awsmo were visible on a search by #fxhelp, although there were plenty listed in his account.

After the problem is solved on the twitter end of things we'll be able to capture new tweets correctly but recovering old tweets isn't possible (the API doesn't return old results). So overall totals will always reflect that problem unfortunately.

Comment 12

7 years ago
Another piece of information: http://search.twitter.com/search?q=from:awsmo doesn't return any results.
Will you try to contact twitter about it? If they deliberately filter our tweets, I'd like to know why that's the case, and if they don't then they probably want to know about the bug.

Comment 14

7 years ago
They also "deliberately" filter oprah's twittes :)


We can try to reach them
Do we have any inside contact information?  I don't think a simple e-mail would really help much, and the issue linked in comment #7 is probably the only official contact mechanism they'd endorse using.

Comment 16

7 years ago
Vito: to try to isolate possible causes could you check if your tweets aren't marked as private or something like that?

You should probably contact twitter support to complain about this situation. You just don't show up on search and in that situation there's nothing we can do.

Comment 17

7 years ago
Any news on this issue? "awsmo" and "from:awsmo" still return 0 results on twitter searches.
http://twitter.com/#!/awsmo seems to load tweets that indeed use #fxhelp though.

Metrics, what are the next steps here?
I remember someone mentioning that we have a contact at twitter that we might ask, but I don't think there was any follow up.  We are at a dead-end on our end.  The APIs just don't return his data even though it is visible on the website.  Without a reply from Twitter on the forum issue we opened or internal assistance from them, there isn't much we can do further.

Comment 20

7 years ago
awsmo counts may even be correct - I have stopped doing it;) So my counts are maybe representative and non-saying at the same time.

Comment 21

7 years ago
we replaced the backend. I assume that, issues with twitter itself apart , shpuld be ok now.

Check the link from the main metrics site
Status: NEW → RESOLVED
Last Resolved: 7 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.