Closed Bug 1039823 Opened 10 years ago Closed 10 years ago

Filters for Army of Awesome should not suggest tweets that simply mention a @username that contains "firefox" in it

Categories

(support.mozilla.org :: Army of Awesome, task, P3)

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: hi, Assigned: rehandalal+mozilla)

References

()

Details

(Whiteboard: u=contributor c=AoA p=1 s=2014.14)

Attachments

(1 file)

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:30.0) Gecko/20100101 Firefox/30.0 (Beta/Release)
Build ID: 20140605174243

Steps to reproduce:

Visit https://support.mozilla.org/en-US/army-of-awesome
Look down the list of Tweets


Actual results:

Witness several unrelated tweets, such as tweets to user @firefox_53


Expected results:

The filter which finds the tweets should be more specific, searching for tweets only containing #firefox @firefox or firefox (ensuring no other characters are attached)
Severity: normal → minor
(In reply to Chip from comment #0)
> User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:30.0) Gecko/20100101
> Firefox/30.0 (Beta/Release)
> Build ID: 20140605174243
> 
> Steps to reproduce:
> 
> Visit https://support.mozilla.org/en-US/army-of-awesome
> Look down the list of Tweets
> 
> 
> Actual results:
> 
> Witness several unrelated tweets, such as tweets to user @firefox_53
> 
> 
> Expected results:
> 
> The filter which finds the tweets should be more specific, searching for
> tweets only containing #firefox @firefox or firefox (ensuring no other
> characters are attached)

firefox_53 has now been blacklisted as per request in #sumo on MozNet

<pmcclard> Thanks c. I added it to the blacklist. It won't take affect right away, so give it some time.
Component: Support → Army of Awesome
Product: Webmaker → support.mozilla.org
Assignee: nobody → pmcclard
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true
I'm going to close this bug out as fixed. If you have more feedback, feel free to drop it here: 
https://support.mozilla.org/en-US/forums/contributors/710495
https://support.mozilla.org/en-US/forums/contributors/710479
Status: ASSIGNED → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
(In reply to Andrew Truong [:feer56] from comment #2)
> I'm going to close this bug out as fixed. If you have more feedback, feel
> free to drop it here: 
> https://support.mozilla.org/en-US/forums/contributors/710495
> https://support.mozilla.org/en-US/forums/contributors/710479

What's the point of closing the bug, it's not fixed. Tweet filtering is not yet correct.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Bug are normally used to fix/deploy actual items. You are reporting a request to fix an issue, that's already being done in the threads mentioned. Until an actual solution is brought up there is no use to file a bug because there is no clue on what is to be done just yet. Keeping things in one place, makes it easier and accessible for the group to make decisions. 

See also: https://support.mozilla.org/en-US/forums/contributors/710495?last=60986

You also mentioned:
"<pmcclard> Thanks c. I added it to the blacklist. It won't take affect right away, so give it some time."

Hasn't your issue been resolved from that?
> Until an actual solution is brought up

One was, in comment 0 (and duplicate bug 1041771):
> The filter which finds the tweets should be more specific, searching for
> tweets only containing #firefox @firefox or firefox (ensuring no other
> characters are attached)

Until the filter has been made more intelligent along those lines, the issue here hasn't been fixed.
IMHO we really just need this filter-tweak when we're matching "firefox" in a twitter-handle inside of a tweet -- not necessarily when we've found it in a hashtags or elsewhere in the body of a message. I can imagine people using creative hashtags like #stopcrashingfirefox or #firefoxhatesme, or in message body like "firefox31 isn't working". Tweets with those strings (which include firefox and some characters appended) may merit a response, and it's probably worth suggesting them.

However, it's *not* worth suggesting a response to every tweet that mentions a username like "@firefox_53" or "@happy_firefox_user" (made up) or @firefoxsuckslol (made up).  Right now, it seems that Army of Awesome is accidentally designed in such a way to encourage spamming anyone with such a username (until we add them to our blacklist), which is kind of horrible.  We should improve the filtering so that this doesn't happen.
You're right Daniel, we're working on updating the filter options that we already have. We can ignore twitter handles, blacklist certain words and ban AoA members who aren't getting the coaching our mods are offering. 

Here's the etherpad we're using to track ignored names. https://sumo.etherpad.mozilla.org/twitterignorelist?

You can also add blacklist words at the bottom if there's any that you think we should avoid.
Thanks, Patrick.

[/me clarifies bug summary to make it less generic, & more about the expected results actually described in comment 0, modulo my clarifications in comment 8]
Summary: Filters for Army of Awesome are picking up unrelated tweets → Filters for Army of Awesome should not suggest tweets that simply mention a @username that contains "firefox" in it
Blocks: 1043431
We should still be showing tweets from @FirefoxBrasil and @firefox, but not anything else with firefox in the handle
Whiteboard: u=contributor c=AoA p= s=2014.14
(In reply to Kadir Topal [:atopal] from comment #12)
> We should still be showing tweets from @FirefoxBrasil and @firefox, but not
> anything else with firefox in the handle

Why should we be showing tweets from the Firefox accounts? They don't need support
I think atopal meant "tweets that mention", not "tweets from".

(There are probably a few other handles we should look for as well; but we definitely don't need to have a complete list up-front.)
(In reply to Daniel Holbert [:dholbert] from comment #14)
> I think atopal meant "tweets that mention", not "tweets from".
> 
> (There are probably a few other handles we should look for as well; but we
> definitely don't need to have a complete list up-front.)

Maybe localisation is a good idea. Depending on which version of the AoA page you visit, you see the appropriate language of tweets with both @firefox and the twitter handle of the respective language.

Another option is filtering, options to filter the tweets for certain languages. If you're multilingual, you should be able to choose multiple languages.
The tweets of users that mention @firefox usually (maybe not always) get a response from the @firefox account (which is unrelated to AoA). There have been cases that both the @firefox account and AoA replied to the same user with the risk of becoming a nuisance. Either tweets mentioning @firefox should be filtered out or there should be some sort of coordination between @firefox and AoA regarding who's replying to what.
Whiteboard: u=contributor c=AoA p= s=2014.14 → u=contributor c=AoA p=1 s=2014.14
Assignee: pmcclard → nobody
Assignee: nobody → rdalal
Priority: -- → P3
Landed:
https://github.com/mozilla/kitsune/commit/e52f78953956139a309ec9f29a6a7d25c7e71716

Pushed to prod by me, right now.
Status: REOPENED → RESOLVED
Closed: 10 years ago10 years ago
Resolution: --- → FIXED
(In reply to Chip from comment #13)
> (In reply to Kadir Topal [:atopal] from comment #12)
> > We should still be showing tweets from @FirefoxBrasil and @firefox, but not
> > anything else with firefox in the handle
> 
> Why should we be showing tweets from the Firefox accounts? They don't need
> support

As Daniel said, I meant tweets mentioning @FirefoxBrasil and @firefox not tweets from those accounts. Sorry for the confusion. 

Rehan can you confirm we are showing mentions of those accounts but not tweets posted by those accounts?
Flags: needinfo?(rdalal)
(In reply to CAK from comment #16)
> The tweets of users that mention @firefox usually (maybe not always) get a
> response from the @firefox account (which is unrelated to AoA). There have
> been cases that both the @firefox account and AoA replied to the same user
> with the risk of becoming a nuisance. Either tweets mentioning @firefox
> should be filtered out or there should be some sort of coordination between
> @firefox and AoA regarding who's replying to what.

That's a good point, CAK. 

Madalina, can we ask that the people running the @Firefox account use AoA to respond to support issues?
Flags: needinfo?(mana)
I'm sorry guys but these tweets have shown up on the queue.
Kadir: as discussed yesterday on Vidyo I implemented it so that we ignore tweets from all users with "firefox" in their handle except those on the whitelist (@Firefox, @FirefoxBrasil, @Firefox_ES).

What i'm getting from your comment is that you actually want me to filter out *ALL* tweets from users with "firefox" in their handle and only show tweets that mention those accounts on the whitelist. I can make this change, just confirm that this is correct.

Also to follow up on CAK's attachment, this is related to the way that we've always filtered out mentions. Changes to this was not within the original scope for this bug (as per comment #12) but Kadir I can change how we filter out mentions because I don't think our current implementation works well.
Flags: needinfo?(rdalal)
(In reply to Rehan Dalal [:rehan, :rdalal] from comment #21)
> Kadir: as discussed yesterday on Vidyo I implemented it so that we ignore
> tweets from all users with "firefox" in their handle except those on the
> whitelist (@Firefox, @FirefoxBrasil, @Firefox_ES).
[...]
> Also to follow up on CAK's attachment, this is related to the way that we've
> always filtered out mentions. Changes to this was not within the original
> scope for this bug

No, that was exactly the original scope of this bug... See comment 0, and comment 8, and the summary of this bug (Army of Awesome should not suggest tweets that simply mention a @username that contains "firefox" in it").

This bug is about tweets that *mention* usernames with "firefox" in them (and get mistakenly picked up by AoA).  It's not about tweets *from* usernames with Firefox in them.

If you ended up fixing something related to tweets *from* usernames with Firefox in them, and want to retarget this bug to be just about that, though, that's fine -- but in that case, could you update the bug-summary to reflect what actually changed here, and re-open e.g. duplicate-bug 1041771 to address what we were originally getting at here?
(In reply to Rehan Dalal [:rehan, :rdalal] from comment #21)
> Kadir: as discussed yesterday on Vidyo I implemented it so that we ignore
> tweets from all users with "firefox" in their handle except those on the
> whitelist (@Firefox, @FirefoxBrasil, @Firefox_ES).
> 
> What i'm getting from your comment is that you actually want me to filter
> out *ALL* tweets from users with "firefox" in their handle and only show
> tweets that mention those accounts on the whitelist. I can make this change,
> just confirm that this is correct.

Yes, indeed. Sorry for the confusion, my message in comment #12 should have said "to" not "from".
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Landed:
https://github.com/mozilla/kitsune/commit/2b2d82235c8d0e7994d734d598e5b21330756317

Pushed to prod on Friday by willkg.
Status: REOPENED → RESOLVED
Closed: 10 years ago10 years ago
Resolution: --- → FIXED
Thanks!

One note on the tests included in the push -- it looks like it checks that "Hey @someone" is rejected, whereas "Hey @firefox" and "Hey @FirefoxBrasil" are accepted.

It's probably worth including testcases closer to the original scenario here -- mentions of usernames that have "firefox" in them, which are *not* not official Firefox channels.  Right now, it looks like that's un-tested.

So, e.g. adding tests like "Hey @FirefoxInMyUsername" and "Hey @NotReallyFirefox"
Flags: needinfo?(mana)
[needinfo=rdalal to see if we can include a test for the main issue here, per comment 25]
Flags: needinfo?(rdalal)
Flags: needinfo?(rdalal)
(RE needinfo-canceling -- does the test actually test this bug now, per comment 25? If not, please don't just silently cancel the needinfo -- it's totally understandable if you don't have cycles to extend the test right now, but in that case, maybe file a followup that someone can take down the line?  Better to have it tracked somewhere (and pending) than to lose track of the fact that it needs doing.)
Flags: needinfo?(rdalal)
Flags: in-testsuite?
Sorry Daniel, I meant to post a quick follow up. 

If you look at that commit we've actually removed the text of the tweet from the tests completely because we do not look at the username when checking for mentions, we have a hard-coded list of allowed user id's which we look at and the tests as they stand ensure that all the cases are covered.

That is why I cancelled the needsinfo. Apologies for not posting that explanation when I cancelled the needsinfo.
Flags: needinfo?(rdalal)
(In reply to Rehan Dalal [:rehan, :rdalal] from comment #28)
> Sorry Daniel, I meant to post a quick follow up.

Sure -- no worries, thanks for responding.

> If you look at that commit we've actually removed the text of the tweet from
> the tests completely

Right, but that's beside my point. What I was asking was, do we have a test for the original issue here? (a test checking that we do *not* promote tweets like "Hey @NotReallyFirefox")

It looks like we still do not, and I think we need one, or else this could easily regress down the line when we rewrite/modify this code.
As I mentioned in my last comment we have changed the way that we check for mentions. We no longer look at the text of the tweet, we use the meta data provided by the new Twitter API to check for user ID's of those mentioned. I don't see any reason why we would regress to checking the text of the tweet, so there are no tests that do so, because that would just be a redundant test.
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: