623348 - Keep a fixed number of tweets per locale

Status: VERIFIED FIXED

(support.mozilla.org :: Army of Awesome, task)

Description

[Paul Craciunoiu [:paulc]]

Right now we're only including tweets from users whose language is English.

AoA pages in other languages probably want to see at least tweets in their language.

Kadir and William will be discussing what tweets should localizers see.

As James pointed out, we need to reconsider how we limit the tweets after this, since right now we're only storing 500 tweets, and en-US will dominate that.

Comment 1

[William Reynolds [:williamr]]

Would it be possible to store the 500 most recent English tweets and 100 most recent tweets for each locale we're supporting? For example: we'd store 500 en-US, 100 de, 100 fr, etc.

Comment 2

[Paul Craciunoiu [:paulc]]

Some reworking will be needed but in principle, yeah, it would be.

Comment 3

[William Reynolds [:williamr]]

We'd like to just show tweets from that locale. If someone wants to see English tweets, they can view the en-US/army-of-awesome page.

Can we store the 500 most recent tweets for each locale?

Comment 4

[Paul Craciunoiu [:paulc]]

(In reply to comment #3)
> Can we store the 500 most recent tweets for each locale?
Well, one constant is simpler than 2, and we can revisit if there will be a need to restrict to 100 for non-English tweets.

I do expect that most locales won't have much, but e.g. German, French might be fairly active.

Comment 5

[Paul Craciunoiu [:paulc]]

There's a fallback message that needs tweaking, for which bug 627831 has been filed. This commit really doesn't do what the bug summary says, so I'm changing it.

https://github.com/jsocol/kitsune/commit/886c7a4ff675e38af78e72deab51cacd57964618

The locale-specific tweet list was taken care of in bug 623341.

QAing this is not that easy, but if there are enough, say, German or French tweets, then it's proof that the cleanup works differently from before (when it would just keep the newest 500 regardless of locale, overwhelmingly English).

Comment 6

[James Socol [:jsocol, :james]]

Did you file a bug for adding the purge_tweet cron to allizom?

Comment 7

[Paul Craciunoiu [:paulc]]

(In reply to comment #6)
> Did you file a bug for adding the purge_tweet cron to allizom?
I did now!

Comment 8

[Rebecca Billings [:rbillings]]

It looks like we're only getting tweets from 8 hours ago, no scrolling is in effect on any other locale ex:
https://support-release.allizom.org/fr/army-of-awesome

The tweets are coming in by the hour so I expect there should be enough data to scroll.

Comment 9

[James Socol [:jsocol, :james]]

In non-English locales, it will take a while to build up to the maximum number of tweets we store. French has tweets going back about 21 hours now, and there is a second "page".

At 17:25-8 it was around 2 am in France. Give them time ;)

Comment 10

[James Socol [:jsocol, :james]]

Japan has been awake longer, and has a bunch of data already:

https://support-release.allizom.org/ja/army-of-awesome

Comment 11

[Vito Smolej]

Good stuff, guys, makes me feel good.

Comment 12

[Rebecca Billings [:rbillings]]

Verified many more tweets in non-US locales. I don't think any of them will exceed 500 prior to the release, but will keep an eye on it- it's definitely more than 100.