Created attachment 753418 [details] .png - mixed languages in reviews Discovered on FxOS by participant in usability testing. App details page show reviews in different languages. Region is set to "United States", user is not signed in. Observed behavior: When viewing reviews on an app details page (Twitter), there are reviews in English, Spanish, and French. Expected behavior: The Marketplace only shows me reviews in my language.
The thing is how do we know which language the review was written in? I might have my language set as French, but I'm going to write a review as English because the app is an English app. We would have to do some magic like Google Translate does to detect which language the text is most likely in.
Do we do anything now, like guess that someone who set Marketplace to French will only see reviews from others with the same setting? This might need to be related to stores, so Twitter reviews in the French store are different than Twitter reviews in the US store?
So the thing is French language != French region. The only thing we do is if you are in *France* (and that region is a "mature region") we show reviews only from users writing those reviews in France. (See bug 763699.)
Our current solution in comment 3 is to show reviews only added in that region for mature regions. Many regions are multi-lingual and I don't think that's a problem for now. At some point you raise questions of what language "ur gr8" is in, I suppose. Anyway, if you have UX recommendations, we're happy to hear them. :) In the meantime, I'm going to wontfix.
Status: NEW → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → WONTFIX
It sounds like we store language with reviews now. There will be the occasional broken case here (eg. I'm viewing the pages in French but writing in English) but I think the percentage of reviews falling into that scenario will be minimal. This is worth doing, even if we only get it 90% right, imo.
Status: RESOLVED → REOPENED
Resolution: WONTFIX → ---
Whiteboard: [see comment 5]
Showing reviews only in your language surface a bunch of issues a) The app may have 100 reviews in English but only 2 in the user's locale. What review count should we show? b) Even though locale is more important, region may be important too. Yelp may have good content and thus higher reviews in the US, but the reviews for Spain may be poor. What do we show in that case?
Whiteboard: [see comment 5] → [see comment 5][uxwanted]
Whiteboard: [see comment 5][uxwanted] → [see comment 5]
I think we should show all reviews, but prioritize the order. If we can tell region for the review, I would suggest the following prioritized order: 1. Same Region->same locale of user->latest date first 2. Other region->locale of user->latest date first (all regions that match the locale in date order) 3. Region->other locales->latest date first (other reviews in the region but in a different language than the user) 4. Other region->other locales->latest date first (non matching regions and non-matching languages in date order)
As Wil said in Comment #5, we can just do languages for now.
The solution to this seems to be to always show the total number of reviews, but to filter by default to only show the reviews in the users language. Because regions are mostly about laws and languages are about people, filtering by regions doesn't make a lot of sense to solve this problem. People can be in a country where they don't speak the language, those geo-political borders don't play a part in this problem. UI: We can give them an option (a "all reviews" "only reviews in my language" segmented control, see: http://pwalm.github.io/marketplace-style-guides/buttons.html#controls for Marketplace Style segmented control) to filter results and this control could live below the current "Write a review" button.
Whiteboard: [see comment 5] → [see comment 5][comms-needed]
I think this is something we really ought to do. With most reviews in non-English languages, the reviews aren't quite useful for English speakers. (And, I'm sure the same happens for other languages.) Can product set priority here please?
Whiteboard: [see comment 5][comms-needed] → [see comment 5][cvanwilltreat]
I agree that this should be high priority and is a p2. I think Tony's suggestion in Comment 10 makes sense. I think there are 2 areas we need to consider, both in a signed in or not signed in condition. 1. In the app details page, below the description area, we have the first few reviews. This may not be a good place for a dropdown - so need ui help here. 2. On the ratings page. One point of clarification: I would think that if the person selects something (like 'all reviews') it should persist for the session. In that way, moving from the app details page to the ratings page would persist the current preference until changed. I think this remains a P2 - but can be put on repoman possibly.
Whiteboard: [see comment 5][cvanwilltreat] → [see comment 5][cvanwilltreat] [repoman][see comment 10]
Whiteboard: [see comment 5][cvanwilltreat] [repoman][see comment 10] → [see comment 5][cvanwilltreat][see comment 10]
Whiteboard: [see comment 5][cvanwilltreat][see comment 10] → [see comment 5][cvanwillbuyyouadrink][see comment 10]
Whiteboard: [see comment 5][cvanwillbuyyouadrink][see comment 10] → [cvanwillbuyyouadrink][see comment 10]
Whiteboard: [cvanwillbuyyouadrink][see comment 10] → [cvanwillbuyyouadrink][see comment 10][userman]
Closed PRs since this got hung up on desktop UI changes. https://github.com/mozilla/zamboni/pull/2818 https://github.com/mozilla/fireplace/pull/878
Whiteboard: [cvanwillbuyyouadrink][see comment 10][userman] → [cvanwillbuyyouadrink][see comment 10][userman][ktlo]
I ran the language classifier https://github.com/saffsd/langid.py over our existing reviews. Main things I learned: * The shorter a review is, the harder time it has guessing the language correctly. Adding some key words to check short reviews for will probably improve that a good bit ("buena", "malo", "good", "bad", "very", "muy", "exelente"). Or perhaps we decide that 1-3 word reviews are less important. Here's a scatter plot of review size vs language-id confidence: http://people.mozilla.org/~ashort/875455/scatter.png * Half of reviews are pretty short, under 10 words. http://people.mozilla.org/~ashort/875455/wordcount.png * As-is, accuracy is fairly good. 53% of reviews are above the 0.99 confidence level, and 83% of reviews longer than 3 words are above the 0.99 confidence level: http://people.mozilla.org/~ashort/875455/confidence.png Eyeballing the results didn't reveal any at that level that were misclassified, so I feel pretty good about trusting its scoring. langid also supports restricting the languages to check for, which will probably improve accuracy. I think using langid to classify comments, plus simple common-word filtering for low-confidence cases, is a worthwhile idea.
Added language guessing for comments, and API for filtering reviews by language: https://github.com/mozilla/zamboni/commit/b51e9d5d05243f46f35a8145a65dd79769c01d4b
This is close to ready in case the consumer pages wanted to use this information somehow.
Status: REOPENED → RESOLVED
Last Resolved: 5 years ago → 2 years ago
Resolution: --- → WONTFIX
Whiteboard: [cvanwillbuyyouadrink][see comment 10][userman][ktlo] → [cvanwillbuyyouadrink][see comment 10][userman][ktlo][possible_future_need]
You need to log in before you can comment on or make changes to this bug.