1058257 - Add in all the countries

Status: VERIFIED FIXED

(Marketplace Graveyard :: Developer Pages, defect, P3)

Description

[Andy McKay]

I'm working on an app that only works in certain countries. Namely Canada and US because that's where the data is available. However the Marketplace only lists some countries.

Why not list all the countries? Is there any downside to that? The upside is I can limit restriction to the countries for my app.

Comment 1

[Andy McKay]

Any reason for uiwanted here? It just makes the forms on devhub bigger.

Comment 2

[Wil Clouser [:clouserw]]

You asked if there were downsides.  I figured UX would have the best input.

Comment 3

[Wil Clouser [:clouserw]]

"I don't think there are any downsides for this."  -- uiwanted triage

Comment 4

[Mathieu Pillard [:mat]]

Some thoughts after discussing it on IRC:
- With android and desktop, we can't limit by Firefox OS regions anymore, we should be global

- Adding all the regions is going to be a problem in consumer pages, because of the increase in regions.js size, and because of the various flags we'd have to include in the true packaged app if we ever do it - not to mention the size of all the translations for all the countries names

- One solution could be to stop displaying the country name+flag in the footer and in the newsletter form (and just rely on autodetected country for the newsletter)

There is still the question of whether we should display a huge list of countries in developer tools, reviewer tools, curation tools, statistics.

Comment 5

[Andy McKay]

I don't see whats wrong with a list of countries that includes all the countries and doesn't exclude the majority of the worlds countries like we do now. Some people might take it personally.

I'd be all up for just removing all the flags in the consumer pages. I don't know how you can avoid an increase in regions.js.

Comment 6

[Mathieu Pillard [:mat]]

If we stop displaying the region (translated) name everywhere in consumer pages (footer, newsletter, account settings), we can just generate a region slugs list, as well as a mcc->slug table for fireplace and we don't need anything else - there would still be an increase in size, but not as significant - what would make region.js and the lang js files big is the slug -> full name and translations for 250 countries.

Or we can take make our js files bigger and take the perf hit.

Comment 7

[Andy McKay]

https://github.com/mozilla/marketplace-constants/pull/21

Comment 8

[Wil Clouser [:clouserw]]

Below is the countries and rating authorities:

Australian Classification Branch (ACB)
Australia

Classificação Indicativa (CLASSIND)
Brazil

Entertainment Software Rating Board (ESRB)
Antigua and Barbuda, Argentina, Bahamas, Barbados, Belize, Bolivia, Canada, Chile, Colombia, Costa Rica, Cuba, Dominica, Dominican Republic, Ecuador, El Salvador, Greenland, Grenada, Guatemala,  Guyana, Haiti, Honduras, Jamaica, Mexico, Nicaragua, Panama, Paraguay, Peru, Saint Kitts and Nevis, Saint Lucia, Saint Vincent and the Grenadines, Suriname, Trinidad and Tobago, United States of America, Uruguay, and Venezuela

Pan European Game Information (PEGI)
Albania, Andorra, Austria-Denmark, Belgium, Bosnia and Herzegovina, Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, France, Greece, Hungary, Iceland, Ireland, Israel, Italy, Kosovo, Latvia, Liechtenstein, Lithuania, Luxembourg, Macedonia, Malta, Moldova, Monaco, Montenegro, Netherlands, Norway, Poland, Portugal, Romania, San Marino, Serbia, Slovak Republic, Slovenia, Spain, Sweden, Switzerland, andTurkey, United Arab Emirates, United Kingdom, and Vatican City

Unterhaltungssoftware Selbstkontrolle (USK)
Germany

Generic
Afghanistan, Albania, Algeria, Andorra, Angola, Armenia, Azerbaijan, Bahrain, Bangladesh, Belarus, Benin, Bhutan, Bosnia and Herzegovina, Botswana, Brunei, Burkina Faso, Burundi, Cambodia, Cameroon, Cape Verde, Central African Republic, Chad, China, Comoros, Democratic Republic of the Congo, Republic of the Congo, Cote d'Ivoire, Croatia, Denmark, Djibouti, East Timor, Egypt, Equatorial Guinea, Eritrea, Ethiopia, Federated States of Micronesia, Fiji, Gabon, The Gambia, Georgia, Ghana, Guinea, Guinea-Bissau, India, Indonesia, Iran, Iraq, Japan, Jordan, Kazakhstan, Kenya, Kiribati, Kosovo, Kuwait, Kyrgyzstan, Laos, Lebanon, Lesotho, Liberia, Libya, Liechtenstein, Macedonia, Madagascar, Malawi, Malaysia, Maldives, Mali, Marshall Islands, Mauritania, Mauritius, Moldova, Monaco, Mongolia, Montenegro, Morocco, Mozambique, Myanmar, Namibia, Nauru, Nepal, New Zealand, Niger, Nigeria, North Korea, Oman, Pakistan, Palau, Papua New Guinea, Philippines, Qatar, Russia, Rwanda, San Marino, Sao Tome and Principe, Saudi Arabia, Senegal, Serbia, Seychelles, Sierra Leone, Singapore, Solomon Islands, Somalia, Samoa, South Africa, South Korea, South Sudan, Sri Lanka, Sudan, Swaziland, Syria, Taiwan, Tajikistan,  Tanzania, Thailand, Togo, Tonga, Tunisia, Turkey, Turkmenistan, Tuvalu, Uganda, Ukraine, United Arab Emirates, Uzbekistan, Vanuatu, Vatican City, Vietnam, Yemen, Zambia, and Zimbabwe

Comment 9

[Wil Clouser [:clouserw]]

The PR in comment 7 was closed, but I don't think it should have been.  I'll nominate this for KTLO because it comes up, what, once a week?  I don't have a good idea of how much work it would be to do


(someone is currently looking for a list of all the regions we support.  Sure would be easy just to say "all of them"!)

Comment 10

[Caitlin Galimidi [:c8o]]

Hey there team - 
Time to rekindle this?

Once a month someone has to go here: 
https://github.com/mozilla/marketplace-constants/blob/master/mpconstants/countries.py

compare them to here to get the full country names:
http://en.wikipedia.org/wiki/ISO_3166-1_alpha-3

and manually update them in a pdf list of regions for partners here: 
https://mobilepartners.mozilla.org/learn/detailed-firefox-marketplace-features/supported-regions/ 

Am i whinging too much? Likely, yes.
but it's a pain in the keester. 
so i whinge. 

and hope we'll get to this sooner rather than later.
or that someone knows a more automated way to generate this list of regions for Marketplace.

Comment 11

[Andy McKay]

(In reply to Wil Clouser [:clouserw] from comment #9)
> The PR in comment 7 was closed, but I don't think it should have been.  I'll
> nominate this for KTLO because it comes up, what, once a week?  I don't have
> a good idea of how much work it would be to do
> 
> 
> (someone is currently looking for a list of all the regions we support. 
> Sure would be easy just to say "all of them"!)

I agree, I think we are making have been making way too much work for ourselves. But the pull request hadn't been looked at in 2 months by the team and in pull request 28, a subset of countries were added. So not sure what people want to do. Happy to re-open the pull request (or someone else can re-basing on 28).

> The PR in comment 7 was closed, but I don't think it should have been.  I'll
> nominate this for KTLO because it comes up, what, once a week?  I don't have
> a good idea of how much work it would be to do
> 
> 
> (someone is currently looking for a list of all the regions we support. 
> Sure would be easy just to say "all of them"!)

Comment 12

[Mathieu Pillard [:mat]]

Only reason I made the pull request 28 is because there was some urgency in adding those other countries. I'll be glad to add them all once we solve the issue highlighted in comment 4 and comment 6.

Comment 13

[David Durst [:ddurst]]

(In reply to Mathieu Pillard [:mat] from comment #12)
> Only reason I made the pull request 28 is because there was some urgency in
> adding those other countries. I'll be glad to add them all once we solve the
> issue highlighted in comment 4 and comment 6.

Region is now only shown in Account Settings. And the packaged app is out.

So there's still the newsletter signup.

> There is still the question of whether we should display a huge list 
> of countries in developer tools, reviewer tools, curation tools, statistics.

And there's still the question for dev & reviewer tools. Statistics and curation would seem to be less popular enough to warrant a performance dip.

How many countries are now outstanding?

Comment 14

[Wil Clouser [:clouserw]]

Performance dip in any of those would be fine.  We could make a separate "regions" endpoint if we wanted to XHR it.

How many countries?  https://bug1151079.bugzilla.mozilla.org/attachment.cgi?id=8595671 is the current list.  So...triple digits?

Comment 15

[Mathieu Pillard [:mat]]

We'd go from 50 to 250 countries.

Comment 16

[Mathieu Pillard [:mat]]

To develop what needs to be done before adding all regions:

- In zamboni, the regions API endpoint already exists, it's /api/v2/services/region/ ; we need to make sure it handles translations like the rest of our APIs
- In fireplace, we need to stop shipping translations of the country names. In addition, it needs to stop relying on regions.REGION_CHOICES_SLUG and get the regions asynchronously using the API (the endpoint already exists:  /api/v2/services/region/) when needed. It's important that it does not block loading Marketplace itself, so for the newsletter that is shown on the homepage, it should be done when the newsletter is fully shown, or something similar
- In transonic, marketplace-operator-dashboard, marketplace-stats, we probably need to use the XHR too instead of depending on regions.js
- In marketplace-constants, regions.js need to stop exporting REGION_CHOICES_SLUG.

Comment 17

[Mathieu Pillard [:mat]]

I gave a shot to the XHR approach in fireplace. My WIP (which focused on getting the settings page working) is at https://github.com/diox/fireplace/commit/b22c7df788a1f538a7d91b4d8e00a124723d7804

Ultimately, it's pretty hard to do it in a clean way, because the newsletter needs the region list and is displayed on every page, including the homepage. Because of this, if we simply XHR'ed the region API then we'd also affect perf (the endpoint is lighter, since it's only translated in the current language, but it's still an extra HTTP request). And we'd need to handle the case where the XHR isn't loaded yet.

I think the best way to handle this would have been some kind of lazy {% defer %} block that would only be triggered in some particular cases (like, focusing the newsletter email field or clicking the newsletter submit button) but I am not sure it's such a good idea to go down that path - especially we'd still have to handle edge cases like when the XHR isn't loaded yet but the user wants to see/submit the newsletter form.

TL;DR: Sadly, I think the most pragmatic approach is probably to take the perf hit and have the 250 translations in our locale files (which are then loaded synchronously).

ni? Kevin in case he has better ideas.

Comment 18

[Kevin Ngo [:ngoke]]

XHR seems doable. It's an extra HTTP request, but it's async.

UI-wise, we could just {% defer %} the whole newsletter block, which would cache the regions.

Comment 19

[Mathieu Pillard [:mat]]

Bug 1160290 would have us drop the region dropdown entirely, making this much, much easier.

Comment 20

[Mathieu Pillard [:mat]]

Lots of little details across all repos (basically fix all the places where we display a list of regions), this won't be done this week. 

I merged https://github.com/mozilla/zamboni/commit/dca4bbf3e89f0e4d9b63e80970f303bd40a3cfbb which doesn't impact anything yet, and I have a bunch of pull requests for various repos that I'll merge on Monday.

Comment 21

[Mathieu Pillard [:mat]]

marketplace-stats no longer depends on regions.js to get the full list of region names, it uses the API instead: https://github.com/mozilla/marketplace-stats/commit/6d5e2da627e18b17b6a840bbab7dafbd335403a9

Comment 22

[Mathieu Pillard [:mat]]

same deal for fireplace, no longer depends on regions.js to get the full list of region names, it uses the API instead : https://github.com/mozilla/fireplace/commit/d26b516756fc07cb5a2bbf010caa298595c17182

Comment 23

[Mathieu Pillard [:mat]]

I've been following ISO 3166 like andym did in his pull request before me, but there are a number of places that have a ISO 3166 code but are actually a territory, dependency, part, etc. of another country. And there are also contested states like Palestine. What shall I do with those ?

Keep in mind the list is going to be the same for everyone regardless of their country of origin ; The full list is going to be shown in a couple places (statistics page has a dropdown, developer pages have a full list in compatibility page, etc).

Here is the list:

 Åland Islands: Part of FI
 American Samoa: Territory of US
 Anguilla: Territory of GB
 Antarctica: International
 Aruba: Part of NL
 Bermuda: Territory of GB
 Bonaire, Sint Eustatius and Saba: Part of NL
 Bouvet Island: Territory of NO
 British Indian Ocean Territory: Territory of GB
 Cayman Islands: Territory of GB
 Christmas Island: Territory of A,
 Cocos (Keeling) Islands: Territory of AU
 Cook Islands: Associated with NZ
 Curaçao: Part of NL
 Falkland Islands (Malvinas): Territory of GB
 Faroe Islands: Part of DK
 French Guiana: Part of FR
 French Polynesia: Territory of FR
 French Southern Territories: Territory of FR
 Gibraltar: Territory of GB
 Greenland: Part of DK
 Guadeloupe: Part of FR
 Guam: Territory of US
 Guernsey: Crown dependency of GB
 Heard Island and McDonald Islands: Territory of AU
 Hong Kong: Part of CN
 Isle of Man: Crown dependency of GB
 Jersey: Crown dependency of GB
 Macao: Part of CN
 Martinique: Part of FR
 Mayotte: Part of FR
 Montserrat: Territory of GB
 New Caledonia: Territory of FR
 Niue: Associated with NZ
 Norfolk Island: Territory of AU
 Northern Mariana Islands: Commonwealth of US
 Palestine, State of: In contention
 Pitcairn: Territory of GB
 Puerto Rico: Commonwealth of US
 Réunion: Part of FR
 Saint Barthélemy: Part of FR
 Saint Helena, Ascension and Tristan da Cunha: Territory of GB
 Saint Martin (French part): Part of FR
 Saint Pierre and Miquelon: Part of FR
 Sint Maarten (Dutch part): Part of NL
 South Georgia and the South Sandwich Islands: Territory of GB
 Svalbard and Jan Mayen: Territory of NO
 Tokela: Territory of NZ
 Turks and Caicos Islands: Territory of GB
 United States Minor Outlying Islands: Territories of US
 Virgin Islands, British: Territory of GB
 Virgin Islands, U.S.: Territory of US
 Wallis and Futuna: Territory of FR
 Western Sahara: In contention


Just in case there might be some political/legal implications I'm marking the bug as employee-confidential.

Comment 24

[Andy McKay]

I don't know. I've had people agonise over these lists in the past so I think that we shouldn't get into making any decision, unless there's a Mozilla decision someone feels comfortable supporting.

Personally I feel we should just add them all and follow the ISO standard and then there will be nothing to do in the future.

Comment 25

[Wil Clouser [:clouserw]]

Follow the ISO standard.

"there will be nothing to do in the future" isn't true -- we'll need to update this as the ISO standard evolves, but we should follow what they publish and not attempt to take any political sides.

Comment 26

[Mathieu Pillard [:mat]]

ISO 3166 has a field that denotes whether it's a country or a dependent territory. The list I compiled above is every territory that had a value other than just "Yes" and the value itself. But it does not specify what to do with the value.

I don't mind adding them all, including dependencies, but I just want everyone to be aware of it. Any move (whether we include dependencies as a separate territory, and whether or not we include contested territories) is taking a political side - ISO 3166 is considered neutral because it is not intended to be a master list of all existing countries, simply a way to communicate by assigning codes to them.

Comment 27

[Andy McKay]

(In reply to Wil Clouser [:clouserw] from comment #25)
> "there will be nothing to do in the future" isn't true -- we'll need to
> update this as the ISO standard evolves, but we should follow what they
> publish and not attempt to take any political sides.

Thats true. It should be a lot less, is what I meant and means we don't need to revisit any classification Mozilla makes.

Comment 28

[Mathieu Pillard [:mat]]

All countries added to marketplace-constants : https://github.com/mozilla/marketplace-constants/commit/157e3942468c60da965a844315efc216b72fc109

Now I'll bump all the projects using it.

Comment 29

[Mathieu Pillard [:mat]]

Fixed in:
- https://github.com/mozilla/zamboni/commit/f2112c8acd775f7434f14e8c21dd04c3211820a9
- https://github.com/mozilla/fireplace/commit/8b18d3f44ed08d133ec53fd02d50358482a9fe1c
- https://github.com/mozilla/marketplace-stats/commit/a32080a23822a5880936d8cdabffa34e496a7ddc
- https://github.com/mozilla/marketplace-operator-dashboard/commit/97263efb8935ff5e1e820018c45a0a4c111c4f97
- https://github.com/mozilla/transonic/commit/763ccdfaca4c6bdb989d5c845e8b2a82f2bb4b9f

QA:
- Please check that all countries are available and working in all Marketplace pages (developers compatibility page, consumer pages through debug page, statistics dropdown, curation tools etc)

Comment 30

[Victor Carciu]

Verified as fixed . The countries are available everywhere.