Open Bug 1190910 Opened 9 years ago Updated 2 years ago

Expose country code to services that need it, for better consistency, remove redundant work

Categories

(Toolkit :: General, defect)

Product:
Toolkit
Component:
General
Type:
defect

Tracking

()

People

(Reporter: glind, Unassigned)

Details

Claim:  Several services (translation, pocket, search, about:home, about:newtab, self-repair) need to make decisions based on INTERACTIONS between Geography and Locale / User Language.  

Current state :
- content hits geo.mozilla.org/country.js (ex:  // https://github.com/mozilla/snippets-service/blob/master/snippets/base/templates/base/includes/snippet_js.html#L283
), and `localStorage` caches result.
- search stores it in pref.  http://mxr.mozilla.org/mozilla-central/source/toolkit/components/search/nsSearchService.js#645

Problems with Current State:
- different sources, caching strategies (timing) can lead to inconsistent experience
- redundant work / storage of country (localStorage, prefs), repeated requests.


Requested solution (1):
- mozilla related sites can see a mozilla.countrycode object (or similar) (maybe a new 'permission'?
- central service to store, cache, re-try / update the country code.  

Risks / Warts:
- wrongness of country code for edge cases.  This is inherent to geoIP lookups.
- not triggering 'refresh' soon enough when people change IPs. 
- services that want to use the 'official' answer will need to have added perms.  (they can avoid this by doing their own lookups).

Alternate Solution (2):
- just ping somewhere every time it's needed.  No caching or storage.  

Risks:  This is a lot of hits.  We cache now due to volume.


Alternate Solution (3):
- expose country code to all sites at `window.geo.countrycode` (hand waving), and make it a standard / extension whatever.  This benefits all content creators.
(Florian: ref to search module.)
Flags: needinfo?(florian)
cc :atoll (geodude) and :hannosch (mls), since we have other threads on related topics active
+10 on the idea. I wanted to see this happen for a while. On different platforms like Fennec the country information might also come from different sources, potentially OS-level APIs or the information from the SIM card in a phone.

For the country code or position information in general, there are two different things:
1. The "home country" - the place you live at most of the time
2. The "current country" - the place you are right now

If you are traveling, those two can be different. Depending on the use-case you might want one or the other.

I was only involved in the search service use of MLS, but in that case we don't want to switch your search engine to something different, just because you are in a different country for a week. Currently the logic never refreshes the country code, so it wouldn't catch someone actually moving permanently to a different country. It also means that if someone installs Firefox for the first time, while not being in their home country, we'd default to the wrong country. But both of those are the edge-cases, and the "I'm just traveling for a week" is more common.

I think the use-cases of our browser are more about "home country" information, often also tied to your preferred locale. Personally I think this information should be exposed to the user in the Preferences -> Content screen right next to the language settings. We'd automatically select the country once and prefill the setting with it, but would allow the user to edit it.
Gregg, you didn't have any specific question from me, so I assume the needinfo is just requesting feedback on comment 0.

(In reply to Gregg Lind (User Advocacy - Heartbeat - Test Pilot) from comment #0)
> Claim:  Several services (translation, pocket, search, about:home,
> about:newtab, self-repair) need to make decisions based on INTERACTIONS
> between Geography and Locale / User Language.

Unless I'm missing something, the translation feature cares about the user locale (ie. what language the user can read), not the user's country.

I think this also applies to Pocket.

I assume about:home uses the country to decide which snippets to show, and about:home to decide which tiles to show. Isn't this country detection done on the server side while processing the request, rather than on the browser side?

I don't know anything about the self-repair case.

> Alternate Solution (2):
> - just ping somewhere every time it's needed.  No caching or storage.  
> 
> Risks:  This is a lot of hits.  We cache now due to volume.

This solution won't work for anything that needs the country code during startup (eg. search).


I remember mconnor saying we should have a central place in the browser knowing the country code info, and not make this something that's specifically for search, but I don't remember hearing before about an intent to make this country code info accessible to web content.
Flags: needinfo?(florian)
(can we find an owner / taker for this?  Suggestions welcome!)
"Isn't this done on the server side"

Yes and no.  For home, newtab, self-repair:  As currently written, it pings geo.m.o, which returns a js file. [1]  That returned js file is built based on requesting IP.  That result is cached in localStorage.  

I think you are suggesting that it could be done by smarts on the server for each of those services.  That might be worth exploring as an alternate solution.  That solution shifts this work to web-dev for each project.  [2]  


[1] https://github.com/mozilla/snippets-service/blob/master/snippets/base/templates/base/includes/snippet_js.html#L283   as an example.
[2] self-repair currently has no server and is written as a static page deploy.  This has some benefits and drawback.
Poke and ping.

Got burned by this AGAIN in heartbeat a lot recently.
nthomas, is this (putting a GEO country code into firefox) an appropriate use / expansion of AUS / balrog?  Got a better idea?
Flags: needinfo?(nthomas)
Cloud Services is also running a geo service that uses AWS autoscaling and the same backend dataset that our old Geo service does. Their only issue is that they have to implement the 'country.json' response format, and (to the best of my knowledge) do not currently have engineering time assigned to implementing that.

You may want to reach out to them directly and work out an arrangement - perhaps their source code is readily available, so if they are starved for resources we can find others within the company to help - and so on.
(In reply to Richard Soderberg [:atoll] from comment #9)
> Cloud Services is also running a geo service that uses AWS autoscaling and
> the same backend dataset that our old Geo service does. Their only issue is
> that they have to implement the 'country.json' response format, and (to the
> best of my knowledge) do not currently have engineering time assigned to
> implementing that.

Not quite. We do have engineering time on the MLS side (Mozilla Location Service) But it doesn't make a lot of sense to implement a "new" API (country.json) in this service. It would make more sense to consolidate the client side code and let multiple pieces use the existing MLS API.
The old client code is baked into Firefox actual, so we can't update this interface for older clients. So we'll have to keep the old /country.json interface up and running for the long foreseeable future.
(In reply to Gregg Lind (User Advocacy - Heartbeat - Test Pilot) from comment #8)
> nthomas, is this (putting a GEO country code into firefox) an appropriate
> use / expansion of AUS / balrog?  Got a better idea?

Do you mean - Firefox/Fennec/etc are querying for updates at least once a day, so that's a place to return a geo-IP result ? If so, there are assorted reasons that might not work well, eg updates disabled by user, clock wrong so SSL cert seems invalid. It would also prevent us from doing any caching, so the system would have to scaled up considerably.
Flags: needinfo?(nthomas)
Gregg, I'm working with Hannosch to try and get you a better solution using MLS. May we have a few days to recover from the PHX1 move and to develop a more coherent plan (and, ideally, a timeline)? OR, if there's an immediate urgent issue/outage, let me know ASAP.
MLS will provide a better solution here in terms of backend resources, but the fundamental problem remains - we're making an enormous number of calls that should be redundant and that has costs to the company to provide for all these requests. Any sort of caching would be an enormous help.
Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.