Open Bug 994927 Opened 6 years ago Updated 3 years ago

[geolocation] Mock up a general purpose map

Categories

(Participation Infrastructure :: Phonebook, defect)

defect
Not set

Tracking

(Not tracked)

People

(Reporter: hoosteeno, Unassigned)

References

(Blocks 1 open bug)

Details

(Whiteboard: [kb=1491749] )

Attachments

(9 files, 1 obsolete file)

Let's build a map view that we can use on pages where we have lists of Mozillians. For example, search results or group members. 

* The goal is visualization, not getting directions to someone's house. Knowing this may help decide between various visualization options.

* The map should consider that at any zoom level there will be some people for whom we do not have (or cannot show) location data at that specificity. Region and city are optional; all fields are subject to per-field privacy levels which may mean they cannot be shown to a public viewer of the map. 

* It may be the case that certain zoom levels could conceivably plot people with specific location information (New York, New York) and generic location information (USA). There may be an elegant solution for this but the easiest solution is probably to eliminate the generic points after a certain zoom level.

* We should not plot mozillians with no viewable data. If an anonymous viewer is looking at a map of France, they should see the number of mozillians in France with _public_ country data. Mozillians without public country data should not be counted in aggregate numbers nor plotted on the map.
Attached image mockup.jpg
Option 1 mocked up: create a simple block that shows the map above the list of users on a group. (some buttons were moved around to make space)
For the map tiles themselves, we can create our own set with TileMill. The resulting tiles will be about 5Mb in total for 5 zoom levels. Keeping the zoom levels low will obfuscate the specifics/privacy of each user and keep load down on rendering which took 4 seconds on my macbook.

TileMill can be run serverside on a cronjob if we decide we'd like to use it that way. Since specific data will be handled by markerclusters in realtime though, the cron wouldn't need to run all that often.
Attached image at-zoom-level-5.jpg
Here is a mockup of the map at zoom level 5 (as close as it goes) some example markers made at different cities in NC are clustered together in screen-space. Clicking on the cluster makes the markers expand. Any marker from there reveals full details, in this case "Texas, United States"
Attached image mockup2.jpg
Here's a second mockup mimicking something more Pinterest-like
Attached image markers.png
custom graphics for marker clusters using Open Sans Light from Mozillians' header.
Started a branch on Github for this https://github.com/caktus/mozillians/tree/geo-visualization that displays the map on a group page.

There are a lot of things to hook up on the back, and front ends.

- provide data that the map can use as markers. A simple array of javascript objects will do.
example:

[-14.235004,-51.925280, 'Pedro Smith (Brazil)'],
[31.968599, -99.901813, 'Jane Doe (Texas, United States)'],

How that happens can be in one of multiple ways
1. We can serve a js file generated by python's backend up front
2. We can render the Javascript directly with a jinja template

Either way, until there's real geographic data to play with, I'll keep working on this with a mocked dataset of the Python / Firefox groups so I can start to catch edge cases. Mapbox has a few tutorials for 
using Google Docs to convert a list of addresses to lat/long quickly and i'll be able to get by with that and some more manual entry for the time being as I style the tiles and markers.
https://www.mapbox.com/tilemill/docs/guides/google-docs/

Lastly, to generate our own tiles for the broader data, I can work with a local tilemill export, but eventually we should attempt running tilemill on the server.
Guides like this are floating around:
https://www.mapbox.com/tilemill/docs/guides/ubuntu-service/ (tutorial)
https://www.mapbox.com/tilemill/docs/manual/exporting/ (fill list of cli flags)
Attached image Screen Shot 2014-04-22 at 1.48.36 PM.png (obsolete) —
Today's experiment with importing what available data we already have into a spreadsheet, geocoding it (mocking what we'll actually do), and displaying that data in the map tiles themselves.
a quick render of the whole mercatur map of mozillians geo data
Attachment #8410415 - Attachment is obsolete: true
Woohoo! I got the two connected together and showing up on the page. A good next step would be to pass the right data to the map based on what page you're looking at. I'll need back-end help to do that. We can probably get by not updating the map tiles until later since they aren't interactive anyway.
The data I'll need from the backend ideally would be as follows:

Name,
Latitude,
Longitude,
Country,
Region (if available),
City (if available),
some sort of URI to point to the user's gravitar or a short URL.

The gravitar isn't necessary, but I think would be nice. With a lot of users, the response might get big with full gravitar URLs.
Formatting of that data will eventually get looped through in Javascript. A simple array of objects is fine.
A screenshot of progress up to this point sancus and I have made
Got marker labels working today.
There seems to be a cross-domain issue that's being handled by the Firefox browser even after we get a response from api.tiles.mapbox.com

Looking in the Style editor, the map styles are just plain gone. If I paste in the response from mapbox.css into a new stylesheet, suddenly the map looks fine. The GUI images (zoom in, zoom out, the little `i` for info) are also missing which further reinforces that it's related to a cross-domain thing.

All the settings in base.py are giving plenty of the correct permissions so I'm not sure what the problem is.
See Also: → 920651
Status: NEW → ASSIGNED
Moving here comments from github so we can continue the discussion:

Pull request: https://github.com/mozilla/mozillians/pull/1054


-- glogiotatidis

I have same serious concerns about this implementation:

    We drastically increase processing time depending on the size of the group. And we do re-generate the same geodata on every page of the same group
    Similarly for the size of the downloaded html.
    To measure the difference:
        the 'sf-momument' group needs 8+ seconds on my local machine to render compared to 1+ seconds without this change
        every page of the 'sf-momument' group has 400Kb more data

    One option is to load the map on demand, using ajax or on a separate group view.

    On another note the group page needs some beatification now that we have a map as well. @wraybowling I'm sure you can do miracles :)


-- sancus

@glogiotatidis Why don't we cache the data for groups larger than 100 members by simply sticking the json files on disk? There's only 20 or so such groups now, so it won't take much space, and I don't think it's a performance issue to just do it dynamically for groups smaller than 100. We could even put the threshold down at 20 or 50 members.

We could regenerate these big groups every 24 hours. And since it'll be a JSON file that they link to, they'll take full advantage of browser caching and won't need to redownload the data on every page.


-- wbowling

I second the caching idea.


-- glogiotatidis

Caching is a neat idea. It doesn't have to be a file, it can be a django.CACHE. We can have a special url for all groups like /group/foo/locations.json that will return the json object that we're currently embedding into the html. This view can do all short of fancy caching and a url can get better browser caching.

But bytes transferred are bigger concern than cpu time in this case: Caching will help our servers mostly but will not help users, unless the same user is visiting the same group more than once per day. Also clients are downloading N*2 times the avatars of all members of the group, before they even zoom in. This should be done only at a level that avatars appear.

Caching included I still believe that displaying the map directly on every group page is a bad idea. We should bring the map on demand or have it only in the first page of the group. On demand will help with both bytes transferred and rendering time especially on mobile devices. In both cases the UI needs love, it's just like we threw a (really cool) map in a page that already needed UI love.

I understand that we currently have only a few groups with more than 100 members but they are also the most visited groups. Most -if not all- of the functional areas have more than 100 members, so this is our common case.

Also currently the biggest concern is that we don't respect the "save region" and "save city" settings atm. If a user doesn't share city and/or region then we still feed their lat/lon to the clients.

Also this is based on lat/lon which I understand they are a temp solution? https://bugzilla.mozilla.org/show_bug.cgi?id=1022014
It seems best to keep both the lat/long coordinates *and* the mapbox IDs. It's the only way we're going to be able to keep accurate records (mapbox IDs) and have fast location recall for the visualizations (coordinates). If calling the coordinates, placenames, and user name is bogging us down, cache it.
Whiteboard: [kb=1491749]
(In reply to Wray Bowling [:wbowling] from comment #16)
> It seems best to keep both the lat/long coordinates *and* the mapbox IDs.
> It's the only way we're going to be able to keep accurate records (mapbox
> IDs) and have fast location recall for the visualizations (coordinates). If
> calling the coordinates, placenames, and user name is bogging us down, cache
> it.

Keeping lat/lng for backend use and displaying it only to the user which it belongs to it's fine. But the pending pull requests [0] expose the lat/lng of _all_ users to all other vouched mozillians, disregarding the privacy settings for each profile. This is a no-go in my opinion.

[0] https://github.com/mozilla/mozillians/pull/1054
Well privacy settings aren't really disregarded because you can't hide location info from vouched mozillians anyway, there's no more restrictive privacy setting.

There is an issue where we're storing overly-precise lat/lngs instead of the "generic" lat/lng for your region/country if you've selected not to use city. There should be no absolutely no way to set a lat/lng that is more precise than the generic one for your city -- the map pointer snaps to the closest city bounding box even if you drag it around.

Given this, I think there is no privacy concern as long as the person in question has chosen to use City-level precision in their profile.

The precision issue is that when I remove the checkbox for saving City, the code currently does not reset my lat/lng to the next higher lat/lng(the one for California). It should. This would also require a migration to correct current profiles with this overly-precise lat/lng issue. IMO this is a separate bug.

What I can do for this bug is change the code so that you are only included in the map if you have set your profile to save City. This should alleviate any privacy concerns -- at that point you are sharing the lat/lng for your City with other vouched mozillians, which is not at all private data, anyone who knows your City can look it up for themselves via the Mapbox API.
Andrei please verify the following scenario:
 - In user/edit search for a city e.g. "Athens, Greece"
 - Select to save only country, uncheck region and city
 - Save()

The lat/lon saved, points to the specific city not to the country. The same lat/lon is used by the group map to render the pointers. Therefore one with a vouched account can figure out all locations on city level of all other vouched mozillians.

Can you confirm this?
That's correct. That's exactly what I said happens in Comment #18...
Therefore there is a privacy concern since a user can chose not to share city and the city is being shared.
(In reply to Andrei Hajdukewycz [:sancus] from comment #18)
> What I can do for this bug is change the code so that you are only included
> in the map if you have set your profile to save City. This should alleviate
> any privacy concerns -- at that point you are sharing the lat/lng for your
> City with other vouched mozillians, which is not at all private data, anyone
> who knows your City can look it up for themselves via the Mapbox API.

Yes that probably will work but I think a better solution is to store the lat/lon in the level you chose to share. I.e if I chose to share only country, the lat/lon should be the same as the one returned when I search for country. 

Do you think what will work without breaking something else?
Yes, I think we should fix the underlying issue. I just don't think that the group maps page bug is the place to do it. It will involve a data migration to correct all the profiles that have overly-precise lat/lngs and there may be some complexity/workarounds needed, because mapbox doesn't always properly return lat/lngs for regions and countries last I checked.

I think restricting the map to City-enabled profiles is a reasonable stopgap until someone has time to work on that issue.
Blocks: UX/UI
Assignee: wbowling → nobody
Status: ASSIGNED → NEW
You need to log in before you can comment on or make changes to this bug.