925477 - [tracking] Build recommendation system prototype

Status: RESOLVED FIXED

(Marketplace Graveyard :: General, defect)

Description

[David Bialer [:dbialer]]

This is phase 1 of building recommendations, where we get to a prototype.
Working with a partner, we will build a prototype, test it for its performance, build a UI prototype and test against users.

Comment 1

[David Bialer [:dbialer]]

Added PRD.  Note, for the protoype the things in scope are as follows:
Personal Recommendations for Apps based on past purchase (install since many apps are free) history.


1. As a user, each time I go to my home page,  I would like to receive (suggested number 3 - 5) new suggestions for apps, based on the types of apps that I have installed onto my device that supports my selected language and is available are for my selected country.  These suggested apps should be kept ‘fresh’, possibly by cycling through a list created for me each time I visit a particular page. P1


2. As a user, I would like “Collections” recommended to me that would match the apps that I have installed. P2

Basically - priority 1 is apps recommendations, priority 2 is collection recommendations.

Comment 2

[Matt Basta [:basta]]

We can only suggest apps based on ones you've already installed from the Marketplace. The Marketplace cannot know what you've installed externally to the Marketplace.

- How do we generate recommendations if you haven't installed anything yet? What if you are in a region that does not have sufficient regional data? (e.g.: newly launched regions)
- How do we generate recommendations for anonymous users (i.e.: the vast majority of our users)? We would not be able to use pre-computed data about the user since we don't have anything to associate with them.
- Should this system respect the DNT flag? 
- How do we integrate feature profiling into the mix? We don't know what the user's device is when they're not around, and that information isn't collected. We shouldn't recommend apps that you can't install.
- When we don't know anything about you or your region (i.e.: you're in Worldwide and anonymous, or you're in the US), what do we show? What are our fallbacks? Where is this data coming from?


From our privacy policy:

> We also don't want your personal information for any longer than we need it,
> so we only keep it long enough to do what we collected it for. Once we don't
> need it, we take steps to destroy it unless we are required by law to keep it longer.

How does this impact our recommendation system? Conceivably the information that we will need to start gleaning (what apps anonymous users are installing) is "personal information".

Comment 3

[David Bialer [:dbialer]]

To address some of your overall concerns and provide a bit more context, we are working on a prototype with Telefonica Research Labs who have a lot of expertise in some of the issues. See http://www.tid.es/es/Research/Paginas/TIDProjectProfile.aspx?Project=Recommender+Systems+ 

There is a prototype app for Android here called Frappé:  https://play.google.com/store/apps/details?id=es.tid.frappe


(In reply to Matt Basta [:basta] from comment #2)
> We can only suggest apps based on ones you've already installed from the
> Marketplace. The Marketplace cannot know what you've installed externally to
> the Marketplace.
this will be a limitation for at least this version.  Since most apps come from Marketplace at the moment, it is something we are OK with to start.
> 
> - How do we generate recommendations if you haven't installed anything yet?
This is the classic 'cold start' problem.  Our algorithm partners may be able to address this.  But essentially things like "Popular", "Trending" and Moz Collections also help address this from a user perspective. 
> What if you are in a region that does not have sufficient regional data?
> (e.g.: newly launched regions)
The recommendation engine should deal with this case.  There will be some threshold I imagine where data becomes useful.  
> - How do we generate recommendations for anonymous users (i.e.: the vast
> majority of our users)? We would not be able to use pre-computed data about
> the user since we don't have anything to associate with them.
The only thing we may have is country - and we will have to determine if this is something we can and/or should use for anonymous users.
> - Should this system respect the DNT flag? 
Yes.  It should also be opt-in/opt-out in user settings.
> - How do we integrate feature profiling into the mix? We don't know what the
> user's device is when they're not around, and that information isn't
> collected. We shouldn't recommend apps that you can't install.
Good point.  We will need to filter recommendations or use the device features perhaps as input data into the recommender system.
> - When we don't know anything about you or your region (i.e.: you're in
> Worldwide and anonymous, or you're in the US), what do we show? What are our
> fallbacks? Where is this data coming from?
Again - the cold start problem.  Popular and Trending help take care of these users.  It may be we don't show anything until you have at least X apps.
> 
> 
> From our privacy policy:
> 
> > We also don't want your personal information for any longer than we need it,
> > so we only keep it long enough to do what we collected it for. Once we don't
> > need it, we take steps to destroy it unless we are required by law to keep it longer.
> 
> How does this impact our recommendation system? Conceivably the information
> that we will need to start gleaning (what apps anonymous users are
> installing) is "personal information".

Comment 4

[Rob Hudson [:robhudson]]

We also need to consider app uninstalls, which the Marketplace never gets notified of. E.g. I install a sports app but then deleted it b/c I don't like sports. But now Marketplace keeps recommending other sports apps because it doesn't know I deleted it.

Comment 5

[Andy McKay]

We have a prototype, if there's more to do lets open those bugs.