1120311 - Suggested tiles: Affiliate Release (FX 39)

Status: VERIFIED FIXED

(Content Services Graveyard :: Tiles, defect)

Description

[Ed Lee :Mardak]

The basic idea for related tiles is that we show a single tile if it matches a site from the user's top sites data. The tiles data to show would be packaged similar to how enhanced tiles currently work: Firefox periodically downloads a set of tiles with url/title/image data but related tiles has one additional related field with an array of site strings. The related tiles should not be shown as directory tiles, so we might have a separate object containing just related tiles.

To match the user's top sites, we can take the top 100 links data already available from NewTabUtils and remember which sites to match against. This implementation is both simple in that it uses data easily accessible and has a bias towards sites the user frequently visits (so relatedness should be more stronger than a site the user might have accidentally visited once).

We'll want to report on clicks and impressions just like what we do for enhanced tiles, so tile id with optional position, pinned, etc values. Note, this means we don't need to send the url of the tile as we know it from the tile id, and more importantly, we don't send the url of the matched-top-related site from the user's history.

From a UX perspective, we'll want to help users understand why they're seeing a related tile with some messaging like "popular with site.com visitors" where site.com is the site matched from the user's top sites. We'll also probably need a different indicator than [SPONSORED] especially if it's an organic related tile vs paid. To start, we can limit to just one related tile that appears in the first unpinned position, i.e., prepend one related tile to getLinks().

If there are multiple related tiles, we might just randomly pick one for now or even simpler is just take the first match.

For user control, we might want to add a customize/gear menu item to turn off "show recommended tiles" which could potentially split into "recommendations from sites I've visited" and "recommendations from new sites." This is in addition to the usual (x) to block the recommended url.

On the server side, naively the server just needs to allow for the additional "related" field for the array of sites. Later versions of splice that creates distributions would need to associate related sites to a creative just like title/url/etc.

From a business and privacy perspective, we'll want to have a minimum number of related sites for a valid related sites array. This is to prevent Mozilla from knowing for sure that a related tile was shown because a user visited siteXYZ.com. E.g., if there were 4 sites in the array all with equal likelihood of being in a user's top sites, the impression sent to our server would have enough deniability of which of site1.com vs site2.com vs site3.com vs site4.com the user visited. We have telemetry data that could help us calculate these site probabilities so that the most popular site in an array of related sites would be at most 50% likely. We might also have basic checks in Firefox that require a minimum number of related sites (although an attacker could easily do related: ["realsite.com","fake.com","fake2.com"]..)

As an initial test for Nightly as we implement, we can have a related tile for MDN http://developer.mozilla.org/ that has a related sites list of web development sites, e.g., w3schools.com. So if the user has a w3schools.com link in the top 100 newtab links, we show a MDN tile in the first spot with text underneath it "popular with w3schools.com visitors"

Each of the pieces above will have separate bugs to discuss/decide/design/implement.

Comment 1

[Ed Lee :Mardak]

Attachment: super hacky wip

Comment 2

[Ed Lee :Mardak]

Attachment: related shown because of irs.gov

Comment 3

[Ed Lee :Mardak]

Attachment: related hovered showing explanation

Quick hack replacing title avoided needing to resize a bunch of tile spacing and adjusting for additional space to show the "because" line

Comment 4

[Ed Lee :Mardak]

https://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/[e-mail]-7e56a3e36bfd/

Comment 5

[Ed Lee :Mardak]

Attachment: related.json

The patch points to https://people.mozilla.org/~elee/related.json which currently has this file. The main difference is there's a new value for type and an additional related field:

            "related": [
                "irs.gov",
                "taxact.com",
                "hrblock.com",
                "freetaxusa.com",
                "1040.com",
                "taxslayer.com"
            ],
            "title": "TurboTax",
            "type": "related",

Comment 6

[Ed Lee :Mardak]

There's various design explorations from bug 1121549 attachment 8548988 [details] that could influence what related tiles looks like.

Comment 7

[Ed Lee :Mardak]

No need for points/iterations for story/feature bugs. If we need an implementation bug for some related tiles code changes, we can just file a separate bug blocking this one.

Comment 8

[Marina Samuel [:emtwo]]

oopps, I meant to change a different bug, not this one

Comment 9

[Axel Hecht [:Pike]]

It's news to me that we're doing Tiles feature uplifts for 38, and I don't recall that being mentioned in any 38 meeting.

At this point in the schedule, that worries me.

Erin, can you investigate?

Comment 10

[Robert Kaiser]

If we are really shipping this in 38, who is supposed to test this? Do we need to do this in release quality testing? How can or will this able to be tested?

Comment 11

[[SV Manager] Florin Mezei, QA (:FlorinMezei)]

(In reply to Robert Kaiser (:[e-mail]) from comment #10)
> If we are really shipping this in 38, who is supposed to test this? Do we
> need to do this in release quality testing? How can or will this able to be
> tested?

We (Softvision) started looking at some fixes here, but I see that the test plan (https://wiki.mozilla.org/Firefox/2015#Quality_Test_Plans) indicates kthiessen as the QA Contact, so I'll add him as the QA contact here (and related fixes tracked in sprint). 

As it was the case for other people, it's somewhat surprising for us to see that this is targeted for 38. Note that not having a Target Milestone set is actually hiding this from people's scope. This is only mentioned on the wiki page, without any status or risk profile. From the QA standpoint it's risky to start working on a feature when fixes start to land in the Beta phase... especially when that Beta phase is for a major release like 38.

Comment 12

[Karl Thiessen [:kthiessen, he/him]]

Florin, could you please point me to any bugs related to the fixes you were/are looking at?  My intent by putting my name in the Wiki was certainly not to derail any existing testing, but to serve as a single contact point -- as others in this bug have noted, it looks like the testing efforts for this bug are not well-known.  In particular, if there is Softvision testing going on for this bug, it should certainly continue.

Comment 13

[Ed Lee :Mardak]

Cornel Ionce did the QA for Directory and Enhanced tiles and has already been verifying various Suggested Tiles changes. As kthiessen noted, he's focused on the server side aspects, so I believe it would make sense for Cornel to continue.

Comment 14

[[SV Manager] Florin Mezei, QA (:FlorinMezei)]

(In reply to Ed Lee :Mardak from comment #13)
> Cornel Ionce did the QA for Directory and Enhanced tiles and has already
> been verifying various Suggested Tiles changes. As kthiessen noted, he's
> focused on the server side aspects, so I believe it would make sense for
> Cornel to continue.

Fair enough, I am adding Cornel then to coordinate QA efforts here.

Comment 15

[Ed Lee :Mardak]

bsmedberg/dougt, can both of you confirm that with bug 1152145 hardcoding buckets sites and names to improve verifiability of targeting and to prevent arbitrary site querying, we're good for suggested tiles in 38?

To summarize, suggested tiles uses the existing tiles delivery and reporting mechanism where Firefox downloads a json file once a day to decide whether a tile should be shown, and we report back tile data including a tile id but no urls for when a tile is shown or clicked. The main difference with suggested tiles is that tiles are conditionally shown if the user visits a set of pre-approved sites, so Mozilla could infer that a user went to one of the sites but not know which one.

Comment 16

[Doug Turner (:dougt)]

I don't think we're good until you work that communication plans and people don't revolt.  The tech does what you want it to, and in 38 you don't have a remote operated general purpose browser history query tool.

Comment 17

[Ed Lee :Mardak]

(In reply to Doug Turner (:dougt) from comment #16)
> I don't think we're good until you work that communication plans and people
> don't revolt.
We do have a comms plan: https://docs.google.com/spreadsheets/d/1XBac4QKFgwKKgsgt8oly9vmzjtFRc5QzcDH6nkHDhW8/edit?usp=sharing

Part of it is getting your approval to move forwards before messaging to dev.planning. We can flip that around and message there first before you approve.

Comment 18

[Benjamin Smedberg]

As mentioned at https://wiki.mozilla.org/Firefox/Data_Collection and in email, the primary mechanism I need to review this is patch to the in-tree docs explaining the data collection system at least from the perspective of the client code. I'm not prepared to review an expansive tree of bug comments. Please prepare that as a patch that I can review. I'll also want somebody (perhaps you Ed) to vouch that the docs you write match the actual behavior of the code in the tree.

Comment 19

[Ed Lee :Mardak]

All bugs this feature/[story] bug depends on have been fixed for some time and in 39.

Comment 20

[Cornel Ionce [:noni] [Hubs QA]]

All these bug dependencies were covered and verified. Closing as verified.