Closed Bug 1179878 Opened 9 years ago Closed 9 years ago

Pass 'user agent string' and number of device profiles present from Accounts to ExactTarget

Categories

(Cloud Services :: Server: Firefox Accounts, defect)

Product:
Cloud Services
Component:
Server: Firefox Accounts
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: bniolet, Unassigned)

References

Details

This is a next step in our FxAccounts integration with Journey Builder / ExactTarget. 

Our marketing plan for Firefox Accounts users is to try to encourage Sync users to sign in from more than one device. 

To accomplish this we'll need to know: 

1) How many different devices a given user has used to sign into their Account. 

2) The identity (user agent string) of the device. This will help us understand if our emphasis should be on desktop devices (work and home) or mobile (desktop and phone).
Blocks: 1158021
Ben, this is great, exactly the sort of detail we need to drill down into next steps.

The bad news is that we currently don't record this information so there's going to be significant work on the FxA side to make this happen.  The good news is that this work unlocks a bunch of concurrent goals so I have every confidence it will get done in short order.

How will this information be represented in ExactTarget?

The most obvious way I see this working is we just ship you device-connection events like "user X connected a device with user-agent string Y", and you use them to build up an internal model of each user.  But this may not be the best choice if it ends up duplicating state that we're storing elsewhere on the user's account.  I have a couple of other ideas, but want to hear about how your team thinks it should look from inside ExactTarget.
Great question. 

How or what can we see from the Accounts side? How much / how little detail is there and how standardized is it? Understanding that we're not currently recording that info, are there any previews available on what we would be collecting?
In the most basic case, we'll just start recording the user-agent string of each device that connects to the account.  So we might end you an event like "user X connected device 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:42.0) Gecko/20100101 Firefox/42.0'".

Or we could do a bit of classification on this string and determine that it's a desktop machine, then send you an event like "user X connected a Desktop device".  I'm guessing this is closer to something that you'd actually store in the user model in ExactTarget.

We will also provide an API where you can list what devices are connected to the account, so basket might grow the ability to ask "what devices does user X have connected?" and we'd answer "one of type 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:42.0) Gecko/20100101 Firefox/42.0'".
Here's what we know so far, based on some conversations on our end and some research from Brent: 

1) We do ultimately want metadata of the user agent (desktop/Android/iOS vs. Mac OSx/Android 4.2.3/iOS 6). But because we're picky, we'd love it if the data can be parsed a bit before it gets to ExactTarget. Ideally if the format is consistent or if there is a consistent delimiter, it'd be great to have the user agent simplified so that "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:42.0) Gecko/20100101 Firefox/42.0" would appear as "Operating System = Mac" "Version = OS X" "Browser Version = Firefox 42.0.' In the long run, that type of data may push other Journeys as well.

2) Journey Builder is event-driven and the best way to send this information to ExactTarget is through a transactional database where each user can exist in multiple rows. So each new entry would represent a given user logging in with a new device profile. Journey Builder would then look for users who have added a second profile. This structure is more efficient for us to run processes against in the long run, and Journey Builder could view each line of the table as an event.

3) From Brent: Once we have a structure in place, we can recommend a transmission type. We're most likely looking at an API insert to a Data Extension within ExactTarget.

Brent, did I miss anything? 

Ryan did this help? More questions?
Looks good from my perspective. For the first example, the "=" would essentially be column headers and their associated values per record (just to calrify).
Would we be interested in this kind of browser and system data from other sources? If so we should have FxA send us the UA string and develop facilities in basket to glean from it what we actually want. We shouldn't ask every service that wants to integrate with basket/ET to parse UA strings in the same ways.
> the best way to send this information to ExactTarget is through a transactional database
> where each user can exist in multiple rows.

To be clear, is this "transactional database" a thing that exists inside ExactTarget, that we just need to stuff rows into?

> we should have FxA send us the UA string and develop facilities in basket to glean from
> it what we actually want

I think this is going to be the best way to do it anyway.

Another thing to consider is whether we'll capture these events for users who have not opted-in to the Journey.  I think I'd prefer that we only build up this user profile for users who are opted in, but it might prove difficult to implement that way.
>To be clear, is this "transactional database" a thing that exists inside ExactTarget, that we just need to >stuff rows into?

Yes, the "transactional database" is a date extension that we set up (rows, data types, max characters, default values, etc.) in ExactTarget and then start sending data to. 

>Another thing to consider is whether we'll capture these events for users who have not opted-in to the >Journey.  I think I'd prefer that we only build up this user profile for users who are opted in, but it >might prove difficult to implement that way.

I agree it would be best to only capture for users who have opted in.

That said, would a potential solution be to set something up like we use for our Double Opt In process? (pmac?). The way that works is we send all subscribers to Double Opt In. They have 24 hours to finish the opt in process. If they do, they get moved to the Master Subscribers. If not, they get removed from the system. 

So if we had to collect all device strings for all users, we would dump them all into a big ol table. Step 2 is to sort out who we keep and who we toss.
Also, here comes the part where Relationship Marketing (that's me) makes a highly improbable request: 

What are the odds that we'd be able to start collecting this information by the end of July, beginning of August? I ask because the Windows 10 launch makes this data all the more useful and valuable to a bunch of teams within the org.
> What are the odds that we'd be able to start collecting this information by the end of July,
> beginning of August?

As usual, depends on how much of the work we can shift where :-)

Strawman:  We add a `/fxa-activity` endpoint to basket that's analogous to the current `/fxa-register` endpoint.  I arrange for a JSON blob to be posed to this endpoint when a user connects a device.  Basket checks if the user is subscribed and if so, parses the included user-agent info and adds/updates rows in the database in ET.

It's highly inelegant, but would get us started and is not too divergent from how existing FxA integration works.  Sound plausible at a high level?
Flags: needinfo?(pmac)
Sounds like it would work well enough to me. I know we want to do a pull vs. push thing at some point, but until we have that this will work. We've been discussing on the team what time we'll have for basket feature work, but a thing this simple should be doable this quarter.
Flags: needinfo?(pmac)
OK, to make things slightly more concrete, I propose POSTing the following sort of data to an /fxa-activity endpoint on basket:

  {
     "activity": "account.login",   // potentially other activity types in future
     "fxa_id":  "ABCDEF123456",     // should match a user already in the db
     "user_agent": "Mozilla/5.0 Gecko/20100101 Firefox/42.0",   // raw user-agent string
     "service": "sync"              // or an oauth relier id like "263ceaa5546dce83" for Firefox Hello
  }

Basket's responsibilities will include parsing the desired information out of the user-agent string, updating the tables in ET, and ensuring that we don't record data about users who are not opted in to the journey stream.

Are there other pieces of information that might be useful to include in the event?  We could send the email as well as the fxa_id, but you *should* already have that mapping inside basket from a previous call to /fxa-register.

This is not an ideal solution, but I think it's a nice incremental step that can get us up and running quickly.

Ben, are you abreast of any potential legal/policy ramifications of collecting this data?  Can you please take an action to get signoff from mika if/as necessary?
On it. Will secure legal sign-off.
Picking back up on this thread. The legal bug is progressing. Meanwhile: we need to talk about timing. It appears that the basket work will have to be just a little bit ahead of the Cloud Services team's work. 

Pmac can you help us understand your team's timing? I know there's a lot going on so for scheduling and planning purposes, how are things looking?
Flags: needinfo?(pmac)
We also should define exactly what we want on the basket side. I like :rfkelly's suggestion for the data they're to send us, but if basket is to parse the user_agent string into data we'll need to know exactly what data output you want, and into what fields it's to go. The ET side will need to be altered to store these data, and then basket will need to grow this new endpoint for FxA to POST to. It's hard to discuss timing before we have requirements. That said I believe we committed to completing this in Q3, and I still believe we can meet that. If we get requirements fleshed-out then FxA and basket teams can work in parallel on this and then we'd just need to coordinate the move to production.
Flags: needinfo?(pmac)
Hi team. I've got a start on the fields we need to pass to a data extension in ExactTarget. 

This is a bit of a wishlist and I'm sure it will need some work, but I thought it would help the Basket track and the Cloud Services track all aim at the same target. 

https://docs.google.com/spreadsheets/d/1e1JUz6OJdSj9dxvVr-N8cM7Ztd8NGPPhD01WdEPEnaY/edit

The question I have: Are we able to assign a unique ID to a particular device? The reason I ask, if all we have is user agent string to go on, it's going to potentially be difficult to distinguish between someone logging in from a new device and someone logging in from the same device with an updated browser / OS. 

So someone who creates an account on Firefox 38 in Windows 8.1. Next week, they log in from Firefox 40 on Windows 10. Is that a new computer or the same computer with updated software? We may have an easy way to solve for this, but it's not readily apparent to me. 

Adding a NI to Edwin, although someone else may need to answer that question.
Flags: needinfo?(edwong)
> The question I have: Are we able to assign a unique ID to a particular device?

Not yet, but it's something we're working on this cycle.  It enables a bunch of different things, including better metrics, better emails on our end, and this work here.

> So someone who creates an account on Firefox 38 in Windows 8.1. Next week, they log in from
> Firefox 40 on Windows 10. Is that a new computer or the same computer with updated software?

Hopefully this is unlikely, as they would have stayed connected to sync throughout the Firefox upgrade process so they wouldn't have to "log in" again.  But it's still possible if e.g. they disconnect or get disconnected from sync, change their password, etc.
Flags: needinfo?(edwong)
Depends on: 1190456
Depends on: 1199174
> I propose POSTing the following sort of data to an /fxa-activity endpoint on basket:

From Bug 1190456 and the linked dataextension doc, it sounds like we'll also need to pass a "is this their first device?" flag, something like:


  {
     "activity": "account.login",
     "fxa_id":  "ABCDEF123456",
     "first_device": true,
     "user_agent": "Mozilla/5.0 Gecko/20100101 Firefox/42.0",
     "service": "sync"
  }
Looks perfect.
One thought :pmac, what will happen if we pass you a record for an `fxa_id` that basket has never seen before?  (For example, because they didn't opt in and their fxa-register event somehow went missing).  Are you able to error out in such case?
Flags: needinfo?(pmac)
:rfkelly, the current plan is to store those as well. We're not planning to check to see whether we've seen the fxa_id before first in this case. This is so that we'll have the data if that person later decides to opt-in. Of course the data will only be used for those users that do have an entry in Master_Subscribers, and have opted in. Does that seem reasonable?
Flags: needinfo?(pmac)
This sounds OK as long as legal are on board with it.  Longer term, I'd like to be in a position where you can query this information directly when the user opts in.  So they opt-in, and basket turns around and says "ok, tell me what devices they have connected" and pulls in the data it needs.  But we're not there yet.
(In reply to Ryan Kelly [:rfkelly] from comment #22)
> This sounds OK as long as legal are on board with it.  Longer term, I'd like
> to be in a position where you can query this information directly when the
> user opts in.  So they opt-in, and basket turns around and says "ok, tell me
> what devices they have connected" and pulls in the data it needs.  But we're
> not there yet.

That will be superior for sure. As soon as we move basket to its new home in our Deis cluster we'll be able to iterate much faster. So hopefully this kind of thing can be a goal for Q4.
The changes are now live on our staging instance of basket (basket.allizom.org). We can begin testing the new endpoint (/news/fxa-activity/) at any point.
(I sent an email with these details, but recording in the bug for posterity)


Thanks to Phil Booth's work over in [1], we have a preliminary dev environment for testing the FxA -> Basket -> JourneyBuilder event flow.

Rather than mucking around in about:config to make firefox use the dev environment, you can go to the following URL to simulate a new device connecting to sync:

  https://jbtester.dev.lcip.org/?service=sync

If you sign up or sign in from that page, it should trigger an event that gets passed to the dev basket instance at:

  https://basket-dev.allizom.org/news/fxa-activity/

And then on to JourneyBuilder, assuming we have that connected in the dev environment as well.

:pmac, can yo please try it out and see if everything flows through properly from Basket's point of view?
Flags: needinfo?(pmac)
Verified! I signed up at the above URL and saw an appropriate entry show up in the sandbox ExactTarget account. I also used an artfully crafted "curl" command to send a similar thing to our staging basket instance (basket.allizom.org) and that also worked. I believe we are good to go for more testing.
Flags: needinfo?(pmac)
Depends on: 1207177
This shipped and the journey is live, closing the bug
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.