817535 - [FTU] Hook up e-mail field in the FTU to Mozilla Basket to actually gather the e-mail entered in the Privacy panel

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::First Time Experience, defect, P1)

Description

[Vivien Nicolas (:vingtetun) (:21) - (NOT reading bugmails, needinfo? please)]

In the FTU one of the screen ask for your email. It is not clear to the user what is supposed to happens when the user enter his/her email here.

Also there is no clean way to store it. MozSettings sounds a bad option to share that and I'm concerned about privacy issue but let's CC'ed some people to see what they think.

Comment 1

[Larissa Co [:lco]]

This is a Branding team requirement. Removing it will need their permission. I thought we had already figured out which database this was going to be stored in? cc'ing Pete Scanlon for Branding feedback and Tom and Alina for privacy feedback.

Comment 2

[Andrew Overholt [:overholt]]

Do we need to translate the "De todos. Para Todos." string as well?  I chose English and everything else is in English but this AFAICT.

Comment 3

[Peter Dolanjski [:pdol]]

(In reply to Andrew Overholt [:overholt] from comment #2)
> Do we need to translate the "De todos. Para Todos." string as well?  I chose
> English and everything else is in English but this AFAICT.

That is what I was wondering as well.  Is there a reason for this?

Comment 4

[Larissa Co [:lco]]

(In reply to Peter Dolanjski from comment #3)
> (In reply to Andrew Overholt [:overholt] from comment #2)
> > Do we need to translate the "De todos. Para Todos." string as well?  I chose
> > English and everything else is in English but this AFAICT.
> 
> That is what I was wondering as well.  Is there a reason for this?

See Bug 816639, which is waiting for input from Branding for more information

Comment 5

[Vivien Nicolas (:vingtetun) (:21) - (NOT reading bugmails, needinfo? please)]

(In reply to Larissa Co from comment #1)
> This is a Branding team requirement. Removing it will need their permission.
> I thought we had already figured out which database this was going to be
> stored in? cc'ing Pete Scanlon for Branding feedback and Tom and Alina for
> privacy feedback.

I don't think this field deserve any purpose at the moment and It makes me worried to store the email of the user into a share database.

Comment 6

[Tom Lowenthal]

This addition was discussed a while ago, and there are no outstanding privacy issues, apart from ensuring that the privacy policy link works.

The page is pretty clear about what it's asking for:

When you use Firefox OS, you become part of a global community helping to build a brighter future for the Web. If you'd like to know more about the Mozilla community, our other products and events near you, please enter your email address below.

It's also totally optional. Vivien, I'm not sure I'm clear about your concerns?

Comment 7

[Larissa Co [:lco]]

I think the question here is where the email list will live. I don't know if anyone's really planned that out.

Peter (or Chris Lee), can you please follow up with Pete Scanlon to see if they've thought this out?

Comment 8

[Peter Dolanjski [:pdol]]

Adding Mary to fill in the details on the use of the email address.

Comment 9

[Mary]

Hi all:  I'm including Winston who heads our email engagement programs - we weren't aware that this idea progressed :) I'd *like* to see this kept in, but we'd also like to ensure that it's done right.  It should connect back into ExactTarget (our email platform). 

I'll let Winston pick this up and add in some details.

Comment 10

[Winston Bowden]

Thanks, Mary. 

If we keep the form, (and agree it would be great to keep), we should integrate with our email database of record.

ExactTarget manages email communication to users who have opted in to learn more about Mozilla products.  You can see how this works on moz.org. 

When a subscriber opts in, we have a system called basket that assigns them a unique token for comm preferences.   That token / contact is then subscribed to a list in our email vendors system.  We’d want to integrate Firefox OS with that system so the subscribers can subscribe/unsubscribe from Mozilla email communications.   

Cmore’s group has managed most of these implementations in the past.  It’s a couple of API calls. 

Is this something that we think our partners will allow us to include?

Comment 11

[Gervase Markham [:gerv]]

Do we actually yet have any back-end support for sending this email address to some Mozilla (or third-party) server at some point when the device has a data connection?

Gerv

Comment 12

[Ben Francis [:benfrancis]]

Triage: Chris, can you confirm the requirements here and if this stays where it needs to be hooked up to?

Comment 13

[Joe Cheng [:jcheng] (please needinfo)]

adding Peter to see if he can provide quick feedback.

Comment 14

[Chris Lee [:clee]]

This was agreed upon with Marketing/Privacy/Branding so we need to keep this in. 

However, it's unclear to me what work (if any) exists on the OS client to hook this up with the ExactTarget database.  Winston, can you comment on what we need to do here to hook this up?

If the work becomes significant, we will have to re-evaluate this feature.  

I also commented on bug 816639 regarding the string translation.

Comment 15

[Peter Dolanjski [:pdol]]

cmore, Winston mentioned that your group has managed these implementations in the past.  Could you provide specifics as to what would be required to do this?

Comment 16

[Chris More [:cmore]]

Hi everyone.

Tying this to ExactTarget should be straightforward. With an API, we can send the rest to ExactTarget to store the email address for future uses. The one thing I want to see that we can do from a FTU experience is not slowing down the interface is ExactTarget is down or slow. We changing our ExactTarget integration to be threaded so that the interface isn't dependent on the responsiveness of ExactTarget. We've had a few instances where the API call slowed down and web server timed out. 

We created a Python library called "Basket" that we use on all of our websites that may need email signup. Basket then makes calls to the ExactTarget API. If we switch email providers, we change Basket, and all websites will remain the same. You can see the API info here:

https://github.com/mozilla/basket/blob/master/apps/news/backends/exacttarget.py

:pmac on my team has most of the experience with Basket and ExactTarget integration. Their API isn't perfect, but it works for what we need it to.

Comment 17

[Larissa Co [:lco]]

Just so you know, when we spoke with Pete Scanlon, he had the idea that the email list generated from this screen would be separate from the newsletters we send out. I don't know if that's changed. I'm emailing him to make sure he's aware of the discussion on this bug.

Comment 18

[Winston Bowden]

I think the actual content/campaign piece can be figured out later.  The key thing I wanted to convey is it'd be great to get the emails stored in our primary email engagement database.  It might not be a newsletter - it could be more transactional in nature.  But by including them in ExactTarget, our subscribers will be able to control all their Mozilla email settings in one place.  That's the overarching goal.

Comment 19

[[e-mail]]

Thanks Larissa.  I'm OK with however Winston would like to track the addresses since he oversees the outbound communication thru email.  Pete

Comment 20

[Josh Carpenter [:jcarpenter]]

This is marked RESOLVED WONTFIX. Is that accurate?

Comment 21

[Larissa Co [:lco]]

Who marked it as "wontfix"? I don't know if a decision was made with Pete being involved.

Comment 22

[[e-mail]]

Thanks all.  I appreciate the concerns.  Agree with Matej, John and similar voices that we can remove the headline from the second screen.  Pete

Comment 23

[Larissa Co [:lco]]

Pete, this is regarding removing the email field itself, not the headline on the screen (that's a separate bug, which I believe is fixed already)

Comment 24

[Josh Carpenter [:jcarpenter]]

(In reply to Josh Carpenter [:jcarpenter] from comment #20)
> This is marked RESOLVED WONTFIX. Is that accurate?

Looks like it was David. Should this block release? And if so, is anyone working on it? We can't have an email entry not actually connected to anything. We should implement or remove, and make that call asap.

Comment 25

[Fernando Campo (:fcampo)]

As far as I know this is not connected to anything, and there's no plans in short time for it (no bug opened even, apart from this one)

I can take care of the implementation, but the decision of what to do is up to you guys, connect or remove.

Comment 26

[Winston Bowden]

Engagement would like it to be connected to our email system if at all possible.  Not sure who makes the final call as to whether or not that's plausible given deadlines/timing.

Comment 27

[Chris Lee [:clee]]

Comment 16 sounds like the info we need to drive this bug forward.  

I've marked this a P1, bb+ and assigned to C3.

David, can you assign an engineer to own this work?  We started this feature and never completed it and this was an agreed upon requirement.

Comment 28

[Vivien Nicolas (:vingtetun) (:21) - (NOT reading bugmails, needinfo? please)]

(In reply to Chris More [:cmore] from comment #16)
> Hi everyone.
> 
> Tying this to ExactTarget should be straightforward. With an API, we can
> send the rest to ExactTarget to store the email address for future uses. The
> one thing I want to see that we can do from a FTU experience is not slowing
> down the interface is ExactTarget is down or slow. We changing our
> ExactTarget integration to be threaded so that the interface isn't dependent
> on the responsiveness of ExactTarget. We've had a few instances where the
> API call slowed down and web server timed out. 
> 
> We created a Python library called "Basket" that we use on all of our
> websites that may need email signup. Basket then makes calls to the
> ExactTarget API. If we switch email providers, we change Basket, and all
> websites will remain the same. You can see the API info here:
> 
> https://github.com/mozilla/basket/blob/master/apps/news/backends/exacttarget.
> py
> 
> :pmac on my team has most of the experience with Basket and ExactTarget
> integration. Their API isn't perfect, but it works for what we need it to.

Can you clarify a little bit. What is ExactTarget? If this is a kind of database you have somewhere can we simply send a XMLHttpRequest to a specific url? If yes, who can provide the URL? Also what is supposed to happen if there is no internet connection at this point? (Storing it in settings sounds evil since many apps will be able to access it and this is not for what is designed mozSettings).

Comment 29

[Chris More [:cmore]]

> Can you clarify a little bit. What is ExactTarget? If this is a kind of
> database you have somewhere can we simply send a XMLHttpRequest to a
> specific url? If yes, who can provide the URL? Also what is supposed to
> happen if there is no internet connection at this point? (Storing it in
> settings sounds evil since many apps will be able to access it and this is
> not for what is designed mozSettings).

ExactTarget is the 3rd company/service that Mozilla uses for all of their user email storage, email campaigns, and email newsletters. ExactTarget has an API that we can interact with and it is how our websites HTTP push/pull updates to their system.

Docs:

http://docs.code.exacttarget.com/

https://webservice.s4.exacttarget.com/etframework.wsdl

What about storing it in localStorage and have a process that checks to see if requests are queued? Maybe localStorage isn't relevant here given the FTU vs a webpage. Offline is pretty tricky in this use-case. I wouldn't want the FTU to have to wait for a response from an API if internet is not available.

Probably the best way is to have FTU interact with our Mozilla Basket service since if we ever change from ExactTarget to another email provider, we just change Basket and all Mozilla properties/products don't need to update.

Mozilla Basket: https://github.com/mozilla/basket

Mozilla's internal ExactTarget owner is: Jessyln Davis
Mozilla internal Basket webdev is: Paul McLanahan

Comment 30

[Tim Guan-tin Chien [:timdream] (please needinfo)]

Update title, see comment 27 and comment 16.

Comment 31

[Michał Budzyński (:michalbe)]

Is it already somewhere online? Or we should install in somewhere on our own for now and mock the address in FTU form?

When we don't have a connection and store emails in localStorage, when is a good moment to send them to the db? Is it possible to do it even if FTU has been already closed?

Comment 32

[Michał Budzyński (:michalbe)]

Of course my first question is about the Basket.

Comment 33

[Dietrich Ayala (:dietrich)]

(In reply to Michal Budzynski (:michalbe) from comment #31)
> When we don't have a connection and store emails in localStorage

Eeek. Please use DJF's asyncStorage. IIRC there's a no-localStorage rule for all of Gaia.

Comment 34

[Chris Lee [:clee]]

What's the latest status here?  Do we need more info or is the info in comment 29 sufficient?  Thanks.

Comment 35

[Michał Budzyński (:michalbe)]

Thanks :dietrich, but that's still not answer for

Comment 36

[Michał Budzyński (:michalbe)]

According to :pmac we need to create new campaign & newsletter name for Basket. Winston, are you the right person to do it?

Comment 37

[Winston Bowden]

I am.  Just set the following up:

Field name:  FIREFOX_OS
- When user opts in to email, set to Y on Master_Subscriber table in ExactTarget

Triggers campaign ID
- Campaign name:  firefox_os_double_optin
- ID: 13513

Comment 38

[Winston Bowden]

also filing a bug with pmac to get this set up in our basket system.

Comment 39

[Winston Bowden]

apologies - we'll want to use double opt-in which is a different table.  Updated instructions below:

Field name:  FIREFOX_OS
- When user opts in to email, set FIREFOX_OS to Y on Double_Opt_In table in ExactTarget

Triggers campaign ID
- Campaign name:  firefox_os_double_optin
- ID: 13513

Comment 40

[Dylan Oliver [:doliver]]

Michal, can you provide an update on status of this bug?

Comment 41

[Michał Budzyński (:michalbe)]

I'll push Basket Client and support of this functionality in about an hour, but we are still blocked by #826292.

Comment 42

[Dietrich Ayala (:dietrich)]

Can you land this now anyway?

Comment 43

[Michał Budzyński (:michalbe)]

But then we will subscribe to some of the other mozilla's newsletters (we need the existing newsletterID). I can land it someone will approve that.

Comment 44

[Vivien Nicolas (:vingtetun) (:21) - (NOT reading bugmails, needinfo? please)]

(In reply to Michal Budzynski (:michalbe) from comment #43)
> But then we will subscribe to some of the other mozilla's newsletters (we
> need the existing newsletterID). I can land it someone will approve that.

Maybe we can land your patch (which does nothing if there is no newsletterID) and open a followup to add it so the main code will have landed?

Comment 45

[Michał Budzyński (:michalbe)]

Ok, I'll send it with fake newsletterID.

Comment 46

[Michał Budzyński (:michalbe)]

Attachment: patch

Comment 47

[Staś Małolepszy :stas]

Comment on attachment 699252 [details]
patch

Technically this looks good l10n-wise.

Copy-wise, I'd question the use of the exclamation mark in the success message.  The error title is also very long, and I'd make it shorter and I'd move some of its wording to offline-newsletter-dialog-text.

Lastly, shouldn't we check for `WifiManager.isConnected || DataMobile.isDataAvailable` *before* we let the users type in their email addresses?  Maybe check it when they arrive at this FTU screen?  Or when they focus the input field?

Comment 48

[Francisco Jordano [:arcturus] [:francisco]]

Comment on attachment 699252 [details]
patch

Great job!

r=me

Comment 49

[Michał Budzyński (:michalbe)]

Thanks!

Comment 50

[Staś Małolepszy :stas]

The diff has changed since the time I reviewed it.  Sorry to be a spoilsport, but please use ellipsis (…) instead of three dots.

Comment 51

[Vivien Nicolas (:vingtetun) (:21) - (NOT reading bugmails, needinfo? please)]

(In reply to Staś Małolepszy :stas from comment #50)
> The diff has changed since the time I reviewed it.  Sorry to be a
> spoilsport, but please use ellipsis (…) instead of three dots.

You're right. I have landed a followup: https://github.com/mozilla-b2g/gaia/commit/4fa2d32113dd39b8d8103e7dbeeffcbb5c9578ce

Michal can you also open a followup for using the right newsletterID (and I believe it should be a blocking-basecamp+ as well) ?

Comment 52

[Vivien Nicolas (:vingtetun) (:21) - (NOT reading bugmails, needinfo? please)]

I will let this issue opened until you open a followup.

Comment 53

[Vivien Nicolas (:vingtetun) (:21) - (NOT reading bugmails, needinfo? please)]

Winston do you know if the newsletterID is the ID you linked in comment #39? Otherwise can you provide an ID in advance?

Comment 54

[Winston Bowden]

Hi Vivien - yes, the FirefoxOS newsletter is FIREFOX_OS.  That field should be set to Y after opt-in.

Comment 55

[Staś Małolepszy :stas]

(In reply to Staś Małolepszy :stas from comment #47)
> Copy-wise, I'd question the use of the exclamation mark in the success
> message.  The error title is also very long, and I'd make it shorter and I'd
> move some of its wording to offline-newsletter-dialog-text.

The copy guidelines also recommend to forgo ending punctuation in titles unless it's a question mark.

Josh, what is UX's opinion on this?

For reference, here are the strings in question:

email-successfully-added=Email successfully added!
offline-newsletter-dialog-title=You must be connected to the Internet to subscribe to the newsletter.
offline-newsletter-dialog-text=Please go back and connect to the Internet.
invalid-email-dialog-title=Your email is not valid
invalid-email-dialog-text=Please go back and correct your entry.

> Lastly, shouldn't we check for `WifiManager.isConnected ||
> DataMobile.isDataAvailable` *before* we let the users type in their email
> addresses?  Maybe check it when they arrive at this FTU screen?  Or when
> they focus the input field?

Should I file a follow-up?

Comment 56

[Josh Carpenter [:jcarpenter]]

These strings are quite out of step with the copy guidelines (all the uses of "you", for starters), but there's no time to tweak for v1. Let's remove the exclamation point and we can revisit the rest later on.

Comment 57

[Staś Małolepszy :stas]

(In reply to Josh Carpenter [:jcarpenter] from comment #56)
> These strings are quite out of step with the copy guidelines (all the uses
> of "you", for starters), but there's no time to tweak for v1. Let's remove
> the exclamation point and we can revisit the rest later on.

Filed bug 828332.

Closing this bug as michalbe fixed the exclamation point in https://github.com/mozilla-b2g/gaia/commit/76aa31d2381cfc7191c06fcdf4c7c38f2c11829f.

Thanks!  We can now migrate these strings to HG for the localizers to translate them.