1617257 - Does the average user find an email relay service useful?

Status: RESOLVED FIXED

(User Research :: Consultation, task, P2)

Description

[MarniePW [:marnie]]

Request Description
Does the average user find an email relay service useful? Do they understand email relay/email masking? If they don't understand it, they won't find it useful.

Deadline
Next week; this is being driven by Wennie and a desire to get this experiment out as soon as possible. We're hoping this could simply be a guerilla research effort (3-5 folks, in office) on the light side; or usertesting on the heavier side.

Priority Level
(1 = High 2 = Medium, 3 = Low): 1

Priority Level Description
We have an email relay experiment that we'd like to push out into the wild. We'd like to test that folks actually understand what we're offering.

Supporting Information
https://docs.google.com/document/d/1rJI8orhfnyW_QNAtAWtqd-PB3T-L02vZTq-Jpg4C6nM/edit#

Comment 1

[MarniePW [:marnie]]

Sounds like this may no longer be needed; the team met with Alice on Friday. She pointed them to existing documentation from earlier studies. I suspect this ticket can be closed.

Comment 2

[Sandy Sage]

For posterity, here is the content of that followup:

Q1: Do we have evidence that the Private Relay project addresses a real user problem?

A: Yes. From internal user research, the core user problem that the Private Relay product addresses is as follows:

“I want the benefits that come along with giving out my email (like access to using a website, coupons, etc.), but I’m nervous about giving out my email because…”

"…I don't know if it's safe with the company -- are they going to sell my data? Even if they don't sell my data, the company could get hacked and then my information will be out there..."

“We submitted our name and email information to win a gift card at Ikea, I wondered if our information would be sold, I felt frustrated but decided to put it anyway because I wanted to try and win!” [1]

“I wanted to sign up for updates [to a political campaign] and of course you need to think what will happen to the information? Will this be stolen by someone and used for nefarious purposes later?” - DTW_P5 [1]

"…I don't know if the company is going to use it to spam me with a bunch of messages"

“I’d want to know, why would I need an account? Is it for the next time you log in? I wouldn’t sign up unless it stopped me from accessing Wi-Fi. I don’t want to get junk mail.” [2]

“If you can get rid of spam emails--! I would pay for not an ounce of spam and not blocking anything that needs to come through. Same with my phone: I want my phone back! Verizon gave me a call filter and it only works once every 10 calls!” - DTW_P7 [1]

“…I might get potential malware and phishing attempts in my inbox.”

“When I accept some of my email, I feel that I never know if I will be subject to a virus or some bad thing to damage my computer.” [1]

“Every time I go through my email’s junk mail to search for any emails that might have been sent there by accident, I’m always nervous that I might accidentally click on one that might download a virus to my iPhone/desktop. I just highlight the junk/spam email and group delete them.” [1]

We also have signal from internal research that the actions users currently take to protect themselves online mimic what Private Relay offers—that is, creating fake email addresses to shield their primary email address from spam and junk.

P1 has a separate email address she calls her ‘slush’ email just for junk after signing up for things. [3]

 “My [non-Gmail] recovery email is not well protected. It’s all spam, but it’s not real spam, it’s coupon spam. I use Gmail for other secure stuff like our bank. I let those [sensitive] tax documents hang out there [in Gmail instead].” - SLC_P3 [1]

These data points all come from user research studies conducted within the past year.

[1] Oct 2019 - Field research conducted on perceptions of online/offline protections: https://docs.google.com/presentation/d/1edSnw9Y2JPWipQKW2qfBxHLUwvlGGl_oG7DCMXkgoCg/edit#slide=id.g6b87220860_4_702
[2] May 2019 - Concept testing conducted on Secure Proxy: https://docs.google.com/presentation/d/11PyvHCPGtlFKoZXZzMW_gnVMw_uZDR1G81FvvDYNuak/edit#slide=id.g59fa2fc882_0_59
[3] May 2019 - Field research conducted on Firefox Accounts: https://docs.google.com/presentation/d/10PVQzp5Fwxt5MN3Z6hEHbqVbp7M7dI6awFBlYB_-01A/edit#slide=id.g5acdebc2b9_0_103

Q2: Do we have evidence that users will understand what this is?

A: Yes, we have signal from very similar in-market products that this is worth building and has real, understandable value for users—enough value that users are willing to pay for a subscription. There are several in-market private relay analogies that we can look at here. From largest/smallest:

1. Apple's Hide My Email feature, which is built into their core "Sign in with Apple" flow

2. Burnermail - https://burnermail.io/ - includes a paid subscription service, has 27k users, good/wide press coverage. Customer reviews are glowing.

From a Burnermail user I [Alice] reached out to: "I got privacy interested last year and wanted to stop giving my email to random sites. I like it. There are a couple UI / usability issues that could be better, but overall it does the job well. One of the things I like the most is that sometimes when you Unsubscribe from emails, it seems like companies find a way to keep sending you emails anyways. In Burnermail I can just toggle that email address off and I’ll never see an email again because the company is just sending emails into the void."

"I love the fact that it helps me focus just that much more on what I'm doing instead of having think of a email for a site I just want something from quickly. Ty”

3. AnonAddy – https://anonaddy.com/ - a much smaller paid subscription service which does the same thing. Reviews are also glowing.

"It's been interesting to see which websites that I used my anonymous credentials for have sold/passed them onto other companies, turns out it's quite common. The additional privacy and security contributes to peace of mind when submitting personal email info knowing it's now totally anonymous."

Note that as a potential research input into the design of our own prototype, we could conduct competitive research with participants who use these products to understand more deeply what about these current implementations work well/not well for them.