Closed Bug 1492604 Opened 7 years ago Closed 6 years ago

CRM/Email Marketing Request: IRL Season 4 Emails

Categories

(Marketing :: Email, task)

All
Other
task
Not set
normal

Tracking

(Not tracked)

VERIFIED FIXED

People

(Reporter: mthayer, Assigned: lkashdan)

Details

>> Mozilla Team/Department Mozilla Global Brand Team >> What kind of relationships are you looking to develop? A relationship with Mozilla supporters and Firefox users who are interested in taking back their power online and learning how to do so by listening to IRL Season 4 episodes. >> How will you nurture these relationships? Email messages to those who are interested (as decided through list segmentation). >> How will success be measured? Podcast downloads and subscribers. Brand perception increase. >> What is the risk to not doing this? Less conscious choosers listening to IRL --> less conscious choosers understanding Mozilla --> less conscious choosers choosing our products and services that make the Web better. >> What's the timeline for launch? The timeline for IRL episode releases is below. We'd love for emails to go out the day of each episode release. (If that's not possible for a given episode, we'll be happy to take another day later in the week of the episode's release.) November 12: Episode 1 -- Frenemies with Big Tech November 26: Episode 2 -- Online Shopping December 10: Episode 3 -- Passwords & Data Breaches January 7: Episode 4 -- How the Web Has Changed Reading January 21: Episode 5 -- Content Moderation February 4: Episode 6 -- Economic Inclusion February 18: Episode 7 -- Deep Fakes We'll commit to getting you email copy a week before each episode launch date and will post it here in this bug. >> Any other info we should know? Thank you! Let me know if you have any questions, concerns, or opportunities to highlight.
Thanks, Michaela! Jesse/Luke - can one of y'all get placeholders on the email calendar for these IRL emails? We'll stay tuned for more info from you WO Nov 5th and pick up thoughts on questions/concerns/opportunities when it's time to prioritize this story in the upcoming sprint. Thanks!
Hey Jess, I'll put these emails on the calendar. Thanks, Luke
Hi Luke and team, One quick update. We've decided to launch a season trailer on the 12th of November, rather than the first episode. The slightly tweaked schedule is below: November 12: trailer — ethical tech November 26: episode 1 — online shopping December 10: episode 2 — passwords January 7: episode 3 — reading January 21: episode 4 — content moderation February 5: episode 5 — economic inclusion February 18: episode 6 — deep fakes March 4: episode 7 — frenemies with big tech Let me know if this works for you and your team! Michaela
Morning everyone! Just checking in on the above schedule for IRL Season 4. I plan to submit copy and imagery 1 week before each episode launch (including the launch of the trailer). One note: I'll have slight copy changes to the "What is IRL" footer for Season 4 as well. Let me know if this works! Michaela
Good morning! Thank you for checking in! And thank you for putting these sends on the calendar. We have Season 4 on our radar. I recommend that all of us to sync up in real life (see what I did there?) before the kickoff of next season to review the stats from Season 3, talk about incorporating learnings for Season 4, make sure we're in alignment on the send calendar (EOY Fundraising + Fx Accounts + Firefox audience sends), production, audience segmentations, etc. I'm inviting the crew (you, me, Brad, Michele, Daniel, Luke, Jesse, Will & Lauren) to our LCM intake meeting on Nov 5th, 9:15am PT in Michele's vidyo, so we can sync! \o/ Jess
Perfect, talk then!
One quick note that is probably useful: the episode launch dates are staying as I wrote them in Comment 3. But, three of those dates fall on a Monday holiday. Out of respect for people celebrating those holidays (including us Mozillians), I recommend that emails go out a day later for those releases: - The trailer email go out on Tuesday, November 13 (rather than the 12th) due to Veterans Day. - The episode 4 email go out on Tuesday, January 22 (rather than the 21st) due to MLK Day. And; - The episode 6 email go out on Tuesday, February 19 (rather than the 18th) due to Presidents Day. LMK what you think. Michaela
Hi all! Just met w/ Stephanie, Luke, and Jesse at the LCM intake meeting. Sounds like next week isn't great for the trailer email send, but that the 19th is open. We are flexible and would be happy to secure the 19th. I'll go ahead and change on the email calendar, but let me know if there are any concerns. Meanwhile, I'll work on the email copy and should have it ready by EOD today. @Jess, could we reschedule the LCM intake meeting to talk about IRL Season 4 overall? You'll see in my suggested notes in the agenda that we've got a Season very aligned with Firefox, Firefox features, and Firefox values, and I'd love to talk through opportunities to share IRL more with the Firefox audience as well. Thanks, Michaela
Flags: needinfo?(jdavis)
Hi all — here is copy for the trailer email: https://docs.google.com/document/d/1WxXvfjgXaJZ2hMCSllwtjX7iSsFJ9bt5nIwVCsrPXbA/edit#. Let me know if you have questions or concerns!
Morning team. Just quick check in! Good to go on the IRL Season 4 trailer email launching a week from today on the 19th?
Hi team — met w/ Michele and Lauren today (and Brad, and Janis, and Daniel, and M.J.), and here's what the reco is: - IRL stand alone emails still go to the Mozilla list for Season 4 - IRL then gets incorporated into all-Firefox emails (but not as stand alone) when thematically appropriate moving forward - If in a special case, we think IRL needs a stand alone email to the Firefox list, we'll request it ad-hoc We're good to move forth w/ the trailer email send on Monday to the Mozilla list (because our team is moving a newsletters set for this month to January), so that we can reduce volume. I think that captures it. LMK if this makes sense! Michaela
Thanks, Michaela! We've got it on our board and will get the Trailer email ready for a Monday send to the Mozilla audience. Historically we've sent IRL Season emails to a subset of Mozilla subscribers who have engaged with past IRL emails. Given that this is the launch of the 4th season, I recommend it goes to the full Mozilla list. RE: MIH newsletters for 2018 - We'll close the other newsletter requests and update the calendar. Keep us posted on the new requests when you're ready to explore January dates. Thanks!
Assignee: nobody → lkashdan
Flags: needinfo?(jdavis)
Will do!
Hey Michaela, Are we planning to link to the podcast page? There is no link in the copy doc. Thanks, Luke
Flags: needinfo?(mthayer)
Yep! Those SmartURL links in the comments will go to the page.
Flags: needinfo?(mthayer)
The email is set to deploy in the next couple minutes. Thanks, Luke
Thanks, Luke! And, here is copy for the email for Cyber Monday, November 26: https://docs.google.com/document/d/1WxXvfjgXaJZ2hMCSllwtjX7iSsFJ9bt5nIwVCsrPXbA/edit#. LMK if you have questions or concerns. And, if I don't talk to you prior, Happy Thanksgiving! Michaela
Hi Luke and team, Here is copy for the Password episode of IRL email, set to launch on December 10: https://docs.google.com/document/d/1WxXvfjgXaJZ2hMCSllwtjX7iSsFJ9bt5nIwVCsrPXbA/edit#. Copy is final. We're awaiting the image. I'll add that to the doc once I have it (at the latest, Monday). Let me know what you all decide for the audience for this send. Thanks as always, Michaela
Good morning, The header image for the December 10 email is now in the copy doc: https://docs.google.com/document/d/1WxXvfjgXaJZ2hMCSllwtjX7iSsFJ9bt5nIwVCsrPXbA/edit#. Have a great start to your Monday! Michaela
Hi Luke and team, Here are IRL email stats in reverse chronological order, where you can see the impact of IRL emails on the Mozilla and Firefox lists across the last 6 episodes: https://docs.google.com/document/d/1pdPGKLMctVIrtrqOxNQT-LYGd89CpWtFWjqzjOU8vxE/edit. (More detail in the email I sent this morning!) Looking forward to your thoughts and recommendations on audience segmentation as we move forward. Happy to push our December 10 IRL send if audience segmentation still needs to be nailed down. Let me know. Thanks, Michaela
Hi Michaela - I just sent over proofs of the December 13th S4E2 Mozilla and Firefox emails for review. Please let us know if there are any updates or changes that need to be made. Thanks! Brynne
The email has deployed. We'll have stats on performance early next week. Thanks, Luke
Hi team, Here is copy and imagery for the IRL email set to go out on January 7 (all about how the internet has changed reading): https://docs.google.com/document/d/1WxXvfjgXaJZ2hMCSllwtjX7iSsFJ9bt5nIwVCsrPXbA/edit#. Let me know if you have questions, concerns, or opportunities you'd like to share! Michaela
Thanks Michaela, coding and sending proofs today (12/27/18).
(In reply to jessekleinsasser from comment #24) > Thanks Michaela, coding and sending proofs today (12/27/18). Thank you, Jesse! I actually don't see the proofs in my inbox. Possible to send again?
Michaela and I spoke and proofs were not sent 12/27. Proofs will now be sent by EOD 1/3.

Hey all. Apologies for the late post on this, but the email was successfully launched last week and A/B test results will be emailed today. Stay tuned!

No worries, Jesse! Looking forward to seeing results.

I'll also ping here later today with copy for next week's IRL email. It's just being reviewed now.

Hi team,

Here is copy for next week's IRL email all about content moderators: https://docs.google.com/document/d/1WxXvfjgXaJZ2hMCSllwtjX7iSsFJ9bt5nIwVCsrPXbA/edit#.

Scheduled date to send: Tuesday, January 22.

Let me know what you think, and thanks!
Michaela

See below for A/B test results from the IRL S4E3.

We tested the following subject lines for both email sends:

Subject Line A - Your "I will read more this year" resolution
Subject Line B - Are you a ‘subject-line-only’ reader?

In both the Mozilla and Firefox tests, Subject Line A won.

Subject Line A

MOZ Open rate: 54.9%
MOZ Click rate: 1.3%

FF Open rate: 45.7%
FF Click rate: 1.75%

Subject Line B

MOZ Open rate: 44.7%
MOZ Click rate: 1.2%

FF Open rate: 36.9%
FF Click rate: 1.6%

Thanks for the copy Michaela.

Hi Michaela! I just sent over proofs of next week's (1/22) IRL emails for both Mozilla and Firefox. Please let me know if there are any updates that need to be made, or if everything looks good.

Thanks!

Brynne

Hi Michaela - The emails (Mozilla + Firefox) for S4E4 are scheduled to send first thing Tuesday (1/22/19) morning. Will report back on performance later next week :)

Brynne

Hi Michaela - I just sent over an email with results from this week's S4E4 send, and am posting high level stats here in the Bug as well:

Mozilla Audience

A/B Test Results

Subject Line: Inside the lives of content moderators

  • Sent: 15,472
  • Open Rate: 53.1%
  • Click Rate: 1.3%

Subject Line: Who is your social media feed moderated by?

  • Sent: 15,472
  • Open Rate: 42.2%
  • Click Rate: 1.1%

Remaining Send Results (w/ Winning Subject Line)

Subject Line: Inside the lives of content moderators

  • Sent: 116,472
  • Open Rate: 53.4%
  • Click Rate: 1.2%

Firefox Audience

A/B Test Results

Subject Line: Inside the lives of content moderators

  • Sent: 149,432
  • Open Rate: 43.1%
  • Click Rate: 1.2%

Subject Line: Who is your social media feed moderated by?

  • Sent: 149,432
  • Open Rate: 33.7%
  • Click Rate: 1.1%

Remaining Send Results (w/ Winning Subject Line)

Subject Line: Inside the lives of content moderators

  • Sent: 1,124,315
  • Open Rate: 43.2%
  • Click Rate: 1.3%

Woah - this is lovely news. Thank you Brynne and team on always making us better testers and learners! Insights are really paying off.

Hi team —

Here is copy for next week's IRL email, all about the surveillance economy: https://docs.google.com/document/d/1WxXvfjgXaJZ2hMCSllwtjX7iSsFJ9bt5nIwVCsrPXbA/edit#.

Let me know if you have questions, concerns, or opps to raise!
Michaela

Thanks for this Michaela.

Hey y'all,

Proofs will be in your inboxes within the next half hour for the IRL S4E5 email going out next Monday, February 4th.

Note that Michaela has approved both the Firefox and Mozilla versions of the campaign.

Brynne, ready for you to QA by EOD tomorrow.

Hi Jesse - I've QA'd the emails and provided feedback in the doc here: https://docs.google.com/spreadsheets/d/1ByoLWBZY_BqH4Fr6SgxQOLNOJrnbVAgIeQlNERaz6bE/edit#gid=1651253253

Let me know if you have any questions on my comments.

Brynne

Hey Michaela,

The MOZ and FF version of the IRL S4E5 email has been set up for launch this morning.

I will send email engagement stats early next week.

Hi team,

Here is copy for the IRL email set to launch on Tuesday next week: https://docs.google.com/document/d/1WxXvfjgXaJZ2hMCSllwtjX7iSsFJ9bt5nIwVCsrPXbA/edit#. Next week's episode is all about decentralization. It will be the first time we've talked about this issue to consumers, and we're excited to see the response.

You'll see that I've included two images (one with the >> graphic and one without), so that we can see if that addition helps to generate more clicks.

Let me know if you have questions, and thank you as always.
Michaela

Hi Michaela,

Thanks for the copy.

Hi Michaela,

We wanted to confirm which of the subject lines you'd like to use since we'll be running a header image test for this send instead of our regular subject line test, based on the two graphics you provided.

I'd recommend "Imagine an Internet Free from Corporate Goliaths" since you talk about decentralization in the pre-header text already, and I think they pair nicely together.

Feel free to comment here or update the copy doc - thanks!

Flags: needinfo?(mthayer)

Hi Lisa,

Thank you. I'm on the fence. I think you're right that the corporate goliath subject line nicely comes on the heels of the last IRL email (and is probably more consumer friendly), though I'm also so interested to see if people are curious about decentralization. It's become a bit of a buzz word recently in consumer press without much explanation.

Is there a way to build the testing send to test both the subject line and the image CTR? Or, maybe just test the subject line in the testing send, and then we do the image test in the larger send – one for the Moz audience and one for the Firefox audience?

Do all of these equate to double the labor, or is there a way around that?

If it's double the labor, I stand down — the corporate goliath subject line is just fine to use.

Let me know and thanks again.
Michaela

Flags: needinfo?(mthayer)

Hi Michaela,

We can run one test for the Moz audience and one for the Firefox audience, that's certainly doable without requiring much more added time.

Let me know if you have a preference, but otherwise we'll plan to run the subject line test on the IRL-engaged Mozilla audience to see how your normal crew reacts to the decentralization buzzword, and the header image test on the Firefox audience (who tends to have a higher click rate anyway and will give us more data for that.)

Thanks!
Lisa

Plan sounds excellent. Thanks so much.

Hi all,

Below are the results for the IRL S4E5 email campaign that was sent Monday 2/4/2019.

A/B Test Insights

Below were the subject line versions we tested for this campaign:

Subject Line A: The Surveillance Economy
Subject Line B: Our economy: powered by surveillance?

For both the Firefox and Mozilla audiences, Subject Line A was the overwhelming winner:

Firefox Audience
Subject Line A: 46.6% open rate
Subject Line B: 35% open rate

Mozilla Audience
Subject Line A: 58.8% open rate
Subject LIne B: 45.8% open rate

Campaign Results (Totals)

Firefox Audience
Total Sent: 1,559,870
Open Rate: 46.15%
Click Rate: 1.88%

Mozilla Audience
Total Sent: 143,815
Open Rate: 57%
Click Rate: 1.74%

Thank you! Interesting that the non-question subject line won (even though they said the same ish thing). I think I'll change one of the question subject lines in the next email for decentralization (both currently are questions). Will do now...

Hi Michaela,

I just sent you an email, but wanted to comment here too.

Both of the A/B tests (hero image test for FF audience and subject line test for Moz audience) are setup and set to run on 2/19 at 8 am PST, winner sending at 10 am PST.

Thanks,
Chris

Hi Michaela,

Below are the results for the IRL S4E6 email campaign that was sent Tuesday 2/19/2019.

A/B Testing Insights

Below are the hero image versions we tested for the Firefox email:

Version 1: Image A: (Sans >> symbol)
Version 2: Image B: (With >> symbol)

For the Firefox audiences, Version 2 (with >> symbol) was the winner with the highest CTR:
Firefox Audience

Version 1: 1.27% CTR
Version 2: 1.31% CTR

The different hero images seemed to have a negligible difference in CTR. Looking at the click map of the email, the hero image with the >> symbol tallied 10.4% of clicks in the email compared to 10.3% of the hero image without the >> symbol.

For the Mozilla audiences, below are the subject lines tested:

Subject Line A: An Internet Free from Corporate Goliaths
Subject Line B: Decentralization: the Antidote to Big Tech

Subject Line A was the winner at the end of the two hour A/B testing period and was sent to the rest of the audience, but a few days later, when pulling results, Subject Line B had a marginally higher open rate:

Subject Line A: 53.06% open rate
Subject Line B: 54.06% open rate

A possible takeaway here is that our audience has trending interest in learning more about decentralization and the topic could bode well with engagement if expanded on in future sends.

Campaign Results (Totals)

Firefox Audience

Total Sent: 1,526,021
Open Rate: 41.84%
Click Rate: 1.27%

Mozilla Audience

Total Sent: 140,650
Open Rate: 52.91%
Click Rate: 1.15%

Hi all, here is the copy for the IRL Season 4 Ep7 email. It's scheduled to go out 3/4/2019.

https://docs.google.com/document/d/1WxXvfjgXaJZ2hMCSllwtjX7iSsFJ9bt5nIwVCsrPXbA/edit

Please let me know if you need anything else and I'm psyched to see the tests.

Daniel

Thanks for the copy Daniel!

Hi Michaela,

I sent you email campaign results for the S4E07 email and since that was the last email of the season, I will be closing this bug.

Thanks!

Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED

We'll keep an eye out for Season 5 movement! \o/ Thanks, Michaela

Status: RESOLVED → VERIFIED

Great, congrats everyone on the season! Email is a major driver for IRL and a recent listener survey is showing us that it is impacting not only brand perception, but also browser choice by up to 65%. More to come on that, but major thanks!

You need to log in before you can comment on or make changes to this bug.