996695 - Track All Webmaker sign-up forms (BSD) as metrics goals

Status: RESOLVED INCOMPLETE

(Webmaker Graveyard :: Metrics, defect, P2)

Description

[Adam Lofting (:adamlofting)]

Mentor sign-ups feels like a high-value 'action' to me, so I would like to surface these numbers along with some other top-level goals for webmaker.org (things like adding an event, create an account etc).

That way we can reference the numbers and work on ways to grow them. 

Currently, clicking from here to sign-up >> https://webmaker.org/teach

Goes to our BSD powered 'data-capture' site >> https://sendto.mozilla.org/page/s/mentor-signup

These sites use separate Google Analytics profiles which means the data is not connected.  

--- 

There are many ways we could collect this data, and I suspect many ways we can increase the number of people signing up to be mentors. 

---

Let's discuss how this could work and then open the necessary bugs.

My thoughts:

1) Could this action be better captured using a webmaker account. E.g. add a 'mentor-signup' flag to their record rather than asking them to fill out another form on a separate system?
2) If not, can the form better say what it does and what people get by signing up?

---

@Matt, @Michelle - who else should be looped into this?

Comment 1

[Matt Thompson (@OpenMatt :OpenMatt)]

* Adam: thanks for filing this. You're right -- it's needed. But I'm going to suggest revising the scope:

* We should track *all* Webmaker-related email sign-ups. Not just "mentors" -- especially since *all* Webmaker users are essentially "mentors" now. We need to simplify and streamline accordingly, but we haven't done that yet. Let's start on it.

* This ticket is aimed at gathering the initial documentation needed. To get a complete picture of how we're already gathering sign-ups and segmenting in BSD: 
https://bugzilla.mozilla.org/show_bug.cgi?id=992253

* Once we have that we can then look for opportunities to streamline and simplify these various pages and corresponding BSD segments. And also simplify how we count them, as per this ticket. 

* That separate sign-up page on /Teach is a hold-over from a previous era -- where we thought of "mentors" as a sub-set of a larger Webmaker audience. The target audience strategy has changed -- today, *all* Webmaker users are essentially "mentors." There's no other kind. 

* But we never got around to squaring that thinking with how we're segmenting the Webmaker portion of our larger BSD mailing list. We should do that this quarter. 

* ANDREA: does that make sense, and can you help with 992253?

Comment 2

[Adam Lofting (:adamlofting)]

Thanks Matt, the history make's sense. And that's a better way to divide the tasks here.

I'd like to challenge the idea that all users are mentors, though... 

I think our 'target audience' and the people we want to communicate with are mentors, but the webmaker users (people with accounts) also include learners/students/randoms. And being able to work out which are the teachers will be really helpful. 

Related to this pondering here: http://adamlofting.com/1037/whos-teaching-this-thing-anyway/

Comment 3

[Matt Thompson (@OpenMatt :OpenMatt)]

(In reply to Adam Lofting (:adamlofting) from comment #2)

> I think our 'target audience' and the people we want to communicate with are
> mentors, but the webmaker users (people with accounts) also include
> learners/students/randoms. And being able to work out which are the teachers
> will be really helpful. 

* That makes sense. I'm just not sure relying on email sign-up / BSD segmentation is a good way to measure "mentors" / teaching.

* And also: I don't think the way sign-ups are being collected / segmented does that right now. e.g., if you sign up from /teach you get segmented in BSD as a "mentor." But if you opt in as part of standard Webmaker account creation (through a door marked "We teach the web," for example) you don't. Even though you may very well identify as a "mentor" / teacher. 

* All the more reason we first need to document and simplify our sign-up segmentation first. Thanks for pushing on this.

Comment 4

[Andrea Wood [:andreawood]]

mentor vs. others can be a bigger set of buckets. Each bucket could contain multiple events or forms. Sorting which signups go in which bucket doesn't have to be done in BSD, but there should be a "query key" or saved queries that create the correct buckets. E.g.:

Maker Party event registrants go into "supporter / participant" bucket.
Maker Party hosts go into "mentor / teacher" bucket.

We definitely need to start using a unified file naming protocol to help our ability to query and sort. So whatever solution you create, let's come up with one. Suggestion (but would love Adam's input):

[program name]_[eventname]_yyyymmdd_registrant

[program name]_[eventname]_yyyymmdd_mentor

So in practice these would look like:

Webmaker_makerparty_20140715_registrant
Webmaker_makerparty_20140715_mentor

Comment 5

[Matt Thompson (@OpenMatt :OpenMatt)]

+1 Andrea let's. Me and Jbuck had a look through all our existing sign-up forms and documented the ones relevant to Webmaker here: 
https://etherpad.mozilla.org/Webmaker-email-segments

* For Webmaker, I'm going to propose we have four broad segments. Users can and do belong to multiple saved queries / keys / buckets:

a) Webmaker general 
b) Mentor and Super-Mentor 
c) Training 
d) Maker Party 

(There's more on this here:
https://etherpad.mozilla.org/Webmaker-email-segments )

Can we use those as the basis of our naming protocol, respectively? 
+ Adam, these are probably helpful buckets for counting as well. 


Adam and Andrea: do you guys agree?

Comment 6

[Andrea Wood [:andreawood]]

The naming protocol should only be used for forms or signup channels, including BSD event signups. I wouldn't tag those events or signup channels with a specific broad category.

You can keep a separate set of guidelines for what channels make up a category, and from those build queries. Meaning, within BSD there's no need to put "mentor" in the name of an individual's record or an event name, but you can create a query group of events and designate the query as the "mentor query." That way it's fine if an individual overlaps X queries and groups. (For the purposes of email, BSD automatically dedupes so overlap is not a barrier.) Keeping straightforward conventions for events and channels prevents potential confusion down the road. If you tie "mentor" to a record or event, we could end up changing what a "mentor" means or using another term altogether in 5 years.

Not sure i'm making sense, but I don't think we want to create file naming protocols that include these kinds of categorical designations inside BSD, but we should create and save queries that lump signups into a category such as "mentors."

Comment 7

[Adam Lofting (:adamlofting)]

I'm clearing the current needinfo request on me here, but the task is still open for me to implement the 'tracking' part of this.

Comment 8

[Adam Lofting (:adamlofting)]

I've proposed a way to solve the tracking issue here: Bug 1015117

Comment 10

[Adam Lofting (:adamlofting)]

Renaming this to close another almost identical ticket.

Comment 11

[Matt Thompson (@OpenMatt :OpenMatt)]

Smart. Adam: should we mark this closed then? And just carry on with 
Bug 1015117 ?

Comment 12

[Adam Lofting (:adamlofting)]

Yeah, this bug is just waiting for the single other action item. Let's close this.