1038877 - Increase the number of people subscribed to pages

Status: RESOLVED WONTFIX

(developer.mozilla.org Graveyard :: Wiki pages, enhancement)

Description

[Justin Crawford [:hoosteeno] [:jcrawford]]

What feature should be changed? Please provide the URL of the feature if possible.
==================================================================================
Every page in MDN includes an "Advanced" dropdown that has some functionality for power-users. One option in the dropdown is "subscribe". We should experiment with placement/phrasing of this option to see if we can increase the number of people subscribing to pages.

What problems would this solve?
===============================
MDN currently allows users to "subscribe" to a page, which means "get a rich notification by email anytime a page changes". People who get notification of changes to pages are much, much more likely to edit those pages than people who encounter a page using other means[0]. We hypothesize that more people watching pages would mean more engagement overall. This experiment would determine whether more prominence can improve usage of the subscribe feature.

[0] https://bugzilla.mozilla.org/show_bug.cgi?id=842340#c29

Who would use this?
===================
Initially, the audience would include...
* people who discover the feature and want to watch a page
* product teams interested in learning whether the feature could be more prominent

What would users see?
=====================
* A percentage would see no difference.
* A percentage would see different placement and phrasing -- for example, a checkbox in the editing screen: "[x] Watch this page for future changes".

What would users do? What would happen as a result?
===================================================
Users would engage with the options. Product teams would analyze the results and release the most effective option.

Is there anything else we should know?
======================================

Comment 1

[John Karahalis [:openjck]]

This is wonderful, Justin! Taking advantage the higher rates of contribution among subscribers is a great idea.

I'm excited to work on this. Please let me know when this is a priority, if it's not a priority already, and I'll be sure to get started right away. (If possible, it would be great to prioritize bug 949465 along with this, as we will be able to run more thorough A/B experiments once that's fixed.)

Comment 2

[Justin Crawford [:hoosteeno] [:jcrawford]]

John, could you estimate the level of effort needed for this experiment?

Comment 3

[John Karahalis [:openjck]]

(In reply to Justin Crawford [:hoosteeno] from comment #2)
> John, could you estimate the level of effort needed for this experiment?

To set up the experiment, analyze the data, and run follow-up experiments I would say roughly 2-3 days.

Comment 4

[David Walsh :davidwalsh]

There's a ticket that's recently been submitted, or at least a complaint around, that the "Save and Continue Editing" feature is causing loads of (unwanted) emails to be sent.  We should fix that first.

Comment 5

[John Karahalis [:openjck]]

I should add that, although I'm working on GitHub login right now, I'm super excited to get this experiment started. I've added it to my personal task list. Maris might also be able to say something about scheduling this around some of the other work we have going on.

Comment 6

[John Karahalis [:openjck]]

(In reply to David Walsh :davidwalsh from comment #4)
> There's a ticket that's recently been submitted, or at least a complaint
> around, that the "Save and Continue Editing" feature is causing loads of
> (unwanted) emails to be sent.  We should fix that first.

Looks like that's bug 1053697. I added it as a dependency.

Comment 7

[Luke Crouch [:groovecoder]]

I added a bunch of our existing "subscriber vs. non-subscriber" data to the "MDN Detailed Metrics" tab of the "DevRel Community Metrics" spreadsheet. [1] (And thereby completing my maddening descent into the spreadsheet hell of management.)

tl;dr - Over the last 5 months, "subscribers" have a 72% better "repeat contributor rate", and a 94% better (i.e., lower) "churn rate" than "non-subscribers".

:openjck - what experiment did you have in mind? My idea is to add a "Watch this page" checkbox in a few different places of the "Edit Page":
 * As a new ajax button in the "page-buttons" list
 * After the "Revision Comment" section
 * After the "Review Needed" section

[1] https://docs.google.com/a/mozilla.com/spreadsheet/ccc?key=0Ajtff7J_Ha95dDZSMlJoSVhFX0lJck1BN2lPOGd2U1E&usp=drive_web#gid=5

Comment 8

[John Karahalis [:openjck]]

> :openjck - what experiment did you have in mind? My idea is to add a "Watch
> this page" checkbox in a few different places of the "Edit Page":
>  * As a new ajax button in the "page-buttons" list
>  * After the "Revision Comment" section
>  * After the "Review Needed" section

I plan to test the label, placement, and color and go from there. Adding a button to the edit page will almost definitely increase usage. Rather than writing tests to verify that, I would suggest we add the button now and then run tests to optimize it further.

Comment 9

[Eric Shepherd [:sheppy]]

The main reason this feature is rarely used is simple: no support yet for subscribing to a page and all of its child pages. That is a far more useful feature, and one the engineering teams have been asking for repeatedly. This would let them watch the docs for their areas of interest to help ensure that changes that get made are accurate.

This would in turn lead to higher quality, more accurate docs.

I don't have the bug number handy for this, but it's s long-standing feature request.

Comment 10

[BMO Automation]

MDN Web Docs' bug reporting has now moved to GitHub. From now on, please file content bugs at https://github.com/mdn/sprints/issues/ and platform bugs at https://github.com/mdn/kuma/issues/.