842340 - Allow people to receive notification of changes to any page in a subtree

Status: RESOLVED FIXED

(developer.mozilla.org Graveyard :: Wiki pages, enhancement)

Description

[Eric Shepherd [:sheppy]]

What problems would this solve?
===============================
Users need to be able to watch pages or subtrees that interest them so that they get email notifications when those pages change. This is needed in order to help engineers or other occasional contributors participate in the documentation process without having to actively monitor content.

Who would use this?
===================
Typically this would be used by engineers and other low-frequency contributors.

What would users see?
=====================
Links "Watch this page" and "Watch subpages" probably in the "This page" drop-down.

What would users do? What would happen as a result?
===================================================
After selecting the link, the user's account would be added to the list of watchers for the page. When changes occur, they would receive an email each time the page is changed, with a description of what changed, including the diff. The email should have links to visit the page, edit the page, and see the page's history.

Is there anything else we should know?
======================================
We're getting a lot of requests for this from the dev team. Adding this feature would remove an "excuse" (for lack of a better world) that a lot of people use for not being more engaged in documentation.

Comment 1

[Dan Callahan [:callahad]]

Are per-page/subtree RSS feeds in-scope for this bug?

Comment 2

[Eric Shepherd [:sheppy]]

(In reply to Dan Callahan [:callahad] from comment #1)
> Are per-page/subtree RSS feeds in-scope for this bug?

That's a good question. We had largely abandoned RSS feeds since most of the people who actively use MDN watching features had said they weren't a priority, but we can certainly re-evaluate that.

For what it's worth, you can use our revision dashboard for something resembling what you need. For example, you can use it to watch Persona content here:

https://developer.mozilla.org/en-US/dashboards/revisions?locale=en-US&user=&topic=Persona

This will show you changes to the English-language documents about Persona.

Comment 4

[John Karahalis [:openjck]]

We have considered per-page RSS feeds with bug 508213, but it has not really gone anywhere yet. If we think it would be useful, perhaps we could add more detail to that bug (like what Sheppy did in comment 0).

Comment 5

[David Bruant]

(In reply to Dan Callahan [:callahad] from comment #1)
> Are per-page/subtree RSS feeds in-scope for this bug?
This initial motivation of this bug was revealed in a recent thread about documentation.
To quote Boris Zbarski [1] (and others seemed to agree)
"I don't want to have to make sure to load [some dashboard] every day.  I really need a push mechanism like mail or RSS. "

So I guess RSS is in scope if that's something people are willing to use.

[1] https://groups.google.com/d/msg/mozilla.dev.planning/Md6_CaOcABI/2uaoAt-kQMQJ

Comment 6

[Boris Zbarsky [:bzbarsky]]

> This will show you changes to the English-language documents about Persona.

Again, that requires the user to pull (go and load that url explicitly), unlike either an actual RSS feed or mail notifications.

Personally, I would vastly prefer an RSS feed for watching, but can probably make do with email.

Comment 7

[Eric Shepherd [:sheppy]]

(In reply to Boris Zbarsky (:bz) from comment #6)
> > This will show you changes to the English-language documents about Persona.
> 
> Again, that requires the user to pull (go and load that url explicitly),
> unlike either an actual RSS feed or mail notifications.
> 
> Personally, I would vastly prefer an RSS feed for watching, but can probably
> make do with email.

I get that, totally. I was just suggesting that a temporary workaround exists until we have that capability in place. It's better than manually loading each page to see if it's been contaminated.

Comment 8

[Jean-Yves Perrier [:teoli]]

In one meeting (John and Sheppy were there), we arrived at the conclusion that editors that use MDN every day often prefer dashboard, occasional editors that watch only a part of the MDN prefer a push mechanism (RSS).

Note that most vandalism on Persona happens on the main page by newly created users, and even if we need a push mechanism to monitor sub-pages, we also need the ability to protect or semi-protect pages (don't have the bug at hands). This will also help us to temporary protect important pages, during an event.

Comment 9

[Eric Shepherd [:sheppy]]

(In reply to Jean-Yves Perrier [:teoli] from comment #8)
> In one meeting (John and Sheppy were there), we arrived at the conclusion
> that editors that use MDN every day often prefer dashboard, occasional
> editors that watch only a part of the MDN prefer a push mechanism (RSS).
> 
> Note that most vandalism on Persona happens on the main page by newly
> created users, and even if we need a push mechanism to monitor sub-pages, we
> also need the ability to protect or semi-protect pages (don't have the bug
> at hands). This will also help us to temporary protect important pages,
> during an event.

Yes, this is a good point. Having page locking, which is on the list but has never drifted near the top of the priority list, would let us protect key pages during events -- but more importantly would let us lock down general landing pages, which typically won't change all that often.

This would be especially useful if we can lock landing pages so they can only be revised by users that have been around for a while; this would prevent them from being edited by new accounts (which account for essentially all spam) without impacting regular contributors at all.

Comment 10

[Eric Shepherd [:sheppy]]

This would obviously need to tie into the notification system, which would let users specify how how often they want to be contacted (such as by email, tweet, etc, and whether or be pinged daily or weekly).

Comment 11

[Eric Shepherd [:sheppy]]

Oh, also, RSS; several devs have mentioned an RSS feed of changes in pages/trees they're watching would be really helpful for them to track what they've reviewed.

Comment 13

[John Karahalis [:openjck]]

So this is a tough problem. I would like us to get something started as soon as possible, because Dan cannot (and should not) wait any longer.

We have four or five different feature proposals in here -- page/subtree watching, RSS notifications, a dashboard, some other notification system, page locking, etc.

Let's keep this bug for the feature it was originally intended to describe -- the feature in comment 0. I will start an email thread to discuss the other proposals. Once the other ideas reach a certain level of clarity, we can open separate bugs for them.

Comment 14

[Luke Crouch [:groovecoder]]

FWIW, SUMO uses https://github.com/erikrose/django-tidings which looks like a very versatile library for watching events.

Comment 15

[James Socol [:jsocol, :james]]

AND it's maintained by our very own :ErikRose.

Comment 16

[John Karahalis [:openjck]]

We are having a discussion about this on dev-mdn@.
https://groups.google.com/forum/?fromgroups=#!topic/mozilla.dev.mdn/Uvaj9emz5tw

It sounds like using email as a shorter-term solution would be ideal for now. We will open a separate bug for that. We will keep a more robust page watching feature (as described in comment 0 of this bug) in mind, but I would rather we concentrate our efforts on email for now. I am going to remove this from the "Research" column.

Comment 17

[Eric Shepherd [:sheppy]]

Do we know when work will happen on adding subtree support?

Comment 18

[Luke Crouch [:groovecoder]]

This is still pretty low on the backlog (http://mzl.la/mdn_backlog) and we haven't even removed the waffle flag from the single page watching feature so we're still a ways away.

Comment 19

[Eric Shepherd [:sheppy]]

I thought watching was one of our top priorities. Huh.

Comment 20

[Eric Shepherd [:sheppy]]

(In reply to Luke Crouch [:groovecoder] from comment #18)
> This is still pretty low on the backlog (http://mzl.la/mdn_backlog) and we
> haven't even removed the waffle flag from the single page watching feature
> so we're still a ways away.

What if anything is preventing us from un-waffling single-page watching at this point?

Comment 21

[Luke Crouch [:groovecoder]]

nothing now. I'm going to roll it out slowly though. I'll start with 10% of users and go up day-by-day.

Comment 22

[Luke Crouch [:groovecoder]]

I've dug into this a bit, and hit a snag in django-tidings - it doesn't allow us to set up partial match watch filters. [1] I pinged Erik and we talked a bit about firing an event for each segment along the path, but that seems way too messy and may not work anyway. [2]

The way forward is probably to add partial-match watch filters to django-tidings.

[1] https://github.com/erikrose/django-tidings/blob/master/tidings/events.py#L145-252
[2] http://pastebin.mozilla.org/2549532

Comment 23

[Eric Shepherd [:sheppy]]

(In reply to Luke Crouch [:groovecoder] from comment #22)

> The way forward is probably to add partial-match watch filters to
> django-tidings.

Any idea how long that'll take to do?

Comment 24

[Luke Crouch [:groovecoder]]

The code is pretty meta for me. If I were doing it, I'd say at least a week. :jezdez or :ubernostrum - does 1 week seem about right to add partial-match watch filters to django-tidings?

Comment 25

[James Bennett [:ubernostrum]]

Sounds about right to me unless I'm misunderstanding how it'd work.

Comment 26

[Māris Fogels [:mars] (please needinfo)]

:hoosteeno, I think we have the information to plan this.  :groovecoder says that our measurements showed a big boost in contributor numbers, and this doesn't look like too much work to implement.

Comment 27

[Luke Crouch [:groovecoder]]

Attachment: mdn-sources-for-edit-conversions.gif

Our page watch notification emails have a 50,008.30% better "Edit an Article" conversion rate than other sources of traffic.

Comment 28

[Jean-Yves Perrier [:teoli]]

Is it good or bad? Because, it is obvious that we will have a far better conversion rate in this case.

As obvious as implementing this bug (which is very useful for drivers), will make it go down.

Comment 29

[Luke Crouch [:groovecoder]]

It's still good ...

Of the 187 sessions which came from an article notification email, 32 (17%) went to edit an article. By comparison, some other groups of traffic sources' conversion rates are:

Emails: 2.03% (e.g., Apps & Hacks Newsletters)
Other: 0.27%
Referrals: 0.07%
Direct: 0.05%
Social: 0.04%
Search: 0.03%

I predict this particular conversion rate will not go down much. These 187 sessions are only users who clicked thru on an email link. I.e., this doesn't tell us how many users simply ignored the email.

But, during the same time period, only 10 page-views went to an "unsubscribe" link. So, the notifications' edit/unsubscribe ratio is ~= 3:1.

We want users to opt into watching a page so they can stay engaged with the MDN content. Right now, only 742 users are watching 1,413 pages.

So, perhaps the real measurement for success is to increase the number of users & watches while maintaining a 3:1 edit:unsubscribe ratio? That tells us that, even if the conversion rate drops because we send more emails, users are still finding the email notifications valuable.

Comment 30

[Justin Crawford [:hoosteeno] [:jcrawford]]

For those just joining (like me), this bug is not about page watching anymore. It's about subtree watching. MDN already includes page watching (see "Subscribe" under the gear menu on any page). 

Specifically, this bug is about adding the ability for people to watch subtrees and/or tags in addition to individual pages. It doesn't include changes to notification types or options -- let's file new bugs for those if they come up.

(In reply to Māris Fogels [:mars] from comment #26)
> :hoosteeno, I think we have the information to plan this.  :groovecoder says
> that our measurements showed a big boost in contributor numbers, and this
> doesn't look like too much work to implement.

It does sound like we know how much effort this will require, and we have some idea of how it might work. But it's not clear what impact we expect it to have. Does anyone suggest a way to estimate the potential of this feature (a one-question survey? a button to nowhere?)? Based on what we know now, it doesn't seem like this ought to jump to the front of the priority queue.

Comment 31

[Eric Shepherd [:sheppy]]

(In reply to Justin Crawford [:hoosteeno] from comment #30)

> It does sound like we know how much effort this will require, and we have
> some idea of how it might work. But it's not clear what impact we expect it
> to have. Does anyone suggest a way to estimate the potential of this feature
> (a one-question survey? a button to nowhere?)? Based on what we know now, it
> doesn't seem like this ought to jump to the front of the priority queue.

To quickly summarize the major use cases for this:

* Help topic drivers monitor their content for mistakes, new contributors, and spam.

* Help topic localizers monitor the English content in their topic for changes that need translating.

* Let developers monitor the docs for the technology/technologies they work on, so they can help technical-review changes. This is a big one -- many, many developers have been pleading for this.

There are certainly other uses, but these are probably the biggest ones.

This feature will have an enormous impact on content quality and usability, by helping to improve both tech and editorial review rates, as well as reduce spam.

Comment 32

[Justin Crawford [:hoosteeno] [:jcrawford]]

(In reply to Eric Shepherd [:sheppy] from comment #31)

> 
> * Let developers monitor the docs for the technology/technologies they work
> on, so they can help technical-review changes. This is a big one -- many,
> many developers have been pleading for this.

I just had an IRC chat with :callahad and he agreed completely with the above. Individual page watching is not powerful enough to get him engaged in all the topic areas he is expert in and would contribute to if he was able to see changes at a topic level.

In other words, anecdote suggests implementing subtree watching would entice more watching (and thereby improve content quality and engagement) among some very knowledgeable contributors.

Comment 33

[Jean-Yves Perrier [:teoli]]

Yes, this is obvious. We are saying since years.

Comment 34

[Eric Shepherd [:sheppy]]

(In reply to Jean-Yves Perrier [:teoli] from comment #33)
> Yes, this is obvious. We are saying since years.

Why, that's not even hyperbole! :)

Comment 35

[Justin Crawford [:hoosteeno] [:jcrawford]]

We added the [patchwelcome] flag to the whiteboard which puts this bug on a list of bugs we'd love someone to send a PR for[0]. This isn't a P1 right now.

[0] https://wiki.mozilla.org/Webdev/GetInvolved/developer.mozilla.org#PatchWelcome_Bugs

Comment 36

[Luke Crouch [:groovecoder]]

I intend to get back into this as a good task while I'm on half days.

Comment 37

[Luke Crouch [:groovecoder]]

First version of the code is in a PR here:

https://github.com/mozilla/kuma/pull/3602

Comment 38

[MDN Team (:mdn-dev)]

Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/cca5fdef13de4e07547a63f5989c72cb99835a98
bug 842340 - new tidings event for trees

https://github.com/mozilla/kuma/commit/3fb6ee0a7900fc1e39b0ae0d5a48b78bdcf4fc3d
bug 842340 - fire a single EventUnion on edits

https://github.com/mozilla/kuma/commit/70eac0d01c23765bf5c274a7a22729a77eef4395
bug 842340 - review cleanup

https://github.com/mozilla/kuma/commit/c8f90d1f77bb873b5496532121a3877241aca58f
bug 842340 - watch_menu_items macro for buttons.html

https://github.com/mozilla/kuma/commit/052042e26c9a9ad76164c294cec212020bf28adc
bug 842340: tree subscription copy in edited.ltxt

https://github.com/mozilla/kuma/commit/0280387c8ab81cbad7ac74fae5e2fe7ceb211a8a
bug 842340: watch tree tests: events, models, views

https://github.com/mozilla/kuma/commit/9d2d89b13b313f615deefb77417367074f7fb9a0
bug 842340 - view tests for document watches

https://github.com/mozilla/kuma/commit/c58e966cedc45842758687caadcced35b438b39e
bug 842340 - GA tracking for page watch (menu)

https://github.com/mozilla/kuma/commit/f3cee25db988ee83ca45add8b13bfbac14885aa8
bug 842340 - flake8 test code; context for macro

https://github.com/mozilla/kuma/commit/176be50d14d71e7e3568a0e6452a8c4f07147133
bug 842340 - proper <ul> nesting of watch_menu

https://github.com/mozilla/kuma/commit/b7935cad56cc1dc206f0df05577d9662ff203311
bug 842340 - <bdi> for {language} text

https://github.com/mozilla/kuma/commit/b3d29f28917bfaee0d799ef9d178eea353440f42
bug 842340 - Document.parent_trees_watched_by(user)

https://github.com/mozilla/kuma/commit/cfa69a35e64248aed1f3694eda775d7ac7f3ad9b
bug 842340 - clean edited.ltxt footer

verify expected emails in tests

https://github.com/mozilla/kuma/commit/fabc1885b4741492be202f0626ec8addc33298fb
bug 842340 - clean whitespace in emails

https://github.com/mozilla/kuma/commit/a418f65accc83388f415c77e121e151b0c6b6259
bug 842340 - include parent tree unsubscribe links

https://github.com/mozilla/kuma/commit/c6f7345d4e15fc6a4dbf1be4d8a44ff4c8d8a640
bug 842340 - rename document tree test helper

https://github.com/mozilla/kuma/commit/50ad72807d3205c3821fc5868b1c163f54ac7868
bug 842340 - escape doc title in tree item

move parent tree watch links below page watch links

Comment 39

[MDN Team (:mdn-dev)]

Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/00c0f9ef16342b5a5894bad1e52bc35c87a31521
bug 842340 - only show <ul> if there are items

https://github.com/mozilla/kuma/commit/265d2f9f88066b901a06d844112421ab8a44098f
Merge pull request #3653 from mozilla/fix-trees-watched-ul-842340

bug 842340 - only show <ul> if there are items

Comment 40

[MDN Team (:mdn-dev)]

Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/985d03713185dccd9666a6d5f0c16cad6c97e293
Bug 842340 - Fix template parse error for some document titles

https://github.com/mozilla/kuma/commit/2718ec30492d7779ba5710821409803f7d1e2696
Merge pull request #3655 from robhudson/842340-fix

Bug 842340 - Fix template parse error for some document titles

Comment 41

[Luke Crouch [:groovecoder]]

2 weeks of data testing between a separate watch menu vs. no-watch menu (subscribe links integrated into the existing "advanced" menu) ...

Pages/Session: no watch: 1.73 vs. watch: 1.81
Avg. Session Duration: no watch: 2:18 vs. watch: 2:31
Bounce Rate: no watch: 69.4% vs. watch: 66.7%
Edit Article conversation rate: no watch: 0.03% vs. watch: 0.04%
Watch conversion rate: no watch: <0.01% vs. watch: <0.01%

Since the new watch menu does not detract from Edits and gives us more options to experiment at driving more watch conversions, we're going to keep the separated watch menu.

Sending PR to make the design final ...

Comment 42

[Luke Crouch [:groovecoder]]

https://github.com/mozilla/kuma/pull/3685

Comment 43

[MDN Team (:mdn-dev)]

Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/51db7e8851387b5b34f6586e98d6d0dac53fba22
bug 842340 - remove watch_menu waffle flag

https://github.com/mozilla/kuma/commit/acf40f91b30aacd5593d44ce988ef5981d6719ac
bug 842340 - updateMessageText notification method

When we display a notification message with user-generated
content (e.g., page title), we need to use text() to insert
it, so jQuery will escape the user-generated content.

https://github.com/mozilla/kuma/commit/e14aacb7e08a09c5da994499fca22f8e0a3b5679
bug 842340 - jshint clean components.js

https://github.com/mozilla/kuma/commit/11d44a6828b85bfdb5e9204184d0588bd8bcc6fa
Merge pull request #3685 from mozilla/finalize-watch-menu-842340

bug 842340 - remove watch_menu waffle flag

Comment 44

[Jean-Yves Perrier [:teoli]]

I think this is done. Reopen if I'm mistaken.