665719 - Review and approval work flow

Status: RESOLVED WONTFIX

(developer.mozilla.org Graveyard :: Wiki pages, enhancement)

Comment 1

[Luke Crouch [:groovecoder]]

This bug needs more product direction - how should we do review & approval?

Comment 2

[Jay Patel [:jay]]

Janet and Sheppy need to chime in here about how they work currently and how we can improve this process for them and our community of contributors.

Based on what I know, here are some ideas on how this might work:
1. Flag a document (or section) for review
2. Specify a particular person or group (we may want to create a "reviewers" group that a few people can join if they want to participate in that role)
3. Allow the review state of the document to be set at various stages of the process (manually or automated?): review needed, in review, review completed, approved (not sure what flags we need, so hopefully the docs team has input here)
4. Notify relevant people at each stage
5. Make feeds of various stats available to users (we can use them for the user dashboard as well)
6. Show the current review state somewhere on the page that is visible, but not in your face.

Those all may need to be split up into individual stories, but let's wait to hear from the docs team to make sure my suggestions make sense and if there are things to add.

Comment 3

[Eric Shepherd [:sheppy]]

What we would like is two independent review types, of which one, none, or both can be set at the same time:

Editorial review
- This would mark the document as requiring someone to look over the formatting, grammar, layout, and the like. In theory non-technical people could help with these.

Technical review
- This would indicate that someone familiar with the technical subject matter needs to review the article to ensure it's technical accuracy.

In each case, an appropriate subset of MDN users would be made aware of the need to review the material. Users should have a way to opt in to reviewing certain types of material (that is, editorial review, or technical review by technology area).

We should also have a way for admins to specifically add people to the review groups for given technology areas; key developers should probably not be able to opt out of being in the review group for the technologies they're core developers for.

Once someone has reviewed the article, they should be able to hit a button to quickly flag it as reviewed for whichever type of review they performed.

Comment 4

[Les Orchard [:lorchard]]

Switching this bug's title seems confusing, given the conversation captured so far. This should probably be the umbrella bug for a series of bugs on review workflow

Comment 5

[Les Orchard [:lorchard]]

Moved "page showing all articles that need review" into its own bug, and captured a few more blockers from comments here.

Comment 6

[Florian Scholz (Open Web Docs)]

This weekend at MozCamp people were discussing how to point contributors to the right tasks they could potentially help out with and then I had an idea about how to get more reviews maybe:

So, what came to my mind was having a random list of articles in need of review on the user's profile page. Filter that list by the interests (tags) the user has set in his profile. So if I tagged myself as interested in "MathML", have a box with 5 articles which have the tag "MatML" on it and are in need of review. 

What do you think?

Comment 7

[Eric Shepherd [:sheppy]]

This is in fact on the feature roadmap; if there's not a bug for it already, that's a dreadful oversight.

Comment 8

[John Karahalis [:openjck]]

Hey Sheppy. Having a hard time understanding where this stands right now. Can you please answer the following questions for this bug?

https://bugzilla-stage-tip.mozilla.org/show_bug.cgi?id=686894#c0

Comment 9

[Eric Shepherd [:sheppy]]

What problems would this solve?
===============================
We need the ability to mark articles as needing given types of review: editorial or technical, and groups of people that are both willing to and qualified to perform those reviews. This would help eliminate our ongoing problems of documents remaining drafts forever, and would help engage our engineering community.

Who would use this?
===================
All MDN users that contribute text would use this, as would anyone willing to review content.

What would users see?
=====================
This is hard to say, and needs design work. Some sort of clear interface for marking an article as needing review (technical or editorial) when saving changes. Something more obvious than the existing checkboxes.

Profiles need the option to opt-in to being willing to perform reviews, and the ability to specify expertise areas. These would be matched to articles, presumably by tags.

Then site users that have opted-in would see a list of articles they can review on their profile page, as well as possibly other places on the site (in a sidebar maybe). Also, we should provide an RSS feed of articles they can review, as well as possibly eventually a widgety thing they could have appear on their blog, in their Firefox start page, or whatever they look at often.

What would users do? What would happen as a result?
===================================================
That list would provide the ability to click through to read the content, which would have the sections that have changed since the last review marked with ribbons along the side. They would be able to switch to the editor to make corrections, or simply click a button saying they've reviewed the content and it's fine (without having to even resort to the editor).

Ideally, this would tie in with being able to mark individual sections for review, and with section editing. It would also be nice to tie it into the annotation system we plan to build; let the reviewers note down ideas or mark up the content to "interact" with the writing team about possible revisions.

Is there anything else we should know?
======================================
Probably, but this is all I can think of right now. I think this is a broad bug that comprises many parts, which can be deployed in chunks. We just need to keep in mind all the possibilities and not get too locked into a simple form of this capability when there's so much room to do amazing things later.

Comment 10

[karabajic]

It would be helpful to analyze how various DMS/CMS systems addressed issues of document creation, user and group rights and privileges, review and approval workflow, collaboration, version control, etc. 

Nuxeo open source ECM has very good documentation system and it is definitely an excellent resource to examine how to deal with these issues. 

http://doc.nuxeo.com/display/USERDOC56/Starting+a+workflow

http://doc.nuxeo.com/display/USERDOC56/Document+Management+concepts 

Kuma should have such features and some sort of agile approach should be used to address the most important issues and then include other features as needed/desired. 

A balance between level of complexity and ease of use should be established and some other less complex systems such as Drupal (with decent user permission and workflow features implemented) and simpler solutions such as WP Document Revisions wordpress plugin. 

There is also a very nice list on Mediawiki/Wikipedia user access levels and permissions:

https://www.mediawiki.org/wiki/Manual:User_rights#List_of_permissions
http://en.wikipedia.org/wiki/Wikipedia:User_access_levels#Table

Comment 11

[Janet Swisher]

(In reply to karabajic from comment #10)
> It would be helpful to analyze how various DMS/CMS systems addressed issues
> of document creation, user and group rights and privileges, review and
> approval workflow, collaboration, version control, etc. 

Thanks for the links. It's good to keep these ideas and models in mind as we design this functionality for Kuma. 

In general, I'd like to keep the workflow as lightweight as possible, to preserve the wiki-ness of Kuma, rather than turning it into a CMS-like tool. 

That said, it would be good to have the ability to proactively tag a specific person to review a page, rather than passively waiting for someone to see it and do it. For example, if the Mozilla subject-matter-expert on a particular API is the only person who can meaningfully do a technical review, then name them. The "need more information from" widget in Bugzilla is an example of this type of functionality.

Comment 12

[Eric Shepherd [:sheppy]]

Yeah. The ideal here would be to have the option to either request that a specific person do a review, or, if you don't specify someone, have the system put it on a list that's presented to everyone who's indicated in their MDN profile that they know the topic area the article covers.

Comment 13

[John Karahalis [:openjck]]

These requirements are very helpful (especially comment 9), but it appears we need even more detail to move forward. Without this clarity, I suspect this important proposal will stagnate.

I see two approaches we can take moving forward.

1. Publish an initial mockup and description of behavior based on the discussion
   that has happened until now, and meet occasionally over time to iterate on
   these assets. This is the approach we are currently taking with the Localization
   Dashboard.
2. Start to build this feature out behind a feature flag, and release it after it
   has reached an appropriate level of maturity. This is the approach we have taken
   with the new homepage and the email notification feature.

I suspect the latter would work better, but what about you? What approach should we take?

Comment 14

[Eric Shepherd [:sheppy]]

I also suspect #2 will work best given the way things tend to go around here. :)

It can probably be broken into smaller bits, as well, such as creating code that can select a given number of articles matching the user's expertise from the list of pages marked for review. Then that can be used to create a list on their profile page, and in some kind of widget we can place on the MDN home page and elsewhere as we see fit.

That's probably the first thing to tackle.

Comment 15

[John Karahalis [:openjck]]

Great. So lets get started with approach #2.

Engineers, does comment 14 sound like a good first step? Whatever the first step is, let's get a card on the Kanban board and (if you think it would help) a dependent bug open.

I gave this bug a P1 so that we don't forget about it. Sheppy, in the meantime please feel free to add a vote to this bug so that it floats to the top of the list.

Comment 16

[Luke Crouch [:groovecoder]]

I created bug 874551 to get us started. Vote there. :)

Comment 18

[John Karahalis [:openjck]]

Bug 671912 has some additional thoughts we might want to keep in mind.

Comment 19

[BMO Automation]

MDN Web Docs' bug reporting has now moved to GitHub. From now on, please file content bugs at https://github.com/mdn/sprints/issues/ and platform bugs at https://github.com/mdn/kuma/issues/.