1540102 - mdnwebdocs-bot, deferred rerender, diff view based on last edit makes maintenance laborious

Status: RESOLVED WONTFIX

(developer.mozilla.org Graveyard :: Editing, enhancement, P3)

Description

[Julien G]

What feature should be changed? Please provide the URL of the feature if possible.

Technical edits (i.e. ones made by bots that don't change the semantic of an article) should not be:

• taken into account for last edit timestamps or counted as last edits for doc status pages
• visible from the diffing view (which appears at the top of an "out of date" localized article when editing it)

What problems would this solve?

With mdnwebdocs-bot / deferred rerender and the current diffing options, my (4 years old) workflow for maintaining pages in French went from

1. Refresh the doc status page for a given section
2. Open every outdated page in edition mode
3. Use the diffing view to apply the relevant changes to French content

to

1. Refresh the doc status page for a given section
2. Open a few pages among the hundreds which are shown out of date
3. For each page, open the $history view for the English page
4. For each page, open the $history view for the French page
5. Determine when the last relevant edit was made for the French and English page
6. Use the English $history view to display the relevant diff in another window
7. Apply the relevant changes to the French content

Aside ranting, this heavily slows down the maintenance process to the point it will take several months of (daily) work to get back to a "sane" state.
There is currently no easy way to get a relevant diff to apply.

Who would use this?

Maintainers for locales -(maybe just me)

What would users see?

Either a "selectable" diffing view to make identifying change easier.
And/or a doc status table which takes into account if the last edit was a "non-semantic" one.

What would users do? What would happen as a result?

Users may not lose motivation of

Is there anything else we should know?

mdnwebdocs-bot only seem to have run the 24th of March.
However, a rerender was triggered on the 28th. The side effect being that, on the 27th, the doc status pages seemed "clean" while on the 29th, status coverage dropped from more than 50%

Comment 1

[Julien G]

Attachment: List of crawled pages which should be "up to date" in French for the HTML/CSS/JavaScript sections

By crawling/scraping the $history view for all of the pages of the sections and locale of interest (in my case HTML/CSS/JS in French), I was able to build a list of 1978 pages which I'll "blank" edit (a rev with only a comment to explain why the edit is empty) to get them "up to date" (using the meaning defined in the localization doc status page's logic).

If any localizer wants me to produce the same kind of list for another section or another locale, do not hesitate to ping me.

Comment 2

[Julien G]

I am done blank editing the pages which were flagged as "out-of-date" (as in English version was updated more recently than the French one). While doing so in the last days, I've thought about the following:

• As long as the diffing view stays the same and as long as the batch processing is also applied to localized pages, it is currently non-trivial to get a good idea of what needs to be applied on the localized version. Regarding this, I'm planning to experiment on a WebExtension to improve the diffing view for localization (incl. being able to choose which revs to diff). In other words, as of now, anyone opening a localized page to edit it won't, most probably, have the right diff. (note: not "having the right diff" may be cause by a lot of other reasons, in a more restricted settings though)

• I pondered about the application of batch processing to the localized documents:

  • if this is necessary for an archived/unmaintained section (e.g. /Mozilla or some parts of it), then applying the process to this subset won't probably hurt
  • if this processing is targeting a maintained section (e.g. JavaScript), either
    • it has been decided to apply the batch processing to the localized docs, then it should first be applied on the localized content and then on the English content. This was rightfully done during mdnwebdocs-bot :) Not doing so will mark all the localized content as "up to date" (which is untrue and indirectly create loss of (meta)data)
    • it is left to the maintainer/localizer's choice (if any...) to apply the process: in the current situation, a manual edit from a localizer is still needed after the automatic batch processing to "mark" the localized page as up-to-date. Thus, if the batch processing is intended for "bleaching"/"post processing" actions which necessarily occur after a save, batch processing + a manual edit from the localizer produce the same side effects for the content, twice.

• I was able to crawl/scrap the data in order to identify the pages that were "already up-to-date" before the batch processing. In theory, it is possible to detect those pages as well and to apply a second edit (on the identified localized content) after the processing of the English section to get it back to an "up-to-date" state.

• I know batch processing/bot/mass editing is a delicate matter (responsibility, power, Uncle Ben, etc.) and that we may have different expectations around it and the future of MDN. That being written, during my manual edits, I took the time (few seconds per page) to tackle some inconsistencies in Fr (eg. {{cssinfo}} rather being located in the Specifications in the CSS section, HTML pages having their "technical" table moved under a "Technical summary" section, fixing the heading levels, etc.). I had a mixed feelings:

  • should we take the time to manually enforce new conventions/styles and fix such things once a year/<other period>?
  • or should we also use bots/mass edit to avoid a situation with heterogeneous docs when conventions/style guides evolve?

N.B. this may be dense or poorly written, do not hesitate to ping me here or anywhere else if there is anything unclear or if you want to discuss this further.

Comment 3

[Ryan Johnson (:rjohnson)]

First of all, I'm sorry for my delayed response Julien. The past two weeks have been crazily busy. Second, I'm really sorry for causing these issues for you by running the mdnwebdocs-bot to clean all of the English and localized documents. I must admit that I was unaware of how this would affect your workflow, and it pains me to see what you've had to do in order to recover. If it's any comfort, I was never going to run the mdnwebdocs-bot but once, so it will never be run again. If we ever do something like this again (nothing is planned) I will discuss with it with you before we do anything, so we can eliminate or at least minimize the consequences for localizers.

Comment 4

[BMO Automation]

MDN Web Docs' bug reporting has now moved to GitHub. From now on, please file content bugs at https://github.com/mdn/sprints/issues/ and platform bugs at https://github.com/mdn/kuma/issues/.