1181158 - Importer should handle deleted MDN translations

Status: RESOLVED FIXED

(developer.mozilla.org Graveyard :: BrowserCompat, defect)

Description

[John Whitlock [:jwhitlock]]

What problem would this feature solve?
======================================
On the overflow-y page [1], the Polish translation has been deleted [2], but still appears as an alternate translation in the metadata [3].  The importer stops on a failed_download issue [4].  However, the English version is available, so parsing should continue.

[1] https://developer.mozilla.org/en-US/docs/Web/CSS/overflow-y
[2] https://developer.mozilla.org/pl/docs/Web/CSS/CSS_Reference/Własności_szablonu
[3] https://developer.mozilla.org/en-US/docs/Web/CSS/overflow-y$json
[4] https://browsercompat.herokuapp.com/importer/797

Who has this problem?
=====================
Staff contributors to MDN

How do you know that the users identified above have this problem?
==================================================================
See the importer page for overflow_y above [4]

How are the users identified above solving this problem now?
============================================================
Users are waiting for the page metadata to catch up with the translation deletion.

Do you have any suggestions for solving the problem? Please explain in detail.
==============================================================================
The importer can be smarter about failed downloads, adding an issue for a failed translation download but continuing if the English translation is available.

Is there anything else we should know?
======================================

Comment 1

[Jean-Yves Perrier [:teoli]]

I think the importer shouldn't be ignoring this error.

This is a real problem: both the pl and ko translation weren't translation of that page. We should sever the link between these document and the en-US one, as it prevents a proper translation to be added by a contributor.

We need to fix this to have a coherent Kuma DB.

Comment 2

[John Whitlock [:jwhitlock]]

I think the right thing to do for the MDN importer is to work around the MDN bug.  I've opened bug 1182510 to track the MDN issue - if it gets fixed first, then we can drop this bug as irrelevant.

Comment 3

[John Whitlock [:jwhitlock]]

Another page to ignore:

https://developer.mozilla.org/en-US/docs/Web/HTML/Element/meter
https://developer.mozilla.org/zh-TW/docs/Web/HTML/HTML_%25E5%2585%2583%25E7%25B4%25A0/meter

MDN issue filed as bug 1183131

Comment 4

[John Whitlock [:jwhitlock]]

The reason to download non-English translations was to automate the importing of localized strings.  However, automation is difficult, imprecise, and will require a lot of manual oversight. I think our effort would be better spent on the contribution interface for translators.

In any case, this is a great example of "You aren't gonna need it" [1]. I'm going to remove downloading the additional translations, which will speed up the import process, and we can add them back in the future if needed.


[1] https://en.wikipedia.org/wiki/You_aren%27t_gonna_need_it

Comment 5

[MDN Team (:mdn-dev)]

Commits pushed to master at https://github.com/mdn/browsercompat

https://github.com/mdn/browsercompat/commit/1c08936f899d526fd6d46323530b28dff850d3cb
bug 1181158 - Stop caching non-English MDN pages

When the importer project started, one potential feature was scraping
localizations of feature names and notes from non-English pages. There
are several issues with this approach, including a lack of strong
mapping from English features to the same feature on the non-English
pages, and content issues fixed on the English page but present on the
non-English page. Instead, we'll focus our efforts on a contribution
interface for translators, and save the time and database space
downloading non-English MDN pages.

https://github.com/mdn/browsercompat/commit/40315853ff87ff5eef7d435318ed8aa64a355162
fix bug 1181158 - Avoid fetch_all_translations() error

fetch_all_translations() decides if the next step is to download another
translation or to parse the page. There is a frequent 500 error when
fetch_all_translations() is called after parsing is complete, which can
happen from fetch_translation(). This change makes
fetch_all_translations() exit early, rather than raise an exception.