1182542 - Scrape MDN URLs nested more than 5 levels

Status: RESOLVED FIXED

(developer.mozilla.org Graveyard :: BrowserCompat, defect)

Description

[John Whitlock [:jwhitlock]]

What problem would this feature solve?
======================================
Pages that are more than 5-levels deep are not imported into the browser compat API.

Who has this problem?
=====================
Core contributors to MDN

How do you know that the users identified above have this problem?
==================================================================
Reported in bug 1132658 by :fscholz

How are the users identified above solving this problem now?
============================================================
They are not able to import data from these deeply nested pages.

Do you have any suggestions for solving the problem? Please explain in detail.
==============================================================================
The $children API has a maximum depth of 5 levels.  Instead of asking for 5 levels of children from /Web, /Navigation_timing, etc., the tool should as for 1 level of children, and then recursively ask for the children of the result URLs.

Is there anything else we should know?
======================================
Example page: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Proxy/handler/getPrototypeOf

Comment 1

[John Whitlock [:jwhitlock]]

This has been fixed:

https://github.com/mdn/browsercompat/pull/36

A recursive scan of /docs/Web/ found getPrototypeOf:

http://localhost:8000/importer/6234

The GitHub hook that annotates Bugzilla on merges needs updating from the move to https://github.com/mdn/browsercompat, but that's another issue.

Comment 2

[John Whitlock [:jwhitlock]]

A link to my local dev server isn't very helpful.  Here's the import of getPrototypeOf:

https://browsercompat.herokuapp.com/importer/6554