1153260 - [Compat Data][Importer] Refactor MDN scraping tests

Status: RESOLVED FIXED

(developer.mozilla.org Graveyard :: General, enhancement)

Description

[John Whitlock [:jwhitlock]]

What problems would this solve?
===============================
The MDN scraping tests include copy-and-pasted test code that make it more difficult to determine what is being tested.

Who would use this?
===================
Code reviewers

What would users see?
=====================
It would be clearer what functionality is being tested, and the differences between tests

What would users do? What would happen as a result?
===================================================
Code reviewers would find it easier to read the scraper tests

Is there anything else we should know?
======================================
:groovecoder mentioned this in the code review for bug bug 1128519, which merged the support footnote and note fields into note.  This changed several scraper tests.  Instead of including unrelated refactoring in that bug, I'm opening this one.

Comment 1

[John Whitlock [:jwhitlock]]

This bug should block bug 1132269, but my user is not able to assign blocks.  :groovecoder, can you assign?

Comment 2

[John Whitlock [:jwhitlock]]

The next screen let me assign the block.

Comment 3

[MDN Team (:mdn-dev)]

Commits pushed to master at https://github.com/mozilla/web-platform-compat

https://github.com/mozilla/web-platform-compat/commit/7e92fed2d6baf1341a2a78fec97c0ee0284aefec
bug 1153260 - Refactor TestGrammar

https://github.com/mozilla/web-platform-compat/commit/02c0b0d53ebe4ddfe6c6c13cf8ba5f4925ae9cbf
bug 1153260 - Use more precises pragma

https://github.com/mozilla/web-platform-compat/commit/24209b7d1c10ae6f04111d3d6a5e5859dcf98e41
bug 1153260 - Refactor TestPageVisitor, pass 1

- Move spec_h2 testing to TestGrammar
- Add ScrapeTestCase.get_instance(), to load individual test instances
  recursively. Replaced usage in TestPageVisitor, some usage in other
  tests. Eventually will replace the add_models() and similar setup
  code.
- Rewrite TestPageVisitor tests to assert_X, test_X_variant pattern

https://github.com/mozilla/web-platform-compat/commit/176d83ddb54b3f7938f94339a3ef987fd2f7e77a
bug 1153260 - Refactor TestPageVisitor, pass 2

Added additional tests so that PageVisitor is covered by tests in
TestPageVisitor, which allows removing or simplifying tests in
TestScrape and TestScrapeFeaturePage.

https://github.com/mozilla/web-platform-compat/commit/f26e7689e079a56abdb6bc5502e6b06c38d57284
bug 1153260 - Reduce TestScrape to essentials

https://github.com/mozilla/web-platform-compat/commit/d919673b0e9570dd148a26c796aea88c5bf874de
bug 1153260 - Refactor scrape_feature_page

Rearrange scrape_feature_page into helper class ScrapedViewFeature

https://github.com/mozilla/web-platform-compat/commit/43af178f6ddc50e1eec267854968bdab628fb763
bug 1153260 - Move tests to TestScrapedViewFeature

Testing ScrapedViewFeature means that the tests can be more precise,
with fake scraped data, rather than round-trip scraping of actual HTML.
This reduces TestScrapeFeaturePage tests to error handling, eliminating
the setup of database models.

https://github.com/mozilla/web-platform-compat/commit/3932e1e5bbaf944b0966426cc11e9d22148afa85
bug 1153260 - Remove/reduce test fixtures

Switched simple_page to sample_page, and reduced usage to places that
actually use it. Simplified the sample table, and dropped references to
instances that are no longer used.

https://github.com/mozilla/web-platform-compat/commit/a55e497a9935b33250acb14c10485696de47f9e6
fix bug 1153260 - New strategy for new/existing

In ScrapedViewFeature, use string-based IDs for linked resources
lookups.  Previously, integer IDs were used to identify existing
resources, and string IDs for new resources, so that existing resources
could be sorted above new resources.

https://github.com/mozilla/web-platform-compat/commit/4c94726a03f4f98e4783bcdf47b8deef5d0307d7
Merge pull request #28 from jwhitlock/1153260_refactor_mdn_tests

Fix bug 1153260 - Refactor MDN Tests