1711337 - Investigate failing webdriver tests from wpt downstream sync bug 1709044

Status: RESOLVED WORKSFORME

(Testing :: geckodriver, defect, P2)

Description

[Henrik Skupin [:whimboo][⌚️UTC+2]]

A recent downstream sync for Remove use of six.integer_types/string_types/text_type (bug 1709044) brought a lot of failing webdriver tests:

https://hg.mozilla.org/integration/autoland/rev/1b852017ee57

It's mostly related to user prompt tests and browsing contexts. Seeing the actual changes it's not clear to me why those tests fail now.

Getting lots of these meta data silently added, and not pro-actively checking the wpt sync bugs in detail for those, a test suite will not be trustworthy at some time. Lets discuss in our next triage meeting how we can better handle that.

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Actually not sure what happened here. While the above mentioned revision shows the file as being present, I do not see it anymore in the latest revision:

https://hg.mozilla.org/mozilla-central/file/1b852017ee57c3b442024b93931041458414440d/testing/web-platform/meta/webdriver/tests/take_element_screenshot/iframe.py.ini

https://hg.mozilla.org/mozilla-central/file/tip/testing/web-platform/meta/webdriver/tests/take_element_screenshot/iframe.py.ini

Also hg log doesn't show anything that this file got removed. James, what's going on here?

Comment 2

[James Graham [:jgraham]]

Lookd like the try run had some new failures, but they disappeared in the landing commit. Could mean that the failures went away with a subsequent commit or that the try run was outdated/misleading for some reason (there was definiely some churn with the PR with earlier revisions causing regressions).

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Going to mark as WFM given that no new multiple status changes have finally been landed.