1789355 - Perma Android /mozilla/tests/webdriver/execute_async_script/execute_async.py | test_no_abort_by_user_prompt_in_other_tab - AssertionError:

Status: RESOLVED FIXED

(Remote Protocol :: Marionette, defect, P5)

Description

[Henrik Skupin [:whimboo][⌚️UTC+1]]

With bug 1506782 fixed this is a new failure that appears when running the Mozilla specific WebDriver tests on Android. Here the failure details from a local run:

/_mozilla/webdriver/execute_async_script/execute_async.py
  FAIL test_no_abort_by_user_prompt_in_other_tab[alert] - AssertionError
session = <Session 19e572a9-0dbe-4405-b3f4-043d91624dc8>, inline = <function inline.<locals>.inline at 0x104dfbb80>, dialog_type = 'alert'

    @pytest.mark.parametrize("dialog_type", ["alert", "confirm", "prompt"])
    def test_no_abort_by_user_prompt_in_other_tab(session, inline, dialog_type):
        original_handle = session.window_handle
        original_handles = session.handles

        session.url = inline(
            """
          <a onclick="window.open();">open window</a>
          <script>
            window.addEventListener("message", function (event) {{
              {}("foo");
            }});
          </script>
        """.format(
                dialog_type
            )
        )

        session.find.css("a", all=False).click()
        wait = Poll(session, timeout=5, message="No new window has been opened")
        new_handles = wait.until(lambda s: set(s.handles) - set(original_handles))
        assert len(new_handles) == 1

        session.window_handle = new_handles.pop()

        response = execute_async_script(
            session,
            """
            const resolve = arguments[0];

            // Trigger opening a user prompt in the other window.
            window.opener.postMessage("foo", "*");

            // Delay resolving the Promise to ensure a user prompt has been opened.
            setTimeout(() => resolve(42), 500);
            """,
        )

>       assert_success(response, 42)

dialog_type = 'alert'
inline     = <function inline.<locals>.inline at 0x104dfbb80>
new_handles = set()
original_handle = 'e93e334e-b4f8-4c40-b489-618215406242'
original_handles = ['e93e334e-b4f8-4c40-b489-618215406242']
response   = <[ValueError('Space not allowed in string format specifier') raised in repr()] Response object at 0x104df3340>
session    = <Session 19e572a9-0dbe-4405-b3f4-043d91624dc8>
wait       = <tests.support.sync.Poll object at 0x104df33a0>

testing/web-platform/mozilla/tests/webdriver/execute_async_script/execute_async.py:55:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

response = <[ValueError('Space not allowed in string format specifier') raised in repr()] Response object at 0x104df3340>, value = 42

    def assert_success(response, value=None):
        """
        Verify that the provided webdriver.Response instance described
        a valid success response as defined by `dfn-send-a-response` and
        the provided response value.

        :param response: ``webdriver.Response`` instance.
        :param value: Expected value of the response body, if any.
        """
        assert response.status == 200, str(response.error)

        if value is not None:
>           assert response.body["value"] == value
E           AssertionError

response   = <[ValueError('Space not allowed in string format specifier') raised in repr()] Response object at 0x104df3340>
value      = 42

And the payload from the call to Execute Async Script:

 0:22.27 pid:70679 1662445887268	webdriver::server	DEBUG	-> POST /session/a9c65549-d978-4dac-abd6-3f0e278b4262/execute/async {"script": "\n        const resolve = arguments[0];\n\n        // Trigger opening a user prompt in the other window.\n        window.opener.postMessage(\"foo\", \"*\");\n\n        // Delay resolving the Promise to ensure a user prompt has been opened.\n        setTimeout(() => resolve(42), 500);\n        ", "args": []}
 0:22.27 pid:70679 1662445887274	webdriver::server	DEBUG	<- 200 OK {"value":null}

Due to some reason null is returned here.

Comment 1

[BMO Automation]

Moving bug to Remote Protocol::Marionette component per bug 1815831.

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Note that this is still a perma-failure for us on Android:
https://treeherder.mozilla.org/jobs?repo=try&revision=220528c922c9d4fe49c2aad726899e7bbb1ce40d&searchStr=android

Here an excerpt from the adb logcat:

08-15 07:46:30.293 19034 19049 I Gecko   : 1692085590293	Marionette	DEBUG	0 -> [0,70,"WebDriver:ExecuteAsyncScript",{"args":[],"script":"\n        const resolve = arguments[0];\n\n        // Trigger opening a user prompt in the other window.\n        window.opener.p ...  // Delay resolving the Promise to ensure a user prompt has been opened.\n        setTimeout(() => resolve(42), 500);\n        "}]
08-15 07:46:30.295 19064 19082 I Gecko   : 1692085590295	Marionette	TRACE	[2147483651] MarionetteCommands actor created for window id 2147483659
08-15 07:46:30.304 19034 19034 D GeckoSession: handleMessage GeckoView:Prompt
08-15 07:46:30.304 19034 19034 D Prompts : handleEvent text
08-15 07:46:30.305 19034 19049 I Gecko   : 1692085590304	Marionette	TRACE	Received observer notification geckoview-prompt-show
08-15 07:46:30.306 19034 19049 I Gecko   : 1692085590306	Marionette	DEBUG	0 <- [1,70,null,{"value":null}]

The test can be found at:
https://searchfox.org/mozilla-central/source/testing/web-platform/mozilla/tests/webdriver/classic/execute_async_script/execute_async.py#17-59

I wonder if we do something wrong in determining if the prompt as detected by the geckoview-prompt-show is for the current window or not. The check that we do is:

     case "geckoview-prompt-show":
        for (let win of Services.wm.getEnumerator(null)) {
          const prompt = win.prompts().find(item => item.id == subject.id);
          if (prompt) {
            this.callbacks.forEach(callback =>
              callback(modal.ACTION_OPENED, prompt)
            );
            return;
          }
        }

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+1]]

The problem might actually be here:
https://searchfox.org/mozilla-central/rev/ce049e593c7d062a039938cabccaab4c14b8ebfd/remote/marionette/driver.sys.mjs#209-210

We do not necessarily check the window but assume that the dialog opened notification we got applies to the current context and we resolve the dialog promise in the Marionette command parent actor.

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Fixed by bug 1824224