1025119 - HTMLElement.send_keys doesn't work for content in loop panels

Status: RESOLVED INVALID

(Remote Protocol :: Marionette, defect, P1)

Description

[Dan Mosedale (:dmosedale, :dmose)]

The test case is at <https://github.com/dmose/gecko-dev/compare/marionette-systems>.  My speculation is that this is related marionette.set_context(marionette.CONTEXT_CONTENT) simply assuming that it should doing stuff in the "content-primary" XUL <browser> element, whereas the loop & social api panels and chat windows are other "content" XUL <browser> elements.

mdas and I spent a bunch of time going over this using hangouts.  She has some thoughts about how to approach this and will fill those in.

Comment 1

[Malini Das [:mdas] - Away, not checking bugmail]

Attachment: test_get_url.py

This isn't a bug, the script was trying to interact with the wrong element. This attachment works with today's Nightly build. I have notes in the file about what elements I'm interacting with.

If you can confirm, we can close the bug

Comment 2

[Malini Das [:mdas] - Away, not checking bugmail]

btw, you may need to put a sleep before line 55, since marionette returns faster than the field can be populated with the url.

Comment 3

[Dan Mosedale (:dmosedale, :dmose)]

Comment on attachment 8442346 [details]
test_get_url.py

This is super helpful; thanks for helping diagnose.  Some questions...

>from marionette import MarionetteTestCase
>
>class TestGetUrl(MarionetteTestCase):
>    def setUp(self):
>        MarionetteTestCase.setUp(self)
>
>        # this is browser chrome, kids, not the content window just yet
>        self.marionette.set_context("chrome")
>
>    def switch_to_panel(self):
>        button = self.marionette.find_element("id", "loop-call-button")
>
>        # click the element
>        button.click()
>
>        # switch to the frame
>        #frame = self.marionette.find_element("id", "loop-panel-frame")
>        fs = self.marionette.find_elements("tag name", "iframe")
>        self.marionette.switch_to_frame(fs[0])

This change seems more fragile than the original (commented-out) code.  What was the intent here?

>        # this doesn't work on the loop API frame, but it may not matter
>        #self.marionette.set_context(self.marionette.CONTEXT_CONTENT)
>
>        # XXX how can we ensure page load has complete?  executeScript?
>
>    def test_get_url(self):
>
>
>        self.switch_to_panel()
>        # NOTE:  we wait for document.readyState to be complete, but url_input isn't available yet,
>        # So I increased the wait period for the element to show up in the dom
>        self.marionette.set_search_timeout(10)

This seems a bit fragile too.  Is there anyway to detect that the page has finished loading?

>        # NOTE: retrieving <input type="text" required="" data-l10n-id="caller" name="caller"....
>        url_input = self.marionette.execute_script("return document.getElementsByName('caller')")[0]

I'm confused.  Why prefer execute_script to find_element?

>        self.assertEqual(url_input.get_attribute('value'), u'',
>                         "call-url element not initially empty")
>        # NOTE: clicking isn't necessary for send_keys
>        #url_input.click()

I was wondering about that.  However, it's presumably a better simulation of how the user will interact with the app, so I'm inclined to leave it.  Thoughts?

>        self.assertTrue(url_input.is_selected, "clicking the URL input "
>                                              "did not select it")
>
>        # XXX this doesn't work, i bet because of the set_context issue
>        url_input.send_keys("rheeeeet")
>
>        get_url_button = self.marionette.find_element("class name",
>                                                      "get-url")
>        get_url_button.click()
>        # XXX can we wait for event?
>
>        print "value after click"
>        #NOTE: not sure how the dom is set up, but we need to look at this object for the value
>        # This is <input type="url" readonly="readonly" id="call-url" />
>        call_url = self.marionette.find_element("id", "call-url")
>        print call_url.get_attribute("value")

I don't understand the NOTE above.  Can you give more context?

Thanks!

Comment 4

[Malini Das [:mdas] - Away, not checking bugmail]

(In reply to Dan Mosedale (:dmose - needinfo? me for responses) from comment #3)
> Comment on attachment 8442346 [details]
> test_get_url.py
> 
> This is super helpful; thanks for helping diagnose.  Some questions...
> 
> >from marionette import MarionetteTestCase
> >
> >class TestGetUrl(MarionetteTestCase):
> >    def setUp(self):
> >        MarionetteTestCase.setUp(self)
> >
> >        # this is browser chrome, kids, not the content window just yet
> >        self.marionette.set_context("chrome")
> >
> >    def switch_to_panel(self):
> >        button = self.marionette.find_element("id", "loop-call-button")
> >
> >        # click the element
> >        button.click()
> >
> >        # switch to the frame
> >        #frame = self.marionette.find_element("id", "loop-panel-frame")
> >        fs = self.marionette.find_elements("tag name", "iframe")
> >        self.marionette.switch_to_frame(fs[0])
> 
> This change seems more fragile than the original (commented-out) code.  What
> was the intent here?
A frame with that id didn't exist in yesterday's Nightly, so this was the only way I could find that frame.
> 
> >        # this doesn't work on the loop API frame, but it may not matter
> >        #self.marionette.set_context(self.marionette.CONTEXT_CONTENT)
> >
> >        # XXX how can we ensure page load has complete?  executeScript?
> >
> >    def test_get_url(self):
> >
> >
> >        self.switch_to_panel()
> >        # NOTE:  we wait for document.readyState to be complete, but url_input isn't available yet,
> >        # So I increased the wait period for the element to show up in the dom
> >        self.marionette.set_search_timeout(10)
> 
> This seems a bit fragile too.  Is there anyway to detect that the page has
> finished loading?

Not from marionette server side. When switching frames, since DOMFrameContentLoaded won't always be fired (since we can switch into frames that have already been loaded), we detect if the document's readyState is complete before we return to you.

From the client side, you can use the Wait module (from marionette.wait import Wait) and do some similar code to this: https://github.com/mozilla-b2g/gaia/blob/master/tests/python/gaia-ui-tests/gaiatest/gaia_test.py#L858

> 
> >        # NOTE: retrieving <input type="text" required="" data-l10n-id="caller" name="caller"....
> >        url_input = self.marionette.execute_script("return document.getElementsByName('caller')")[0]
> 
> I'm confused.  Why prefer execute_script to find_element?

this input element has no "id", so I'm getting the element by its name. If you update your source to give it an id, then you can find it by id.
> 
> >        self.assertEqual(url_input.get_attribute('value'), u'',
> >                         "call-url element not initially empty")
> >        # NOTE: clicking isn't necessary for send_keys
> >        #url_input.click()
> 
> I was wondering about that.  However, it's presumably a better simulation of
> how the user will interact with the app, so I'm inclined to leave it. 
> Thoughts?
Sure, go for it!
> 
> >        self.assertTrue(url_input.is_selected, "clicking the URL input "
> >                                              "did not select it")
> >
> >        # XXX this doesn't work, i bet because of the set_context issue
> >        url_input.send_keys("rheeeeet")
> >
> >        get_url_button = self.marionette.find_element("class name",
> >                                                      "get-url")
> >        get_url_button.click()
> >        # XXX can we wait for event?
> >
> >        print "value after click"
> >        #NOTE: not sure how the dom is set up, but we need to look at this object for the value
> >        # This is <input type="url" readonly="readonly" id="call-url" />
> >        call_url = self.marionette.find_element("id", "call-url")
> >        print call_url.get_attribute("value")
> 
> I don't understand the NOTE above.  Can you give more context?
When I call send_keys() on the input element named 'caller', and I can see the text goes in that field, but if I look at the value of that element, it is blank. The value is actually stored in the input element with id 'call-url'. I can confirm that we are sending keys to the 'caller' element from marionette server, so I don't know why the value goes to call-url. If we send_keys to call-url, nothing happens.

I can't manually inspect this frame with DOM Inspector to confirm if this behaviour happens if you manually input keys either, since the loop frame disappears when you click outside of Nightly.
> 
> Thanks!

Comment 5

[Dan Mosedale (:dmosedale, :dmose)]

(In reply to Malini Das [:mdas] from comment #4)
> (In reply to Dan Mosedale (:dmose - needinfo? me for responses) from comment
> #3)
> > Comment on attachment 8442346 [details]
> > test_get_url.py
> > 
> > This is super helpful; thanks for helping diagnose.  Some questions...
> > 
> > >from marionette import MarionetteTestCase
> > >
> > >class TestGetUrl(MarionetteTestCase):
> > >    def setUp(self):
> > >        MarionetteTestCase.setUp(self)
> > >
> > >        # this is browser chrome, kids, not the content window just yet
> > >        self.marionette.set_context("chrome")
> > >
> > >    def switch_to_panel(self):
> > >        button = self.marionette.find_element("id", "loop-call-button")
> > >
> > >        # click the element
> > >        button.click()
> > >
> > >        # switch to the frame
> > >        #frame = self.marionette.find_element("id", "loop-panel-frame")
> > >        fs = self.marionette.find_elements("tag name", "iframe")
> > >        self.marionette.switch_to_frame(fs[0])
> > 
> > This change seems more fragile than the original (commented-out) code.  What
> > was the intent here?
> A frame with that id didn't exist in yesterday's Nightly, so this was the
> only way I could find that frame.

The frame only exists (ie is inserted into the DOM) after the button is pressed.

> > >    def test_get_url(self):
> > >
> > >
> > >        self.switch_to_panel()
> > >        # NOTE:  we wait for document.readyState to be complete, but url_input isn't available yet,
> > >        # So I increased the wait period for the element to show up in the dom
> > >        self.marionette.set_search_timeout(10)
> > 
> > This seems a bit fragile too.  Is there anyway to detect that the page has
> > finished loading?
> 
> Not from marionette server side. When switching frames, since
> DOMFrameContentLoaded won't always be fired (since we can switch into frames
> that have already been loaded), we detect if the document's readyState is
> complete before we return to you.
> 
> From the client side, you can use the Wait module (from marionette.wait
> import Wait) and do some similar code to this:
> https://github.com/mozilla-b2g/gaia/blob/master/tests/python/gaia-ui-tests/
> gaiatest/gaia_test.py#L858

Good pointer; thanks!  More in a bit.

Comment 6

[Dan Mosedale (:dmosedale, :dmose)]

> > >        # NOTE: retrieving <input type="text" required="" data-l10n-id="caller" name="caller"....
> > >        url_input = self.marionette.execute_script("return document.getElementsByName('caller')")[0]
> > 
> > I'm confused.  Why prefer execute_script to find_element?
> 
> this input element has no "id", so I'm getting the element by its name. If
> you update your source to give it an id, then you can find it by id.

Whoop!  Good catch.  I've added a class to the element so that it's possible to grab with find_element now.


> > >        self.assertTrue(url_input.is_selected, "clicking the URL input "
> > >                                              "did not select it")
> > >
> > >        # XXX this doesn't work, i bet because of the set_context issue
> > >        url_input.send_keys("rheeeeet")
> > >
> > >        get_url_button = self.marionette.find_element("class name",
> > >                                                      "get-url")
> > >        get_url_button.click()
> > >        # XXX can we wait for event?
> > >
> > >        print "value after click"
> > >        #NOTE: not sure how the dom is set up, but we need to look at this object for the value
> > >        # This is <input type="url" readonly="readonly" id="call-url" />
> > >        call_url = self.marionette.find_element("id", "call-url")
> > >        print call_url.get_attribute("value")
> > 
> > I don't understand the NOTE above.  Can you give more context?
> When I call send_keys() on the input element named 'caller', and I can see
> the text goes in that field, but if I look at the value of that element, it
> is blank. The value is actually stored in the input element with id
> 'call-url'. I can confirm that we are sending keys to the 'caller' element
> from marionette server, so I don't know why the value goes to call-url. If
> we send_keys to call-url, nothing happens.

My original test had bugs in it, which made it hard for you to see the intent.  As it turns out the app is behaving as written, and when I fix the test to check that it works.

Thanks so much for your help here!  Marking RESOVLED/INVALID.