[Linux] synthesizeKey does not send characters to input fields when running tests via the IDE

RESOLVED WONTFIX

Status

Testing Graveyard
Mozmill
P2
major
RESOLVED WONTFIX
8 years ago
10 months ago

People

(Reporter: whimboo, Assigned: harth)

Tracking

Details

(Whiteboard: [qae-p1][mozmill-1.4.2-])

Attachments

(2 attachments, 1 obsolete attachment)

(Reporter)

Description

8 years ago
On Linux the synthesizeKey function inside the EventUtils doesn't send simple characters to the given input field. Shortcuts work instead. Right now this behavior can only be seen on Gnome systems. As Merike reported back it doesn't happen for distibutions which use KDE like Kubuntu.

To test this behavior use Ubuntu 8.10/9.04 and simply install the new Mozmill 1.2 extension in your profile. Run the default template.

After some investigations I believe that something is wrong in nsiDOMWindowUtils. But I wonder why our buildbots who run Mochitests still work. They use the same implementation.
Installed ubuntu-desktop (for gnome on KDE-based distribution) and ran https://bugzilla.mozilla.org/attachment.cgi?id=387640 with Mozmill 1.2.

Didn't have any issues.
(Reporter)

Comment 2

8 years ago
So this only happens for Firefox but not for Thunderbird. Also it fails in Firefox with KDE or Gnome desktop installed as Merike told me on IRC.

Could this be a product bug?
Created attachment 388748 [details]
testcase

It works for me, using Ubuntu8.04 on Gnome, with this testcase.
It seems to me this is likely a bug with the Mozmill script. Is the input and the window correctly focused?
(Reporter)

Comment 4

8 years ago
Martijn, eventually the problem lays inside EventUtils.js? The way how a keypress is emulated is quiet a bit different to your way:

http://mxr.mozilla.org/mozilla-central/source/testing/mochitest/tests/SimpleTest/EventUtils.js#298

Can it happen that the keypress is not send?
(Reporter)

Comment 5

8 years ago
See bug 497839 comment 11 why that happens! I was wrong when writing the patches because I haven't known that we block the main browser test. So after focusing the window we should also use a sleep call as given in bug
Depends on: 497839, 504468
(Reporter)

Comment 6

8 years ago
Mikeal, this is strange. I installed the Mozmill command line client and the tests are executed normally. Any keypress call will succeed. But running the same tests from within the IDE it always fail. No characters are getting entered into text boxes. I wonder why that only occurs on Linux and no-where else.
(Reporter)

Updated

8 years ago
Severity: critical → major
Summary: [Linux] synthesizeKey does not send characters to input fields → [Linux] synthesizeKey does not send characters to input fields when running tests via the IDE
Is the focus getting moved from the IDE window to the Browser window?
(Reporter)

Comment 8

8 years ago
Yes, the focus is on the browser window. But I wonder if it is somehow related to bug 500249. Looks like we have some issues with the IDE.
(Assignee)

Comment 9

8 years ago
Created attachment 407690 [details] [diff] [review]
Wait for window to be focused before sending key events

So I'm pretty sure this is an issue with focus, the code that I attached seems to fix this. When sending a key event, if the window is not already focused, it will focus the window and wait for its focus event to fire and then send the key event after a setTimeout call (Just waiting for onfocus doesn't seem to be enough). It mimics Neil's waitForFocus function in mochitest's SimpleTest.js.

This is kind of important because controller.open depends on this code, so controller.open doesn't work on Linux right now.
Assignee: nobody → harthur
(In reply to comment #9)
> (Just waiting for onfocus doesn't seem to be enough).

I don't know details but that sounds like a bug, and probably one that could be fixed.  If you think that it is a Core issue, then it would deserve a bug report.
(Assignee)

Comment 11

8 years ago
(In reply to comment #10)
> (In reply to comment #9)
> > (Just waiting for onfocus doesn't seem to be enough).
> 
> I don't know details but that sounds like a bug, and probably one that could be
> fixed.  If you think that it is a Core issue, then it would deserve a bug
> report.

Yeah, this does seem like a bug. What made me hesitant to file is that Neil also used setTimeout in his function to wait for focus, I figured there was a reason. Neil, any insight on the need for setTimeout?
The timeout in SimpleTest.waitForFocus probably isn't needed. I added it currently temporarily in case there were tests that did need it as I was more interested in seeing if waitForFocus fixed the random test failures. As it did fix them, it would be worth investigating removing the timeout (as well as the debugging info).

For your tests though, I don't know what the issue is. Could be specific to the test, for instance, the need to wait for load or a paint to occur.
(Reporter)

Comment 13

8 years ago
Comment on attachment 407690 [details] [diff] [review]
Wait for window to be focused before sending key events

>-    // Only focus the element when it's not focused yet
>-    if (!node)
>-      element.focus();
>+
>+      // Only focus the element when it's not focused yet
>+      try {
>+        element.focus();
>+        EventUtils.synthesizeKey(aKey, modifiers, win);
>+      } catch(e) {
>+        throw new Error("Synthesizing key event failed. \n"+e)
>+      }

So you always focus the element now. The !node check is necessary. Otherwise already entered characters by using type will be completely selected.

>+  if(fm.activeWindow)
>+    fm.getFocusedElementForWindow(fm.activeWindow, true, focusedWindow); 

When calling this function we could use the return value to determine if the element which should receive the key event is already focused. That way we could eliminate the following line;

> var focusedElement = utils.getChromeWindow(win).document.commandDispatcher.focusedElement;

Otherwise nice approach Heather!
(Reporter)

Updated

8 years ago
Blocks: 521762
(Assignee)

Comment 14

8 years ago
(In reply to comment #12)
> For your tests though, I don't know what the issue is. Could be specific to the
> test, for instance, the need to wait for load or a paint to occur.

Hmm, that's interesting. These tests are trying to type into the url bar in Firefox, so they wouldn't need to wait for a load or for the url bar to appear.

(In reply to comment #13)
> >+  if(fm.activeWindow)
> >+    fm.getFocusedElementForWindow(fm.activeWindow, true, focusedWindow); 
> 
> When calling this function we could use the return value to determine if the
> element which should receive the key event is already focused. That way we
> could eliminate the following line;
> 
> > var focusedElement = utils.getChromeWindow(win).document.commandDispatcher.focusedElement;

Can't really depend on fm.activeWindow working. Also, I completely forgot that this focus manager code will only work on 3.6+, so that's no good.

I'll look into this a bit more.
(Assignee)

Comment 15

8 years ago
Created attachment 407841 [details] [diff] [review]
just do setTimeout

Alright, I propose just putting a setTimeout in until we can figure out the correct way to deal with this.
Attachment #407690 - Attachment is obsolete: true
(Reporter)

Comment 16

8 years ago
Heather, I have problems to simulate the current behavior. Do you have a testcase where you can always see this bug happen?
(Assignee)

Comment 17

8 years ago
for me just saying window.open("www.weather.com"); fails (nothing gets typed into the url bar)...I've only tried on Ubuntu 9.04 though.
(Assignee)

Comment 18

8 years ago
sorry, controller.open() not window.open()...
(Reporter)

Comment 19

8 years ago
Comment on attachment 407841 [details] [diff] [review]
just do setTimeout

Heather, have you tested this patch with a complete test run against all Firefox tests? When I do this with your patch applied it fails all over the place when we call synthesizeKey really fast which will happen eg. for the Private Browsing tests. As result textboxes (the location bar in that case) aren't cleared anymore and text is added at the end even when the element hadn't focus before.

Even entering a couple of characters will give us a noticeable lag when we use 200ms as timeout for each call.
Attachment #407841 - Flags: review-
(Assignee)

Comment 20

8 years ago
Interesting, this test will not work on Ubuntu 9.04:

var locationBar = new elementslib.ID(controller.window.document, "urlbar");
controller.keypress(null, "l", {accelKey: true});
controller.type(locationBar, "http://www.google.com");

But this test works:

var locationBar = new elementslib.ID(controller.window.document, "urlbar");
controller.window.focus();
controller.sleep(0);
controller.keypress(null, "l", {accelKey: true});
controller.type(locationBar, "http://www.google.com");

And this test works:

var locationBar = new elementslib.ID(controller.window.document, "urlbar");
controller.keypress(null, "l", {accelKey: true});
controller.sleep(0);
controller.type(locationBar, "http://www.google.com");

The fact that this is failing via the IDE only makes me think it's a focus issue even more. Maybe instead of putting a setTimeout around every keypress isn't the answer, but exposing something like a 'focusWindow' method (that will focus the window and sleep) is?
(Reporter)

Comment 21

8 years ago
That remembers me that we have to call sleep(0) after each key or mouse event. Right now we only do that for mouse events but not for keypress. I'm still not able to reproduce it locally but could you check if the following change works for you?

http://github.com/whimboo/mozmill/commit/e332c3ecb04515a0686ed9b48c1cf42d643b759a
(Assignee)

Comment 22

8 years ago
The first keypress event will be lost this way, but calling sleep(0) after focusing the window (in triggerKeyEvent in events.js) and before simulating the key event does seem to work though!
(Reporter)

Comment 23

8 years ago
After a discussion on IRC we want to go that way:
http://github.com/whimboo/mozmill/commit/102e5ec47c2fcfae86d61bdfe54f265edaab2848

It fixes the problems for Heather. Mikeal please pull into your master. thanks
(Reporter)

Comment 24

8 years ago
Patch has been merged to Mikeal's master.
Status: NEW → RESOLVED
Last Resolved: 8 years ago
Resolution: --- → FIXED
Whiteboard: [mozmill-1.2.4]
(In reply to comment #22)
> The first keypress event will be lost this way, but calling sleep(0) after
> focusing the window (in triggerKeyEvent in events.js) and before simulating the
> key event does seem to work though!

I'm seeing the first character getting lost. I take it that you're not seeing it any more as this bug is marked fixed?
(Assignee)

Comment 26

8 years ago
Yeah, this doesn't seem to always work. I just lost the first two characters. So we'll have to sleep for longer. I would seriously recommend finding a way to test if the window is already focused, and only focusing it (and sleeping) if it isn't already focused.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Is there an API outside of accessibility that can tell you what the focus window is?
(Reporter)

Comment 28

8 years ago
Neil, on 1.9.2 we have the waitForFocus function for Mochitests to make sure the tests start once the browser window has been focus. It's the work Heather has been done above (comment 9). Sadly this is not working on 1.9.1. So how do you do the focus stuff there to make sure the correct window has been focused? I think that's really the better way to go.

http://mxr.mozilla.org/mozilla1.9.2/source/testing/mochitest/tests/SimpleTest/SimpleTest.js#231
Whiteboard: [mozmill-1.2.4]
(Reporter)

Comment 29

8 years ago
Until we have a final solution we decided to increase the sleep value to 5ms for now. Given by the tests from Heather it will fix.

Merike, we will upload the 1.3 beta version to github. Can you please verify that this change fixes your problem too?
(In reply to comment #29)
> Until we have a final solution we decided to increase the sleep value to 5ms
> for now. Given by the tests from Heather it will fix.
> 
> Merike, we will upload the 1.3 beta version to github. Can you please verify
> that this change fixes your problem too?

Unfortunately, no. It still skips first character, once even two.
(Reporter)

Comment 31

8 years ago
Merike, can you please tell us in detail which application version you are using and which test your performing. Even some STR would be helpful.
(In reply to comment #31)
> Merike, can you please tell us in detail which application version you are
> using and which test your performing. Even some STR would be helpful.

1.3b1 and running http://hg.mozilla.org/comm-central/file/81be9737934c/calendar/test/mozmill/testTaskView.js (needs addition of ./ to modules path). This is against Lightning and therefore 1.9.1. On Firefox 3.6b I can't repeat it. Also most of Thunderbird inputs work just fine, always have for me. So maybe there's something about that specific input. Whatever the case there was nothing typed before adding that sleep so the situation has improved.
(Reporter)

Updated

8 years ago
Whiteboard: [qae-p1]
The our entire event system works is going to change for e10s.

I think that this bug, along with a few others where the workaround is a sleep() call are all going change. I don't know if it's worth trying to invest in a fix for this before we do the e10s work.

From the sound of it, this is a pretty hairy bug. We might want to see what happens if we do sleep(0) before/after each keypress call.

Updated

7 years ago
Priority: -- → P2

Updated

7 years ago
Whiteboard: [qae-p1] → [qae-p1][mozmill-1.4.2-]
(Reporter)

Comment 34

5 years ago
We have dropped support for the Mozmill IDE in the upcoming Mozmill 2.0 release. Therefore it finally has been removed from the extension by bug 791625. Closing the bug as WONTFIX.
Status: REOPENED → RESOLVED
Last Resolved: 8 years ago5 years ago
Resolution: --- → WONTFIX
Product: Testing → Testing Graveyard
You need to log in before you can comment on or make changes to this bug.