Last Comment Bug 503480 - [Linux] synthesizeKey does not send characters to input fields when running tests via the IDE
: [Linux] synthesizeKey does not send characters to input fields when running t...
Status: RESOLVED WONTFIX
[qae-p1][mozmill-1.4.2-]
:
Product: Testing
Classification: Components
Component: Mozmill (show other bugs)
: unspecified
: x86 Linux
: P2 major (vote)
: ---
Assigned To: Heather Arthur [:harth]
:
Mentors:
Depends on: 497839 504468
Blocks: 521762
  Show dependency treegraph
 
Reported: 2009-07-10 03:53 PDT by Henrik Skupin (:whimboo)
Modified: 2012-09-26 03:36 PDT (History)
11 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---


Attachments
testcase (1.12 KB, text/html)
2009-07-15 11:52 PDT, Martijn Wargers [:mwargers] (not working for Mozilla)
no flags Details
Wait for window to be focused before sending key events (2.37 KB, patch)
2009-10-21 20:17 PDT, Heather Arthur [:harth]
no flags Details | Diff | Review
just do setTimeout (1.58 KB, patch)
2009-10-22 13:34 PDT, Heather Arthur [:harth]
hskupin: review-
Details | Diff | Review

Description Henrik Skupin (:whimboo) 2009-07-10 03:53:09 PDT
On Linux the synthesizeKey function inside the EventUtils doesn't send simple characters to the given input field. Shortcuts work instead. Right now this behavior can only be seen on Gnome systems. As Merike reported back it doesn't happen for distibutions which use KDE like Kubuntu.

To test this behavior use Ubuntu 8.10/9.04 and simply install the new Mozmill 1.2 extension in your profile. Run the default template.

After some investigations I believe that something is wrong in nsiDOMWindowUtils. But I wonder why our buildbots who run Mochitests still work. They use the same implementation.
Comment 1 Merike (:merike) 2009-07-10 11:10:53 PDT
Installed ubuntu-desktop (for gnome on KDE-based distribution) and ran https://bugzilla.mozilla.org/attachment.cgi?id=387640 with Mozmill 1.2.

Didn't have any issues.
Comment 2 Henrik Skupin (:whimboo) 2009-07-10 13:42:21 PDT
So this only happens for Firefox but not for Thunderbird. Also it fails in Firefox with KDE or Gnome desktop installed as Merike told me on IRC.

Could this be a product bug?
Comment 3 Martijn Wargers [:mwargers] (not working for Mozilla) 2009-07-15 11:52:46 PDT
Created attachment 388748 [details]
testcase

It works for me, using Ubuntu8.04 on Gnome, with this testcase.
It seems to me this is likely a bug with the Mozmill script. Is the input and the window correctly focused?
Comment 4 Henrik Skupin (:whimboo) 2009-07-15 14:33:02 PDT
Martijn, eventually the problem lays inside EventUtils.js? The way how a keypress is emulated is quiet a bit different to your way:

http://mxr.mozilla.org/mozilla-central/source/testing/mochitest/tests/SimpleTest/EventUtils.js#298

Can it happen that the keypress is not send?
Comment 5 Henrik Skupin (:whimboo) 2009-07-17 00:02:43 PDT
See bug 497839 comment 11 why that happens! I was wrong when writing the patches because I haven't known that we block the main browser test. So after focusing the window we should also use a sleep call as given in bug
Comment 6 Henrik Skupin (:whimboo) 2009-08-18 02:42:24 PDT
Mikeal, this is strange. I installed the Mozmill command line client and the tests are executed normally. Any keypress call will succeed. But running the same tests from within the IDE it always fail. No characters are getting entered into text boxes. I wonder why that only occurs on Linux and no-where else.
Comment 7 Mikeal Rogers [:mikeal] (mrogers@) 2009-08-18 10:05:16 PDT
Is the focus getting moved from the IDE window to the Browser window?
Comment 8 Henrik Skupin (:whimboo) 2009-08-18 10:09:00 PDT
Yes, the focus is on the browser window. But I wonder if it is somehow related to bug 500249. Looks like we have some issues with the IDE.
Comment 9 Heather Arthur [:harth] 2009-10-21 20:17:35 PDT
Created attachment 407690 [details] [diff] [review]
Wait for window to be focused before sending key events

So I'm pretty sure this is an issue with focus, the code that I attached seems to fix this. When sending a key event, if the window is not already focused, it will focus the window and wait for its focus event to fire and then send the key event after a setTimeout call (Just waiting for onfocus doesn't seem to be enough). It mimics Neil's waitForFocus function in mochitest's SimpleTest.js.

This is kind of important because controller.open depends on this code, so controller.open doesn't work on Linux right now.
Comment 10 Karl Tomlinson (ni?:karlt) 2009-10-21 20:32:55 PDT
(In reply to comment #9)
> (Just waiting for onfocus doesn't seem to be enough).

I don't know details but that sounds like a bug, and probably one that could be fixed.  If you think that it is a Core issue, then it would deserve a bug report.
Comment 11 Heather Arthur [:harth] 2009-10-21 20:53:57 PDT
(In reply to comment #10)
> (In reply to comment #9)
> > (Just waiting for onfocus doesn't seem to be enough).
> 
> I don't know details but that sounds like a bug, and probably one that could be
> fixed.  If you think that it is a Core issue, then it would deserve a bug
> report.

Yeah, this does seem like a bug. What made me hesitant to file is that Neil also used setTimeout in his function to wait for focus, I figured there was a reason. Neil, any insight on the need for setTimeout?
Comment 12 Neil Deakin 2009-10-22 01:15:31 PDT
The timeout in SimpleTest.waitForFocus probably isn't needed. I added it currently temporarily in case there were tests that did need it as I was more interested in seeing if waitForFocus fixed the random test failures. As it did fix them, it would be worth investigating removing the timeout (as well as the debugging info).

For your tests though, I don't know what the issue is. Could be specific to the test, for instance, the need to wait for load or a paint to occur.
Comment 13 Henrik Skupin (:whimboo) 2009-10-22 04:13:48 PDT
Comment on attachment 407690 [details] [diff] [review]
Wait for window to be focused before sending key events

>-    // Only focus the element when it's not focused yet
>-    if (!node)
>-      element.focus();
>+
>+      // Only focus the element when it's not focused yet
>+      try {
>+        element.focus();
>+        EventUtils.synthesizeKey(aKey, modifiers, win);
>+      } catch(e) {
>+        throw new Error("Synthesizing key event failed. \n"+e)
>+      }

So you always focus the element now. The !node check is necessary. Otherwise already entered characters by using type will be completely selected.

>+  if(fm.activeWindow)
>+    fm.getFocusedElementForWindow(fm.activeWindow, true, focusedWindow); 

When calling this function we could use the return value to determine if the element which should receive the key event is already focused. That way we could eliminate the following line;

> var focusedElement = utils.getChromeWindow(win).document.commandDispatcher.focusedElement;

Otherwise nice approach Heather!
Comment 14 Heather Arthur [:harth] 2009-10-22 11:44:55 PDT
(In reply to comment #12)
> For your tests though, I don't know what the issue is. Could be specific to the
> test, for instance, the need to wait for load or a paint to occur.

Hmm, that's interesting. These tests are trying to type into the url bar in Firefox, so they wouldn't need to wait for a load or for the url bar to appear.

(In reply to comment #13)
> >+  if(fm.activeWindow)
> >+    fm.getFocusedElementForWindow(fm.activeWindow, true, focusedWindow); 
> 
> When calling this function we could use the return value to determine if the
> element which should receive the key event is already focused. That way we
> could eliminate the following line;
> 
> > var focusedElement = utils.getChromeWindow(win).document.commandDispatcher.focusedElement;

Can't really depend on fm.activeWindow working. Also, I completely forgot that this focus manager code will only work on 3.6+, so that's no good.

I'll look into this a bit more.
Comment 15 Heather Arthur [:harth] 2009-10-22 13:34:39 PDT
Created attachment 407841 [details] [diff] [review]
just do setTimeout

Alright, I propose just putting a setTimeout in until we can figure out the correct way to deal with this.
Comment 16 Henrik Skupin (:whimboo) 2009-10-26 03:47:33 PDT
Heather, I have problems to simulate the current behavior. Do you have a testcase where you can always see this bug happen?
Comment 17 Heather Arthur [:harth] 2009-10-26 14:06:51 PDT
for me just saying window.open("www.weather.com"); fails (nothing gets typed into the url bar)...I've only tried on Ubuntu 9.04 though.
Comment 18 Heather Arthur [:harth] 2009-10-26 14:07:15 PDT
sorry, controller.open() not window.open()...
Comment 19 Henrik Skupin (:whimboo) 2009-10-27 03:07:37 PDT
Comment on attachment 407841 [details] [diff] [review]
just do setTimeout

Heather, have you tested this patch with a complete test run against all Firefox tests? When I do this with your patch applied it fails all over the place when we call synthesizeKey really fast which will happen eg. for the Private Browsing tests. As result textboxes (the location bar in that case) aren't cleared anymore and text is added at the end even when the element hadn't focus before.

Even entering a couple of characters will give us a noticeable lag when we use 200ms as timeout for each call.
Comment 20 Heather Arthur [:harth] 2009-10-27 13:12:06 PDT
Interesting, this test will not work on Ubuntu 9.04:

var locationBar = new elementslib.ID(controller.window.document, "urlbar");
controller.keypress(null, "l", {accelKey: true});
controller.type(locationBar, "http://www.google.com");

But this test works:

var locationBar = new elementslib.ID(controller.window.document, "urlbar");
controller.window.focus();
controller.sleep(0);
controller.keypress(null, "l", {accelKey: true});
controller.type(locationBar, "http://www.google.com");

And this test works:

var locationBar = new elementslib.ID(controller.window.document, "urlbar");
controller.keypress(null, "l", {accelKey: true});
controller.sleep(0);
controller.type(locationBar, "http://www.google.com");

The fact that this is failing via the IDE only makes me think it's a focus issue even more. Maybe instead of putting a setTimeout around every keypress isn't the answer, but exposing something like a 'focusWindow' method (that will focus the window and sleep) is?
Comment 21 Henrik Skupin (:whimboo) 2009-10-28 05:44:10 PDT
That remembers me that we have to call sleep(0) after each key or mouse event. Right now we only do that for mouse events but not for keypress. I'm still not able to reproduce it locally but could you check if the following change works for you?

http://github.com/whimboo/mozmill/commit/e332c3ecb04515a0686ed9b48c1cf42d643b759a
Comment 22 Heather Arthur [:harth] 2009-10-28 12:48:58 PDT
The first keypress event will be lost this way, but calling sleep(0) after focusing the window (in triggerKeyEvent in events.js) and before simulating the key event does seem to work though!
Comment 23 Henrik Skupin (:whimboo) 2009-10-28 15:14:15 PDT
After a discussion on IRC we want to go that way:
http://github.com/whimboo/mozmill/commit/102e5ec47c2fcfae86d61bdfe54f265edaab2848

It fixes the problems for Heather. Mikeal please pull into your master. thanks
Comment 24 Henrik Skupin (:whimboo) 2009-10-28 15:49:01 PDT
Patch has been merged to Mikeal's master.
Comment 25 Merike (:merike) 2009-10-30 09:59:41 PDT
(In reply to comment #22)
> The first keypress event will be lost this way, but calling sleep(0) after
> focusing the window (in triggerKeyEvent in events.js) and before simulating the
> key event does seem to work though!

I'm seeing the first character getting lost. I take it that you're not seeing it any more as this bug is marked fixed?
Comment 26 Heather Arthur [:harth] 2009-10-30 10:55:47 PDT
Yeah, this doesn't seem to always work. I just lost the first two characters. So we'll have to sleep for longer. I would seriously recommend finding a way to test if the window is already focused, and only focusing it (and sleeping) if it isn't already focused.
Comment 27 Mikeal Rogers [:mikeal] (mrogers@) 2009-10-30 11:43:18 PDT
Is there an API outside of accessibility that can tell you what the focus window is?
Comment 28 Henrik Skupin (:whimboo) 2009-11-06 10:35:40 PST
Neil, on 1.9.2 we have the waitForFocus function for Mochitests to make sure the tests start once the browser window has been focus. It's the work Heather has been done above (comment 9). Sadly this is not working on 1.9.1. So how do you do the focus stuff there to make sure the correct window has been focused? I think that's really the better way to go.

http://mxr.mozilla.org/mozilla1.9.2/source/testing/mochitest/tests/SimpleTest/SimpleTest.js#231
Comment 29 Henrik Skupin (:whimboo) 2009-11-06 15:59:31 PST
Until we have a final solution we decided to increase the sleep value to 5ms for now. Given by the tests from Heather it will fix.

Merike, we will upload the 1.3 beta version to github. Can you please verify that this change fixes your problem too?
Comment 30 Merike (:merike) 2009-11-08 23:11:01 PST
(In reply to comment #29)
> Until we have a final solution we decided to increase the sleep value to 5ms
> for now. Given by the tests from Heather it will fix.
> 
> Merike, we will upload the 1.3 beta version to github. Can you please verify
> that this change fixes your problem too?

Unfortunately, no. It still skips first character, once even two.
Comment 31 Henrik Skupin (:whimboo) 2009-11-09 05:19:10 PST
Merike, can you please tell us in detail which application version you are using and which test your performing. Even some STR would be helpful.
Comment 32 Merike (:merike) 2009-11-09 12:14:53 PST
(In reply to comment #31)
> Merike, can you please tell us in detail which application version you are
> using and which test your performing. Even some STR would be helpful.

1.3b1 and running http://hg.mozilla.org/comm-central/file/81be9737934c/calendar/test/mozmill/testTaskView.js (needs addition of ./ to modules path). This is against Lightning and therefore 1.9.1. On Firefox 3.6b I can't repeat it. Also most of Thunderbird inputs work just fine, always have for me. So maybe there's something about that specific input. Whatever the case there was nothing typed before adding that sleep so the situation has improved.
Comment 33 Mikeal Rogers [:mikeal] (mrogers@) 2009-12-18 10:51:47 PST
The our entire event system works is going to change for e10s.

I think that this bug, along with a few others where the workaround is a sleep() call are all going change. I don't know if it's worth trying to invest in a fix for this before we do the e10s work.

From the sound of it, this is a pretty hairy bug. We might want to see what happens if we do sleep(0) before/after each keypress call.
Comment 34 Henrik Skupin (:whimboo) 2012-09-26 03:36:16 PDT
We have dropped support for the Mozmill IDE in the upcoming Mozmill 2.0 release. Therefore it finally has been removed from the extension by bug 791625. Closing the bug as WONTFIX.

Note You need to log in before you can comment on or make changes to this bug.