1242435 - clipboard tests that pass with manual testing fail when automated, strange hangs in Action, focus issues?

Status: RESOLVED INVALID

(Remote Protocol :: Marionette, defect)

Description

[Hallvord R. M. Steen [:hallvors]]

I've attempted to automate the upcoming test suite for the W3C clipboard API, but I'm seeing odd problems that I think are issues with Marionette.

The tests generally follow a pattern similar to this:
1) Load test page
2) Write some initial value to the clipboard
3) Simulate shortcut key presses like ctrl-v or ctrl-c, alternatively simulate user clicking button in page
4) Check either clipboard contents or text inside the page to see if it's a pass or a fail

Problems:
* Sometimes the test appears to run fine but concludes with a failure even though actual manual testing says pass.
* Sometimes the Marionette script hangs where it's supposed to perform() the key press actions.

To test this (sorry it's a bit involved):

1) Get a clone of 
https://github.com/w3c/web-platform-tests/tree/hallvors/clipboard-automation
2) Run a server in web-platform-tests either by using the 'serve' script (requires getting git submodules too) or by simply doing "python -m SimpleHTTPServer" as nothing in this part of the test suite requires special features on the backend.
3) Make sure you have a Marionette-enabled desktop instance running
4) Try to run a test - for example like this

python clipboard-apis/support/automation/marionette_single_test.py http://localhost:8000/clipboard-apis/getdata_method_in_paste_event_retrieving_plain_text_paste_on_document.html

It should run OK and say pass (given a build with sufficient clipboard-api support of course)

Comment 1

[Hallvord R. M. Steen [:hallvors]]

(some of the copy tests fail in writing to the clipboard and my speculation is that the focus/selection isn't set correctly until after the keyboard simulation run - but that's mainly guessing)

Comment 2

[Andreas Tolfsen ❲:ato❳]

Should work with spec conforming actions API.