clipboard tests that pass with manual testing fail when automated, strange hangs in Action, focus issues?

NEW
Assigned to

Status

Testing
Marionette
2 years ago
2 years ago

People

(Reporter: hallvors, Assigned: automatedtester)

Tracking

Trunk
Points:
---

Firefox Tracking Flags

(firefox46 affected)

Details

(Reporter)

Description

2 years ago
I've attempted to automate the upcoming test suite for the W3C clipboard API, but I'm seeing odd problems that I think are issues with Marionette.

The tests generally follow a pattern similar to this:
1) Load test page
2) Write some initial value to the clipboard
3) Simulate shortcut key presses like ctrl-v or ctrl-c, alternatively simulate user clicking button in page
4) Check either clipboard contents or text inside the page to see if it's a pass or a fail

Problems:
* Sometimes the test appears to run fine but concludes with a failure even though actual manual testing says pass.
* Sometimes the Marionette script hangs where it's supposed to perform() the key press actions.

To test this (sorry it's a bit involved):

1) Get a clone of 
https://github.com/w3c/web-platform-tests/tree/hallvors/clipboard-automation
2) Run a server in web-platform-tests either by using the 'serve' script (requires getting git submodules too) or by simply doing "python -m SimpleHTTPServer" as nothing in this part of the test suite requires special features on the backend.
3) Make sure you have a Marionette-enabled desktop instance running
4) Try to run a test - for example like this

python clipboard-apis/support/automation/marionette_single_test.py http://localhost:8000/clipboard-apis/getdata_method_in_paste_event_retrieving_plain_text_paste_on_document.html

It should run OK and say pass (given a build with sufficient clipboard-api support of course)
(Reporter)

Updated

2 years ago
Flags: needinfo?(dburns)
(Reporter)

Comment 1

2 years ago
(some of the copy tests fail in writing to the clipboard and my speculation is that the focus/selection isn't set correctly until after the keyboard simulation run - but that's mainly guessing)
(Assignee)

Updated

2 years ago
Assignee: nobody → dburns
Flags: needinfo?(dburns)
You need to log in before you can comment on or make changes to this bug.