918685 - Running mutt tests under FF Metro should not fail

Status: RESOLVED WONTFIX

(Testing Graveyard :: Mozmill, defect)

Description

[Andrei Eftimie]

We have several failures while running mutt tests under Metro.

The following tests fail:
> testElementsLib/testMozElementWindow.js
> testElementsLib/testRadioButtons.js
> testElementsLib/testMozElementWindow.js
> testElementsLib/testMozElement.js
> testElementsLib/testMenuList.js
> testController/testScreenshot.js
> testController/testDndContentChrome.js
> testController/testContextMenu.js

Some of them might be real failures, and we may be able to fix them.

Others like testContextMenu and testMenuList make use of UI features which are not available in Metro at all, or need special handling.

ATM mutt test need to be run directly with mozmill.

We also need to enable mutt to run direclty with metro, like:
> mutt testjs -b %path% --app=metrofirefox
This is handled in bug 895841

I'll look into each failure, see if we can fix it.
For those that can not be run under metro we might need to skip them (find a good way to do that).

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Most of the existing tests are not targeted to be run with Firefox Metro. So something else we need is the application support for manifests. Or we might want to check how this has been done for b2g and Marionette. There you can specify something like 'b2g=true'. So we could have 'metro=true'.

Comment 2

[Andrei Eftimie]

The question is, should mutt test rely on any UI at all?
Most of them test low level API's that don't depend on specific UI features.

We could argue that all of the mutt tests should follow this cree.
The problem is that some functionality might be hard or impossible to test without specific UI features.

But (at least in theory) we should be able to run all mutt tests under metro as well

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+1]]

It will only work for content but not for chrome pages. That's why I tried to avoid to have tests in the mutt suite which rely on Firefox specific chrome elements.

Comment 4

[Andrei Eftimie]

This is not valid anymore.