1239988 - Remove packaging for firefox-ui-tests (and its dependency in firefox-ui-harness)

Status: RESOLVED FIXED

(Testing :: Firefox UI Tests, defect)

Description

[Henrik Skupin [:whimboo][⌚️UTC+2]]

As of now we are packaging firefox-ui-tests to be able to access the resources folder and the manifests. We should drop those dependencies and make the tests a normal folder. With that we would also be able to run a single test via mach once bug 1237179 got implemented.

http://hg.mozilla.org/mozilla-central/file/default/testing/firefox-ui/harness/firefox_ui_harness/runners/base.py#l13

I referenced the mirrored code here because it uses the packaging for all parts, which will also land soon in our github repository.

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Removing the package for tests would mean the user has to specify the path to the local test files when calling the firefox ui scripts. To ensure that this would not cause to confusion right now, I would wait with this bug until a proper documentation is in place. See bug 1237552 for details.

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Thinking more about it I would wait even a bit more until we have our tests running via TaskCluster on inbound and try. If we do this now developers won't notice a difference because they use mach, but everyone else who uses the github repo would have to specify the path. I suggest to wait for bug 1237550.

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Attachment: MozReview Request: Bug 1239988 - Remove dependency for firefox-ui-tests in firefox-ui-harness. r?maja_zf

This will remove any dependency for firefox-ui-tests from inside firefox-ui-harness. Affected test types are functional and update tests. To keep update tests working and testable I also added them to the mach command.

Review commit: https://reviewboard.mozilla.org/r/39205/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/39205/

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Attachment: MozReview Request: Bug 1239988 - Remove packaging for firefox-ui-tests. r?maja_zf

The firefox-ui-tests package is no longer required. Removing it will allow us later to move tests to the appropriate places in the tree. The commit also has updates for the build system and mozharness.

Review commit: https://reviewboard.mozilla.org/r/39207/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/39207/

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Attachment: MozReview Request: Bug 1239988 - Move resources for firefox-ui-tests out of tests folder. r?maja_zf

The resources folder should actually not be part of the tests folder. It contains test data for all tests and once tests are moved to other places the tests folder will not exist anymore.

Review commit: https://reviewboard.mozilla.org/r/39209/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/39209/

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Attachment: MozReview Request: Bug 1239988 - Remove top-level manifest for functional and puppeteer tests. r?maja_zf

This was always a hack which made it easier for us to trigger both type of tests at once. Now with mach and mozharness we should finally get rid of it.

Review commit: https://reviewboard.mozilla.org/r/39211/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/39211/

Comment 7

[Maja Frydrychowicz :impossibus (was :maja_zf) (needinfo me)]

Comment on attachment 8728972 [details]
MozReview Request: Bug 1239988 - Remove dependency for firefox-ui-tests in firefox-ui-harness. r?maja_zf

https://reviewboard.mozilla.org/r/39205/#review35905

::: testing/firefox-ui/mach_commands.py:71
(Diff revision 1)
> -    parser.verify_usage(args)
> +    return test_types[testtype]['cli_module'].cli(args=args)

This will return 1, 10, or 0, whereas only 1 or 0 was possible in the original version. I don't know if mach has any assumptions around only 1 or 0 being possible; highlighting just in case.

Comment 8

[Maja Frydrychowicz :impossibus (was :maja_zf) (needinfo me)]

Comment on attachment 8728973 [details]
MozReview Request: Bug 1239988 - Remove packaging for firefox-ui-tests. r?maja_zf

https://reviewboard.mozilla.org/r/39207/#review35909

::: testing/mozharness/mozharness/mozilla/testing/firefox_ui_tests.py:215
(Diff revision 1)
> -            'abs_test_install_dir': tests_dir,
> +            'abs_test_install_dir': abs_tests_install_dir,

It seems you don't really need the abs_tests_install_dir variable.

Comment 9

[Maja Frydrychowicz :impossibus (was :maja_zf) (needinfo me)]

Comment on attachment 8728974 [details]
MozReview Request: Bug 1239988 - Move resources for firefox-ui-tests out of tests folder. r?maja_zf

https://reviewboard.mozilla.org/r/39209/#review35913

Comment 10

[Maja Frydrychowicz :impossibus (was :maja_zf) (needinfo me)]

Comment on attachment 8728975 [details]
MozReview Request: Bug 1239988 - Remove top-level manifest for functional and puppeteer tests. r?maja_zf

https://reviewboard.mozilla.org/r/39211/#review35915

Comment 11

[Henrik Skupin [:whimboo][⌚️UTC+2]]

https://reviewboard.mozilla.org/r/39205/#review35905

> This will return 1, 10, or 0, whereas only 1 or 0 was possible in the original version. I don't know if mach has any assumptions around only 1 or 0 being possible; highlighting just in case.

Maybe I better revert this change then. It's clearly not related to the other changes.

Comment 12

[Henrik Skupin [:whimboo][⌚️UTC+2]]

https://reviewboard.mozilla.org/r/39207/#review35909

> It seems you don't really need the abs_tests_install_dir variable.

We need this for the requirements.txt file under `/tests/configs` below. So I cannot remove this.

Comment 13

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Comment on attachment 8728972 [details]
MozReview Request: Bug 1239988 - Remove dependency for firefox-ui-tests in firefox-ui-harness. r?maja_zf

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/39205/diff/1-2/

Comment 14

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Comment on attachment 8728973 [details]
MozReview Request: Bug 1239988 - Remove packaging for firefox-ui-tests. r?maja_zf

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/39207/diff/1-2/

Comment 15

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Comment on attachment 8728974 [details]
MozReview Request: Bug 1239988 - Move resources for firefox-ui-tests out of tests folder. r?maja_zf

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/39209/diff/1-2/

Comment 16

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Comment on attachment 8728975 [details]
MozReview Request: Bug 1239988 - Remove top-level manifest for functional and puppeteer tests. r?maja_zf

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/39211/diff/1-2/

Comment 17

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/074e0fe57a52
https://hg.mozilla.org/integration/mozilla-inbound/rev/36cd83ba0bb3
https://hg.mozilla.org/integration/mozilla-inbound/rev/76a0add3c8de
https://hg.mozilla.org/integration/mozilla-inbound/rev/1163a9d31ca6

Comment 18

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/074e0fe57a52
https://hg.mozilla.org/mozilla-central/rev/36cd83ba0bb3
https://hg.mozilla.org/mozilla-central/rev/76a0add3c8de
https://hg.mozilla.org/mozilla-central/rev/1163a9d31ca6