1088534 - Add a test to verify that basic Firefox operations work ok with the enabled complete theme or persona

Status: RESOLVED WONTFIX

(Mozilla QA Graveyard :: Mozmill Tests, defect)

Description

[Cosmin Malutan, [:cosmin-malutan]]

We need to cover the last step from the moztrap below to have full automation coverage.

My suggestion is to make it a mozmill tests, since we have libraries for all the steps needed, and for chrome we have the helper functions in diffrent head.js files and it will be harder to develop the tests

https://moztrap.mozilla.org/manage/case/11507/

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

This cannot be a browser-chrome test? None of those actions require a restart or shotcuts.

Comment 2

[Cosmin Malutan, [:cosmin-malutan]]

It can, it was only a suggestion.

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+2]]

If we don't have specific requirements for a Mozmill test, we should always think about a browser chrome or mochitest first. Those will be run per check-in and will inform about regressions immediately. Only if it cannot be done as such a test, Mozmill would be an option.

Comment 4

[Andreea Matei [:AndreeaMatei]]

I will continue here.

Comment 5

[Mihaela Velimiroviciu (:mihaelav)]

Attachment: v1

try: https://treeherder.mozilla.org/#/jobs?repo=try&revision=b3f924381b71

Gijs, can you please help with reviewing this or suggest someone else who can? Thank you!

Comment 6

[:Gijs (he/him)]

Comment on attachment 8546722 [details] [diff] [review]
v1

Review of attachment 8546722 [details] [diff] [review]:
-----------------------------------------------------------------

Why the head.js changes? The functions don't seem in use, and most of them are already in there in one form or another (esp. waitFor).

The summary also talks about complete themes, not just lightweight themes. You'd need a restart for complete themes, so probably you need

The patch also relies on theme.xpi from a wildly different directory, which seems a little scary...

Finally, is it really useful to do this kind of testing in an automated way? I expect that if the manual tests found issues before, it's because there was a human sitting in front of the computer that could also see and think. I'm much more skeptical about the kind of testing this code does to find similar issues...

All in all I would be inclined to wontfix. :-\

Comment 7

[:Gijs (he/him)]

(In reply to :Gijs Kruitbosch from comment #6)
> The summary also talks about complete themes, not just lightweight themes.
> You'd need a restart for complete themes, so probably you need

... a marionette test.

Comment 8

[Sylvestre Ledru [:Sylvestre]]

Mozmill is dead, WONTFIX the remaining bugs.