799406 - Stub installer should support silent (unattended) installation

Status: RESOLVED WONTFIX

(Firefox :: Installer, defect)

Description

[Henrik Skupin [:whimboo][⌚️UTC+1]]

As seen the current version of the stub installer does not support silent installation via /S /D. Not sure if that is by design or not but at least we can't have automation around those installers to test them. Also if we are heading away from full installers in the future we have to give admins the chance to run the installation unattended.

Comment 1

[Robert Strong (they/them - no direct email)]

It would be a much better test if there were a test harness that verifies UI elements and performs clicks than to support silent. Also, the vast majority of admins prefer full installers for unattended installations.

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Robert, does it mean we will definitely keep the full installers? If that's the case we would be ok with it in terms of Mozmill. If you think we do not need a silent mode please close it.

Regarding testing of NSIS installers, is there a test framework out there we could leverage to test the UI? I can remember that I have looked out for one more than a year ago but didn't found something.

Comment 3

[Robert Strong (they/them - no direct email)]

(In reply to Henrik Skupin (:whimboo) from comment #2)
> Robert, does it mean we will definitely keep the full installers? If that's
> the case we would be ok with it in terms of Mozmill. If you think we do not
> need a silent mode please close it.
Yes, we will definitely keep the full installers.

> Regarding testing of NSIS installers, is there a test framework out there we
> could leverage to test the UI? I can remember that I have looked out for one
> more than a year ago but didn't found something.
I used MS Visual Test back in the 90's for this type of testing and I would be highly surprised if there wasn't something similar today. For quick, one off tests I've used vbscript though I don't think it can check UI elements.

Comment 4

[Robert Strong (they/them - no direct email)]

We might one day want to add a silent mode but we shouldn't need it for testing especially since the main thing that need to be tested is the UI so closing.

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+1]]

So in case of possible Mozmill tests which could use the sub installer to install Firefox on the testing machine for further tests, we do not want to test the ui of the stub installer. Instead we want to install Firefox to prove that the right binary gets installed. See bug 909734 for the latest problems. So reopening this bug.

Comment 6

[Robert Strong (they/them - no direct email)]

No, see bug 799844 comment #11. I know it is work for you to get a tool to test with that can interact with ui (you and I discussed this a long time ago as mentioned in the comment I referenced) but better that than to add functionality to code that users use and has to be maintained across all possible supported systems as well as configurations. This is wontfix.

Comment 7

[Robert Strong (they/them - no direct email)]

Henrik, you really need to extend your testing framework instead of wanting to add code that goes to users.

Comment 8

[Henrik Skupin [:whimboo][⌚️UTC+1]]

This is not a feature which Mozmill will ever support, and it can't because it requires Gecko to work. Also all install / uninstall functionality is part of mozinstall (http://pypi.python.org/pypi/mozInstall) which is not part of any test framework.

I would like to get Jeff's opinion here, given that he is one of the owners of mozbase.

Comment 9

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Something we might want to test is AutoIt (http://www.autoitscript.com/site/autoit/). If that works for installing the build I'm happy. As said above we will not test the installer. This is part of bug 799844.

Comment 10

[Robert Strong (they/them - no direct email)]

When I say extend your testing framework I am not saying extend mozmill is the right answer. Just as we have several testing frameworks for testing builds as they are built I am saying that something (where something could be something like visual test) should be used to test these if mozmill can't be extended to do this then another tool should be used. Adding code to the stub installer requires all users to have that code which means more complexity in the code that the clients run, binary size increase for a binary that we are very sensitive of size increases however slight, and additional code to maintain for the couple of people that maintain this code. Also, silent installs are different than UI driven installs so the test wouldn't be testing all of the same code that users actually hit.

Comment 11

[Jeff Hammel]

I'm not in a position to be an authority here, but after some thought have decided that, from my perspective, it is not up to the SUT, in this case, to support the testing system.  If we have the time/luxury/priority to support testing the stub installer...ABICT, it can be done I think. It just may not be with mozmill. Otherwise, as per comment 3, comment 4, and comment 10, there is no show stopper here.

Comment 12

[Henrik Skupin [:whimboo][⌚️UTC+1]]

(In reply to Robert Strong [:rstrong] (do not email) from comment #10)
> code. Also, silent installs are different than UI driven installs so the
> test wouldn't be testing all of the same code that users actually hit.

Robert, I'm not talking about testing the installer ui, but being able to use the stub installer at all to install Firefox. So I will try the Autoit solution for mozinstall and filed bug 911048.

Comment 13

[Robert Strong (they/them - no direct email)]

Henrik, I already understood that you weren't talking about testing the stub installer or installer ui. I am saying that even if there was a silent install method for the stub installer which would go against the other issues I stated your test wouldn't be testing all of the same code that executes when a user installs AND it wouldn't test the UI.