Closed Bug 799844 Opened 10 years ago Closed 8 years ago

Create automated UI tests for stub and full installers

Categories

(Firefox :: Installer, defect)

17 Branch
All
Windows XP
defect
Not set
normal

Tracking

()

RESOLVED DUPLICATE of bug 423754

People

(Reporter: whimboo, Unassigned)

References

Details

(Whiteboard: [stub=])

As talked over on bug 799406 it would be very helpful to have a suite of ui tests for the NSIS installers we support. Those are specifically the full and stub installers.

As Rob answered on that bug:

>> Regarding testing of NSIS installers, is there a test framework out there we
>> could leverage to test the UI? I can remember that I have looked out for one
>> more than a year ago but didn't found something.
>
> I used MS Visual Test back in the 90's for this type of testing and I would be
> highly surprised if there wasn't something similar today. For quick, one off
> tests I've used vbscript though I don't think it can check UI elements.

Once we find the time we should check what we can do here, given that Windows is the major platform for us and having no ui tests makes me nervous.
Not blocking ship for v1 of the English deployment, but this is most certainly a want.
Whiteboard: [stub=]
Summary: Create UI tests for stub and full installers → Create automated UI tests for stub and full installers
I haven't checked the details of Cobra but it could be an option if there is no other framework dedicated to NSIS installers is available.
I got in contact with the NSIS team yesterday and asked if there is an ui testing framework available for NSIS. But there isn't. So we have to find an alternative solution.
Blocks: 910459
What's the status and priority of this bug? We'd like to get it resolved as it blocks getting automated coverage for a recent stub installer regression (bug 910459).
Might be a good thing to have a bouncer specific test since that has been very consistently where the problem has happened.
(In reply to Robert Strong [:rstrong] (do not email) from comment #5)
> Might be a good thing to have a bouncer specific test since that has been
> very consistently where the problem has happened.

Who can take care of that? Is that a RelEng responsibility?
Not sure but perhaps Laura Thomson
Robert, I have a side-question. Would it be possible to add at least a silent mode to the stub installer? That way we could at least install Firefox and check if the correct build has been selected.
Possibly at some point in the future but not likely anytime soon.
Oh, I missed that there was bug 799406, which has been marked as wontfix at some point. Looks like it was not clear why we need that mode. I will reopen the other bug.
No longer blocks: 910459
(In reply to Henrik Skupin (:whimboo) from comment #10)
> Oh, I missed that there was bug 799406, which has been marked as wontfix at
> some point. Looks like it was not clear why we need that mode. I will reopen
> the other bug.
Specifically, any organization needing to deploy using a silent installer should be using the full installer. It would be really a great thing if there were a simple test of bouncer vs. adding code to the stub (which we really try to avoid due to size and complexity) for a silent option that deployments should never use so it can be automated with mozmill. Ideally, an automation framework that can interact with UI would be better especially since that also tests the UI (you and I spoke of visual test which I used in the 90's for this purpose and I am sure there are newer ones though I don't have time to investigate and evaluate such items)
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → DUPLICATE
Duplicate of bug: 423754
You need to log in before you can comment on or make changes to this bug.