474763 - Add automated tests for xpinstall

Status: RESOLVED FIXED

(Core Graveyard :: Installer: XPInstall Engine, defect)

Description

[Dave Townsend [:mossop]]

We currently have very little of xpinstall tested automatically. Just a few bits that have come up as part of extension manager tests. I have some work in progress to change that.

Comment 1

[Dave Townsend [:mossop]]

Attachment: test suite rev 1

This is a pretty well featured test suite for the xpinstall module. Sorry it's a bit big to start with.

The majority of the work is in harness.js which watches windows opening in order to accept/cancel a requested install, deal with any authentication and dismiss error dialogs. It also listens to the EM's install events to find out the status of installs as well as watching for installed blocked by the whitelist.

The testcases all follow a pretty similar pattern which hopefully will make reviewing and developing new cases easier. They register for callbacks from the harness and then trigger an install of some kind and test that the right things happen.

Some of the additional files:

authRedirect.sjs:   This is a server executed script to redirect to a url when the right authentication is given.
cookieRedirect.sjs: The same but redirects when the right cookie is present.

enabled.html:             Tests that InstallTrigger.enabled() works.
installtrigger.html:      Runs InstallTrigger.install.
installchrome.html:       Runs InstallTrigger.installChrome.
startsoftwareupdate.html: Runs InstallTrigger.startSoftwareUpdate.

unsigned.xpi:         A simple unsigned xpi extension.
signed.xpi:           A simple signed xpi extension.
signed2.xpi:          Another signed xpi extension.
signed-tampered.xpi:  A signed xpi that has had its contents changed since signing.
signed-untrusted.xpi: An xpi signed by an unknown signer.
corrupt.xpi:          This is a corrupt zip file, just a text file really.
empty.xpi:            An xpi with no install script.

The tests navigateaway and navigateaway2 will fail until bug 473060 is fixed. Also navigateaway2 is disabled completely for now as it triggers bug 475347.

Comment 2

[Dave Townsend [:mossop]]

Checking in these tests without these assertion fixes would make things very noisy

Comment 3

[Dave Townsend [:mossop]]

Pushed with browser_navigateaway.js and browser_navigateaway2.js disabled for now: http://hg.mozilla.org/mozilla-central/rev/e321d96f8145

Comment 4

[Dave Townsend [:mossop]]

Disabled the tests for now since they are failing on OSX:

TEST-UNEXPECTED-FAIL | chrome://mochikit/content/browser/xpinstall/tests/browser_signed_multiple.js | Timed out
TEST-UNEXPECTED-FAIL | chrome://mochikit/content/browser/xpinstall/tests/browser_signed_multiple.js | Should be 2 items listed in the confirmation dialog - Got 1, expected 2
TEST-UNEXPECTED-FAIL | chrome://mochikit/content/browser/xpinstall/tests/browser_signed_multiple.js | Should have seen the name from the trigger list - Got unsigned.xpi, expected Signed XPI
TEST-UNEXPECTED-FAIL | chrome://mochikit/content/browser/xpinstall/tests/browser_signed_multiple.js | Should have listed the correct url for the item - Got http://example.com/browser/xpinstall/tests/unsigned.xpi, expected http://example.com/browser/xpinstall/tests/signed.xpi
TEST-UNEXPECTED-FAIL | chrome://mochikit/content/browser/xpinstall/tests/browser_signed_multiple.js | Should have seen the signer - Got (Author not verified), expected (Mozilla Testing)
TEST-UNEXPECTED-FAIL | chrome://mochikit/content/browser/xpinstall/tests/browser_signed_multiple.js | Should have listed the item as signed - Got false, expected true
TEST-UNEXPECTED-FAIL | chrome://mochikit/content/browser/xpinstall/tests/browser_signed_tampered.js | Timed out
TEST-UNEXPECTED-FAIL | chrome://mochikit/content/browser/xpinstall/tests/browser_signed_tampered.js | Should have seen the name from the trigger list - Got unsigned.xpi, expected Tampered Signed XPI
TEST-UNEXPECTED-FAIL | chrome://mochikit/content/browser/xpinstall/tests/browser_signed_tampered.js | Should have listed the correct url for the item - Got http://example.com/browser/xpinstall/tests/unsigned.xpi, expected http://example.com/browser/xpinstall/tests/signed-tampered.xpi
TEST-UNEXPECTED-FAIL | chrome://mochikit/content/browser/xpinstall/tests/browser_signed_tampered.js | Should have seen the signer - Got (Author not verified), expected (Mozilla Testing)
TEST-UNEXPECTED-FAIL | chrome://mochikit/content/browser/xpinstall/tests/browser_signed_tampered.js | Should have listed the item as signed - Got false, expected true
TEST-UNEXPECTED-FAIL | chrome://mochikit/content/browser/xpinstall/tests/browser_signed_tampered.js | Install should fail - Got 0, expected -260
TEST-UNEXPECTED-FAIL | chrome://mochikit/content/browser/xpinstall/tests/browser_signed_trigger.js | Timed out
TEST-UNEXPECTED-FAIL | chrome://mochikit/content/browser/xpinstall/tests/browser_signed_untrusted.js | Timed out
TEST-UNEXPECTED-FAIL | chrome://mochikit/content/browser/xpinstall/tests/browser_signed_untrusted.js | Install should fail - Got 0, expected -260
TEST-UNEXPECTED-FAIL | chrome://mochikit/content/browser/xpinstall/tests/browser_signed_url.js | Timed out
TEST-UNEXPECTED-FAIL | chrome://mochikit/content/browser/xpinstall/tests/browser_softwareupdate.js | Timed out
TEST-UNEXPECTED-FAIL | chrome://mochikit/content/browser/xpinstall/tests/browser_unsigned_trigger.js | Timed out
TEST-UNEXPECTED-FAIL | chrome://mochikit/content/browser/xpinstall/tests/browser_unsigned_trigger.js | Shouldn't have been blocked by the whitelist
TEST-UNEXPECTED-FAIL | chrome://mochikit/content/browser/xpinstall/tests/browser_unsigned_trigger.js | Shouldn't have been blocked by the whitelist
TEST-UNEXPECTED-FAIL | chrome://mochikit/content/browser/xpinstall/tests/browser_unsigned_trigger.js | Shouldn't have been blocked by the whitelist
TEST-UNEXPECTED-FAIL | chrome://mochikit/content/browser/xpinstall/tests/browser_whitelist2.js | Timed out
TEST-UNEXPECTED-FAIL | chrome://mochikit/content/browser/xpinstall/tests/browser_whitelist5.js | Timed out

Comment 5

[Dave Townsend [:mossop]]

The tests are failing because Firefox stays in offline mode when switching online for some reason.

Comment 6

[Dave Townsend [:mossop]]

Attachment: fix offline switch

The tests are failing because randomly the switch to offline will fail. What happens is the call to set offline to true will spin an event loop, during that the add-ons manager opens, calls nonBrowserWindowDelayedStartup which sets offline to whatever the pref browser.offline is.

This fixes that by avoiding the browser.js method for setting offline and doing it itself, but setting the preference first to avoid the issue.

Comment 7

[Dave Townsend [:mossop]]

Landed: http://hg.mozilla.org/mozilla-central/rev/37fe6a2a0628

Comment 8

[Dave Townsend [:mossop]]

Landed these tests on the 1.9.1 branch:

http://hg.mozilla.org/releases/mozilla-1.9.1/rev/0887f54e7e72
http://hg.mozilla.org/releases/mozilla-1.9.1/rev/163dae9c46a8

Comment 9

[Dave Townsend [:mossop]]

Attachment: branch patch

This is a patch for the 1.9.0.x branch. It is essentially all the patches that landed on trunk however some tests are omitted (we don't have the infrastructure to test signed xpis on 1.9.0.x). There is also a couple of bits minor changes to add a quick and dirty JSON encoder and decoder since the tests pass stuff around and native JSON isn't on 1.9.0.x.

Comment 10

[Al Billings [:abillings - ex-MoCo]]

Verified by looking at the tests running for 1.9.0.11.