1384428 - Potential stale cache when using AddonTestUtils.createTempWebExtensionFile

Status: NEW

(Toolkit :: Add-ons Manager, enhancement, P3)

Description

[Osmose [:osmose, :mkelly]]

Attachment: test.js

I ran into this while working on a Shield feature involving add-ons. AddonTestUtils.createTempWebExtension file ends up creating the test XPI in the system temporary directory with the named generated-extension.xpi. It ensures a unique name for this file using nsIFile.createUnique.

However, createUnique only checks for existing filename conflicts; it is possible that, between two different tests both using the function, that the filename will be reused if the add-on file is cleaned up. 

What happened was that two of the tests in Shield created test extensions with the same filename, and when we tried to install the second one, the Add-on Manager used the cached contents from the first test extension, regardless of what was passed into createTempWebExtension. I added a workaround to get our tests working (the tests are still in a PR so there's no easy way to replicate until we do our next sync, sadly): https://github.com/Osmose/normandy/blob/5f048a2a06a2f0d0ce516970802ba035eb24b48f/recipe-client-addon/test/browser/head.js#L59

I tried to write two tests that replicated the issue, but they keep stalling due to an error:

Extension error: Error: Cannot alter the setting for url_overrides:newTabURL as it does not exist. resource://gre/modules/ExtensionSettingsStore.jsm 138

I've attached the test file I was using to try and replicate the issue, in case it's helpful