958126 - Tests which install extensions can fail because the installation can be drastically delayed

Status: RESOLVED FIXED

(Mozilla QA Graveyard :: Mozmill Tests, defect, P2)

Description

[Henrik Skupin [:whimboo][⌚️UTC+2] (away 10/03 - 10/13)]

As what I have seen today is that installing one of our locally hosted extensions can be delayed until our test is marked as failed. If we shutdown the application in such a case, we run into an assertion because the install dialog pops-up when the main window has already been closed.

The problem here is that a request to a remote site as given via extensions.getAddons.get.url happens. We could also change this URL, but I wonder if we could completely disable any remote activity if we install add-ons via localhost.

Dave or Blair, is there a setting for that?

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2] (away 10/03 - 10/13)]

Actually this is the case if the machine you are running the test on, is not connected to the network.

Comment 2

[Dave Townsend [:mossop]]

I think this setting should do the trick https://blog.mozilla.org/addons/how-to-opt-out-of-add-on-metadata-updates/

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+2] (away 10/03 - 10/13)]

Thanks Dave. We will check if that works for us. Andreea and Otilia, could someone from your team could help here? This bug could be the reason why some of our add-on tests are failing with disconnects. We haven't seen it that much lately but I want to make sure that our tests are getting more stable. Thanks.

Comment 4

[Andreea Matei [:AndreeaMatei]]

Cosmin is looking into this.

Comment 5

[Cosmin Malutan, [:cosmin-malutan]]

Hi Henrik I tried to reproduce this in the morning but I couldn't. I followed the steps from bug 958133.
Is this intermittent or is a constant failure for you, because if is constant I might be missing something?

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+2] (away 10/03 - 10/13)]

It happens all the time. Best is most likely when you connect yourself to a router which isn't connected to the internet.

Comment 7

[Cosmin Malutan, [:cosmin-malutan]]

Thanks, I had ran testruns/simple tests whithout internet connection or from VM connected to an internal network without internet connection but I couldn't reproduce it.
We don't have a spare router to test this and because it would be a lot bureaucracy to request a router from IT I will try to reproduce on this at home.

Comment 8

[Cosmin Malutan, [:cosmin-malutan]]

Attachment: patch v0.1.patch

I tested this while connected to a router without internet connection and indeed the add-on installation will hang.
Whit the preference  Dave Townsend suggested it passes.

I guess we have to set the preference everywhere we install ad-dons in functional tests.

Comment 9

[Henrik Skupin [:whimboo][⌚️UTC+2] (away 10/03 - 10/13)]

That's good to hear, Cosmin. Looks like we should do that in a more general basis.

Dave, how should test frameworks handle that preference as best? Shall we completely disable it to not interfere with any e.g. telemetry done with the information? Or wouldn't it hurt the AMO data to leave it enabled for tests?

Comment 10

[Dave Townsend [:mossop]]

It should probably be disabled, it already is for most test harnesses: http://mxr.mozilla.org/mozilla-central/source/testing/profiles/prefs_general.js#57

Comment 11

[Andreea Matei [:AndreeaMatei]]

Comment on attachment 8359360 [details] [diff] [review]
patch v0.1.patch

Review of attachment 8359360 [details] [diff] [review]:
-----------------------------------------------------------------

Since we'll have this change overall, please file a bug in Mozbase for it.

Comment 12

[Cosmin Malutan, [:cosmin-malutan]]

Has been fixed in bug 960976.

Comment 13

[Henrik Skupin [:whimboo][⌚️UTC+2] (away 10/03 - 10/13)]

I really don't understand why this should be a dupe! This is clearly on the dependency list. So as you said it has been fixed by bug 960976, which is in another product!