831686 - Create Mozmill test for proxy.pac file served via a file:// protocol

Status: RESOLVED INCOMPLETE

(Testing :: Firefox UI Tests, defect)

Description

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Over on bug 828202 we had a regression which caused us to fail loading a proxy.pac file if it has been specified via the file:// protocol. Steps to reproduce are as given in bug 828202 comment 25:

> 1. In a Win7 VM open the Control Panel and look for "configure proxy
> settings"
> 2. Click on LAN settings
> 3. Select "Use automatic configuration script"
> 4. Specify the file location such as
> file://User/mozilla/Desktop/proxy-sample.pac (see attached pac file I used)
> 5. Click OK and fire up Firefox and try browsing

One question though to Patrick, do we have to set this setting via the system or can it also be done via Firefox's proxy settings?

Attachment 703116 [details] contains a proxy.pac file with a proxy in the Ukraine. We will have to re-evaluate that if we can find something better.

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

We should land this test across all branches given that we do not want to regress this again in the future.

Comment 2

[Patrick McManus [:mcmanus]]

(In reply to Henrik Skupin (:whimboo) from comment #0)

> 
> One question though to Patrick, do we have to set this setting via the
> system or can it also be done via Firefox's proxy settings?
> 

this particular bug required system defined settings.

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Thanks. So we should figure out how to set those on various platforms. Most likely we should have APIs available.

Comment 4

[Andreea Matei [:AndreeaMatei]]

Closing as WontFix for now, until we can set those system settings.

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+2]]

The problem is not that we can't set the system settings but more about cleanup. If our testrun dies in the middle of the test, Mozmill will not be able to reset the changed system settings and will bust the users machine.

It might be something to automate with Marionette. Dave, what do you think?

Comment 6

[Dave Hunt [:davehunt] [he/him] ⌚BST]

I'm not sure I fully understand what's needed here, or how Marionette would be able to this where Mozmill can't.

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+2]]

You know Marionette better than me. But in Mozmill the tests are decoupled from the harness (Python) side, and changes made to the system can't be reverted if the test itself causes an abort, e.g. crash of the browser. In Marionette we could do that in the teardown method, right?

Also cc'ing Jgriffin for his input.

Comment 8

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Ah, I see what you mean now. Yes, we should be able to continue after a crash, although I'm not sure how well this works in practice for Marionette and Firefox desktop at the moment. For Mozmill, could we not handle this when we detect a disconnect error though?

Comment 9

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to Dave Hunt (:davehunt) from comment #8)
> Ah, I see what you mean now. Yes, we should be able to continue after a
> crash, although I'm not sure how well this works in practice for Marionette
> and Firefox desktop at the moment. For Mozmill, could we not handle this
> when we detect a disconnect error though?

No, because the framework doesn't know what the test did, and we are not that smart to run teardownModule of the failing test before continuing. :/ Lets get feedback from Jgriffin about Marionette.

Comment 10

[Jonathan Griffin (:jgriffin)]

So Marionette uses Python's unittest framework, and in this the tearDown method is always called at the end of a test, regardless of whether the test passed or failed, *unless* the test died during setUp.  This is an unfortunate gotcha which is built into unittest.

In Python 2.7, there is the concept of cleanups which are performed after a test (and after tearDown, if any), and these are always called even if setUp died.

None of this is Marionette-specific, it's just how Python's unittest works.

Comment 11

[u279076]

(In reply to Andreea Matei [:AndreeaMatei] from comment #4)
> Closing as WontFix for now, until we can set those system settings.

Can we please keep this bug in the NEW state until it's decided for sure it's a WONTFIX? Okay, Mozmill might not be the framework this test lands in but it seems like investigation is still ongoing.

Comment 12

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Given that marionette is bound to the in-tree tests at the moment we are not able to implement such a test, except finding a solution to run those tests outside of Buildbot. Not sure if that's worth our time at the moment.

I would suggest to wait until Mozmill itself is based on Marionette, which should make it possible for us to automate via Mozmill. But this will take a bit...

Comment 13

[Mihaela Velimiroviciu (:mihaelav)]

Since Mozmill is being dropped out, is this test still going to be developed in any way?

Comment 14

[Henrik Skupin [:whimboo][⌚️UTC+2]]

As several comments on this bug said we will need Marionette for it. Given that we are setting up the Firefox UI tests now lets move this bug over.

Patrick, when you check the bug as given in comment 0, we really have to specify a proxy pac file in the system settings? It won't match the test if we would simply add the pac file to the Firefox settings, right? If that is the case we might need a Python library which can modify system proxy settings.

Comment 15

[Patrick McManus [:mcmanus]]

comment 2

Comment 16

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Ups, sorry! Totally missed that. :S

So I'm not sure if we can do something like that, given that changing the systems proxy settings can have several impact to our machines in SCL3. We could kinda mess up our nodes. An option we might have is to temporarily switch to use auto config via WPAD, which is available for proxy.dmz.scl3.mozilla.com at http://proxy.dmz.scl3.mozilla.com/wpad.dat. This is also a pac file. Patrick, would that help? Or do we really need a manually set pac file? In the worst case we could use this wpad file, store it locally, and set it up in the system settings.

Comment 17

[Henrik Skupin [:whimboo][⌚️UTC+2]]

As long as a test like this is not really required we will drop the request.