635425 - Request for an API to access test files and paged

Status: RESOLVED WONTFIX

(Mozilla QA Graveyard :: Mozmill Tests, enhancement)

Description

[u279076]

As discussed offline, we need a shared module to streamline loading and usage of test files in Mozmill tests.

One potential usecase, as pointed out by Henrik, would be to hot-switch between different servers in a test.

Everyone, please dump your ideas for a spec into this bug for implementation.  Thanks.

Comment 1

[Geo Mealer [:geo] -- This account is inactive after 2015-07-07]

I'm not a 100% sure what you're looking for, but would like to help.

Let me check some assumptions. Current way is:

BASE = collector.addHttpResource("../data");

...which in turn points the local http server at that dir and returns something like "http://localhost:8888/" right? So then you get:

PAGE = "foo.html";
openUrl(BASE + PAGE); // "http://localhost:8888/foo.html"

Am I right so far?

And when we want to go to an external server instead, you just use it like:

BASE = "http://my.server.com/"

So it sounds like all we need is a small wrapper class around setting BASE:

(pseudocoding for expendiency)

class HttpLoader:
  void useLocalServer(dirName); // add a default that auto-finds data dir?
  void useRemoteServer(serverName);
  string makeUrl(pageName);

http = new HttpLoader();
http.useLocalServer("../data"); // useLocalServer() if we do the default.
loadUrl(http.makeUrl("foo.html"));
http.useRemoteServer("my.server.com");
loadUrl(http.makeUrl("foo.html"));

We can have basic constants defined for standard servers, of course; doesn't have to be magic strings all over. 

The local data dir is a little trickier because it's a relative path, but that can be calculated somehow I'm sure. If a test can get the local pathname of its own file, then we can easily calculate from there, so worst case is we add that to Mozmill. That would be the default case of useLocalServe().

Does that get in the direction you were hoping for?

Comment 2

[Geo Mealer [:geo] -- This account is inactive after 2015-07-07]

If you really want hotswitching, better flow for above might be:

http = new HttpLoader();
http.useLocalServer();
http.page ="foo.html";

loadUrl(http.makeUrl());
http.useRemoteServer("my.server.com");
loadUrl(http.makeUrl());

IOW, we add a property for the page and then let you switch the server independently. Could make it an array or dict property too:

http.useLocalServer();
http.page['foo'] = "foo.html";
http.page['bar'] = "big/onerous/path/bar.html";

print http.makeUrl('bar'); // "http://localhost:8888/big/onerous/path/bar.html"

http.useRemoteServer('my.server.com');

print http.makeUrl('foo'); // "http://my.server.com/foo.html"

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Well, I wouldn't implement anything like that in our tests or shared modules. Both shouldn't be aware of the source of the content. Instead we should go ahead with my proposal from bug 685709. 

We can keep this bug open because all of our tests would have to be updated in anyway.

Comment 4

[u279076]

Any reason this should not be in Shared Modules component? It is a bug to develop a shared module, is it not?

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+2]]

We can't make the code any kind special for test files served by localhost. It also has to work for remote files. So all the logic has to happen in the automation TestRun class, which let us magically flip between servers via a command line option, e.g. --base-url. See the Selenium tests for AOM, which are doing the same for staging and production.

Comment 6

[u279076]

I agree with your comment 5 Henrik, but I'm still unclear as to why this belongs in the Mozmill Tests component.

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Because I'm not sure if it is really worth creating an API around. What should this API contain? We would always need a full path to the test file or page.

The only thing I could imagine is a wrapper so we could use test files as:

BEFORE:

const LOCAL_TEST_FOLDER = collector.addHttpResource('../../../data/');
var file = LOCAL_TEST_FOLDER + 'layout/mozilla_projects.html'
 
AFTER:

var file = getTestData('layout/mozilla_projects.html');

What do you think? If that's the right way to go please move back to shared modules. Thanks.

Comment 8

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to Geo Mealer [:geo] from comment #2)
> http.useLocalServer();
> http.page['foo'] = "foo.html";
> http.page['bar'] = "big/onerous/path/bar.html";

We can even use this snippet from Geo to come around the definition of all the constants.

Comment 9

[u279076]

I think this *should* be moved to Shared Modules. I think we are only hitting on one use case here and there are potentially others which we are not discussing right now.

I don't see a point in changing the way we load test files until Mozmill-2.0 lands.

Comment 10

[Henrik Skupin [:whimboo][⌚️UTC+2]]

It should not depend on the move to Mozmill 2 but the API refactoring project.

Comment 11

[Geo Mealer [:geo] -- This account is inactive after 2015-07-07]

Why dependent on the API refactor project? It's perfectly fine to add APIs before we get there. We'll just move them into the right place (if necessary) when we migrate over.

API refactor already has enough backpressure on it due to the overly long schedule. I hope we don't intend to hang any and all new work on it before tests can benefit.

Comment 12

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Well, we can handle it the same way as we did for the assertions module. So that would also work, but in the current API we aren't able to inject such a helper method/class into the global module scope. That means we would have to partly refactor it later again. If that's fine we could go ahead.

Anthony, please think about the needs you and your team have regarding such an API. Having a clear and well described set of requests would help us to get this implemented faster. Thanks.

Comment 13

[Geo Mealer [:geo] -- This account is inactive after 2015-07-07]

I'm OK with any refactoring we have to do later. That's part of the refactor project, keeping up with a moving target.

That said, if it's a technical reason I can understand the dependency better. Just don't want it to be an "any and all work halts" thing, but sounds like it isn't.

Agree re: needs. I obviously thought it was one thing at first and maybe it wasn't. The command line solution you mention in bug 685709 seems pretty perfect to me (would likely want to surface that in the config file too). So yeah, with some clarification can better give recommendations.

Comment 14

[Maniac Vlad Florin (:vladmaniac)]

I will look into this in the near future

Comment 15

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Mozmill tests are no longer in use. So closing this bug.