605343 - Tracking bug for Shared Module Refactor

Status: RESOLVED WONTFIX

(Mozilla QA Graveyard :: Mozmill Tests, defect)

Description

[Geo Mealer [:geo] -- This account is inactive after 2015-07-07]

This is the tracking bug for the Shared Module Refactor.  At current, it's expected to include the following phases:

I. Development of a basic Widget Element that encapsulates the controller.

II. Development of a tree of Widget children that specialize to control type (button, text box, etc.)

III. Development of a map/tree of Widget children to represent Firefox UI, including a binding "browser" object that represents the AUT.

IV. A set of standards, patterns, and classes used for new development of tests against the UI map.

V. Porting of existing code over to the UI map, including splitting apart "high-level" API functions that use the back end from the "low-level" UI map functionality.

These phases may change as new information is discovered.

Comment 1

[Geo Mealer [:geo] -- This account is inactive after 2015-07-07]

Clarifying,

Phase II is development of the hierarchy of Widget -classes-.

Phase III is development of a tree of Widget object instances, i.e. the actual data structure used by the tests.

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+1]]

I put some efforts today to create a basic set of functionality which match Geo's mentioned parts I - IV.

Check the repository on Github for example code:
http://github.com/whimboo/mozmill-api-refactor

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+1]]

As Geo pointed out yesterday we should incorporate cohesion when doing the refactoring. See http://en.wikipedia.org/wiki/Cohesion_%28computer_science%29

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Move of Mozmill Test related project bugs to newly created components. You can
filter out those emails by using "Mozmill-Tests-to-MozillaQA" as criteria.

Comment 5

[Geo Mealer [:geo] -- This account is inactive after 2015-07-07]

After some false starts, I've done some further exploration on this. My current thoughts are that I want to go with something roughly patterned after Selenium Page Object Model (which I'm currently thinking of as "Region Object Model").

This would include one exported object per "region" (e.g. toolbar, or other well-defined visual portion) that is responsible for encapsulating widget location information and any common high-level functions for that region (e.g. navigating to a web page by typing into the URL bar and hitting enter).

These region objects would be contain instances of the widget classes, as mentioned above. The classes would provide basic constructors for taking location info and feeding it to nodeCollector or Elem constructors for binding, as well as a library of methods appropriate to the widget in question.

There are some open questions, for which I need some feedback from the A-team.

1) My current intention is to take a controller in the constructor for a Region, which will in turn be fed to the widgets. The controller parameter for most regions would likely have a default value of mozmill.getBrowserController() to keep most of the controller stuff out of the tests in the normal case. However, it could be overridden with, say, the result of newBrowserController if appropriate. 

Of course, a region that correponds with another controller method (say, one in Thunderbird) would supply a different default.

There seems to be some concern that controllers may go away in future versions of Mozmill. Do we expect that to be the case? If so, how would we select between different open browser windows?

2) As a side thing, I have started a series of mozmill-test-side asserts that wrap the basic Assert() function to provide a richer and simpler test language without quite so many custom closures/evals. Ultimately, I'd actually like to move to the Selenium verify/assert/waitFor triad, which would require Mozmill to provide a non-fatal way to add a pass/fail to the test results.

Right now, though, my primary interest is separating Asserts from generic Errors, so I can better differentiate a test failure from a scripting issue. My expectation is to eventually put in a top-level error handler that knows whether something is an AssertError (or whatever our class is on that) and treats it somewhat differently.

Can I assume that there will always be -some- way that we can uniquely say "this is a test result"? That's all I need to know right now to establish wrappers.

3) What's the ultimate intention of this code re: moving it to Mozmill? My current assumption is that we might take the widget classes and possibly the assert library and move those to where the other Mozmill .js files live, and that Region Object Model might be communicated as a best-practice type thing. If I come up with other base classes to serve the map, those might also be candidates for handoff.

However, right now I'm definitely proceeding from the standpoint of wrapping current Mozmill technologies (controller, Assert, etc.) rather than redefining them or attempting to write to their implementations. That provides the loosest coupling from where I sit; the wrappers can be changed in the future if the tech changes.

Are there any problems with that approach that I'm missing?

Thanks for any feedback.

Comment 6

[cmtalbert]

(In reply to comment #5)

> This would include one exported object per "region" (e.g. toolbar, or other
> well-defined visual portion) that is responsible for encapsulating widget
> location information and any common high-level functions for that region (e.g.
> navigating to a web page by typing into the URL bar and hitting enter).
This sounds like a good way to organize the objects.
> 
> These region objects would be contain instances of the widget classes, as
> mentioned above. The classes would provide basic constructors for taking
> location info and feeding it to nodeCollector or Elem constructors for binding,
> as well as a library of methods appropriate to the widget in question.
> 
> There are some open questions, for which I need some feedback from the A-team.
> 
> 1) My current intention is to take a controller in the constructor for a
> Region, which will in turn be fed to the widgets. The controller parameter for
> most regions would likely have a default value of
> mozmill.getBrowserController() to keep most of the controller stuff out of the
> tests in the normal case. However, it could be overridden with, say, the result
> of newBrowserController if appropriate. 
> 
> Of course, a region that correponds with another controller method (say, one in
> Thunderbird) would supply a different default.
> 
This sounds very straightforward.  I'm not sure how else you'd do this without unnecessarily re-implementing a ton of window managment stuff which your region objects should not be doing.  In my mind they should be a very streamlined mechanism to access the underlying object as possible.  So, I like your idea to default to the "normal" controller, and provide a means to override that from the test if need be.

How are you planning to get the controller?  If you do a Mozmill.getBrowserController() each time you want a controller, then you will get the controller for the most recent window, which may or may not be the window you expect that you are getting.  Do your object classes have a means to cache the "original" controller from the main window?  I think this "original" controller is the controller you want to default to, most of the time.

It seems to me that we could change Mozmill so that it could help you here if we thought about some way to have it help manage open windows and cache controller objects for you.  That might be more reliable than always stupidly returning the "most recent window" when you ask for a controller.

But anyhow, the answer is yes, this is a fine way to move forward, there are some edge cases, but nothing we can't handle either by having the region objects cache the "default" controller or having mozmill be smarter about managing windows.

> There seems to be some concern that controllers may go away in future versions
> of Mozmill. Do we expect that to be the case? If so, how would we select
> between different open browser windows?
> 
No, the controller APIs are guaranteed to change, and many of them will vanish in 2.0.  There will always be some object that represents a window.  Due to the way Javascript scoping works, this is a hard requirement.  So there will always be a controller object.  What APIs are hung off that controller object are subject to change, but the window management stuff is not one of them.  The APIs I want to change are the action/assert APIs that you start talking about later.

> 2) As a side thing, I have started a series of mozmill-test-side asserts that
> wrap the basic Assert() function to provide a richer and simpler test language
> without quite so many custom closures/evals. Ultimately, I'd actually like to
> move to the Selenium verify/assert/waitFor triad, which would require Mozmill
> to provide a non-fatal way to add a pass/fail to the test results.
> 
Excellent.  Yes, I'd like Mozmill to be able to do this.  Hopefully we will have that mechanism once I've shored some reliable sanity into the Python <--> JS communication of mozmill.  Currently mozmill doesn't have this ability because it can only communicate a few very simple things across that bridge.  Improving that communication should be able to give us this ability.

> Right now, though, my primary interest is separating Asserts from generic
> Errors, so I can better differentiate a test failure from a scripting issue. 
My
> expectation is to eventually put in a top-level error handler that knows
> whether something is an AssertError (or whatever our class is on that) and
> treats it somewhat differently.
I think that is a genericly useful item that could be upstreamed in Mozmill. So, I will likely take your code if I can.
> 
> Can I assume that there will always be -some- way that we can uniquely say
> "this is a test result"? That's all I need to know right now to establish
> wrappers.
> 
Yes, there has to be. 

> 3) What's the ultimate intention of this code re: moving it to Mozmill? My
> current assumption is that we might take the widget classes and possibly the
> assert library and move those to where the other Mozmill .js files live, and
> that Region Object Model might be communicated as a best-practice type thing.
> If I come up with other base classes to serve the map, those might also be
> candidates for handoff.
That's exactly the way I am thinking of it.  My vision is that the items that are genericly useful across products should be upstreamed into Mozmill.  In the context here, that would be the assert classes and the widget classes.  The firefox specific implementation of the Region Object Model would continue to live in your tests domain as that is product specific.  But we would certainly hold that up as a "here's how to get the most out of mozmill" best practice. (No pressure :p )

> 
> However, right now I'm definitely proceeding from the standpoint of wrapping
> current Mozmill technologies (controller, Assert, etc.) rather than redefining
> them or attempting to write to their implementations. That provides the loosest
> coupling from where I sit; the wrappers can be changed in the future if the
> tech changes.
Please please do not write anything to their implementations.  We need to strive for loose coupling everywhere we can.  Write your scripts to what those interfaces *do*, not *how they do it*.  The "how they do it" part can and likely will change.  But there are a bunch of fundamentals in mozmill that will never change, like: there will always be windows, there will always be elements to click on, etc. If you need an interface to do something different, if you need a new interface, then let me know.  
> 
> Are there any problems with that approach that I'm missing?
> 
No, I think you're on the right track.  I'm excited to see what you come up with.  I think that the Regional Object Model is going to be a very useful feature of our Mozmill tests going forward and I'm looking forward to seeing it develop.

Let me know if you have any other questions.

Comment 7

[Geo Mealer [:geo] -- This account is inactive after 2015-07-07]

(In reply to comment #6)

> How are you planning to get the controller?  If you do a
> Mozmill.getBrowserController() each time you want a controller, then you will
> get the controller for the most recent window, which may or may not be the
> window you expect that you are getting.  Do your object classes have a means to
> cache the "original" controller from the main window?  I think this "original"
> controller is the controller you want to default to, most of the time.

Good point!

I think it'd be something along the lines of:

/**
 * get() is just for the final syntax, see example below
 */
function get(aController) {
  return new NavBar(aController);
}

function NavBar(aController) {
  _controller = aController || mozmill.getBrowserController();
}

IOW, it'd be whatever's current at the time the object is instantiated. Create a new window, instantiate a new region object:

navbar = require(MODULES + "navbar");
myNavbar = navbar.get();

...do something that opens a new window...

// get a new region object based on the new window.
myNewNavbar = navbar.get();

// or even do both at once...
myNewestNavbar = navbar.get(mozmill.newBrowserController());

That's all pretty seat of the pants, though. Final implementation might (will) change, but the point is that region objects should also be responsible for caching their controllers. I think it'll work ok.

> It seems to me that we could change Mozmill so that it could help you here if
> we thought about some way to have it help manage open windows and cache
> controller objects for you.  That might be more reliable than always stupidly
> returning the "most recent window" when you ask for a controller.

...but if I'm wrong, that's awesome. I appreciate knowing that.

> to click on, etc. If you need an interface to do something different, if you
> need a new interface, then let me know.  

Definitely, I'll keep you in the loop of course.

> No, I think you're on the right track.  I'm excited to see what you come up
> with.  I think that the Regional Object Model is going to be a very useful
> feature of our Mozmill tests going forward and I'm looking forward to seeing it
> develop.
> 
> Let me know if you have any other questions.

Thanks tons!

Comment 8

[Henrik Skupin [:whimboo][⌚️UTC+1]]

(In reply to comment #6)
> This sounds very straightforward.  I'm not sure how else you'd do this without
> unnecessarily re-implementing a ton of window managment stuff which your region
> objects should not be doing.  In my mind they should be a very streamlined
> mechanism to access the underlying object as possible.  So, I like your idea to
> default to the "normal" controller, and provide a means to override that from
> the test if need be.

Having those region objects, what will they need to be operable? First thought is the window they are contained in. And with window we have to take care, especially for webpages which contain frames. It will not be the controller.window property. That's why the logic to choose a defaultView should be IMO outside of the region objects and fold in as a parameter. It also would apply to the map we have to build up for region objects. Please take care that those elements will need a parent or at least the document, otherwise it's impossible for us to retrieve elements from within a list or an XBL binding.

That's something I had implemented in the first stage of the prototype for the browser object. It's basic and by far complete, but shows how it could work:

https://github.com/whimboo/mozmill-api-refactor/blob/master/modules/ui/browser.js

> It seems to me that we could change Mozmill so that it could help you here if
> we thought about some way to have it help manage open windows and cache
> controller objects for you.  That might be more reliable than always stupidly
> returning the "most recent window" when you ask for a controller.

In nearly all of our cases we only use one single browser window. Other windows are handled with the handleWindow function in the utils.js module. So a caching is indeed helpful, and was already part of the code mentioned above. Does it fit what you are thinking of Geo?

> No, the controller APIs are guaranteed to change, and many of them will vanish
> in 2.0.  There will always be some object that represents a window.  Due to the
> way Javascript scoping works, this is a hard requirement.  So there will always
> be a controller object.  What APIs are hung off that controller object are
> subject to change, but the window management stuff is not one of them.  The
> APIs I want to change are the action/assert APIs that you start talking about
> later.

Which would make it like the i.e. FirefoxDriver in Selenium? Sounds fine.

> > 2) As a side thing, I have started a series of mozmill-test-side asserts that
> > wrap the basic Assert() function to provide a richer and simpler test language
> > without quite so many custom closures/evals. Ultimately, I'd actually like to
> > move to the Selenium verify/assert/waitFor triad, which would require Mozmill
> > to provide a non-fatal way to add a pass/fail to the test results.
> > 
> Excellent.  Yes, I'd like Mozmill to be able to do this.  Hopefully we will
> have that mechanism once I've shored some reliable sanity into the Python <-->
> JS communication of mozmill.  Currently mozmill doesn't have this ability

Do we really need the JS <-> Python communication solved? Mozmill is already able to fire test failures without raising exceptions. Best example is the jum module which calls "frame.events.fail". That adds a failure to the test results but continues the test.

> > expectation is to eventually put in a top-level error handler that knows
> > whether something is an AssertError (or whatever our class is on that) and
> > treats it somewhat differently.
> I think that is a genericly useful item that could be upstreamed in Mozmill.
> So, I will likely take your code if I can.

Geo, do you have made any thoughts on how we want to differentiate those asserts? Will this be part of our goals for this refactoring?

Comment 9

[Dave Hunt [:davehunt] [he/him] ⌚BST]

(In reply to comment #8)
> > No, the controller APIs are guaranteed to change, and many of them will vanish
> > in 2.0.  There will always be some object that represents a window.  Due to the
> > way Javascript scoping works, this is a hard requirement.  So there will always
> > be a controller object.  What APIs are hung off that controller object are
> > subject to change, but the window management stuff is not one of them.  The
> > APIs I want to change are the action/assert APIs that you start talking about
> > later.
> 
> Which would make it like the i.e. FirefoxDriver in Selenium? Sounds fine.

I'm not sure this is true. Selenium does not contain any assertion methods*, instead this is deferred to the test framework in use. Similarly, there is only a waitForCondition, which can then be wrapped for custom conditions such as waitForElementPresent.

The exception to this is Selenium IDE, which does contain assert/verify/waitFor methods. Also, Selenium 2 has implemented a flexible Wait class for waits, rather than the waitForCondition.

Comment 10

[Dave Hunt [:davehunt] [he/him] ⌚BST]

(In reply to comment #5)
> This would include one exported object per "region" (e.g. toolbar, or other
> well-defined visual portion) that is responsible for encapsulating widget
> location information and any common high-level functions for that region (e.g.
> navigating to a web page by typing into the URL bar and hitting enter).

Sounds like a great idea!

Comment 11

[Geo Mealer [:geo] -- This account is inactive after 2015-07-07]

Going ahead and assigning to me, since I'm project lead.

Comment 12

[Henrik Skupin [:whimboo][⌚️UTC+1]]

This project is dead so the bug is no longer necessary.