987046 - [Gaia-ui-tests] Discussion of Gaia-ui-tests/marionette restruct

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: Gaia::UI Tests, defect, P3)

Description

[Walter Chen[:ypwalter][:wachen]]

This bug provides a place for further discussion
-------------------------------------------------------------------------
Here are some thoughts on refactoring:

The reason of refactoring is to improve the ease of joining this testing party from outside of us, eg. community or any newbie. People can see and know how to take out or add different functions, and know how to integrate things better. In addition, I think MTBF can benefit from this refactoring. That is to say, we can reduce maintenance efforts on MTBF framework. Perhaps, we can contribute some other stuffs back to gaia-ui-tests. I hope that one day MTBF won't be a lonely star in the galaxy but to merge into gaia-ui-tests completely.

Thereby, I think it's time again to talk about this thing - the refactoring of gaia-ui-test.

There are many things that can be factored into different python files / functions.

For class/file:
I mean, for example, Endurance test class or Mtbf test class should be another file like EnduranceTestCases.py , and gaia_test.py should import it
Or, things that are not highly relevant to gaia_test.py or any files itself should be split like /app/base.py or /app/(app name)/...py
Things like YoctoDevice, PowerData, and so on...should apply the same rules.

For functions:
We should try to separate out code a little bit more so that people first time use it can understand from its name, and other test framework should be able to take things wanted or take out things unwanted. I know this concept is vague, but let me try to explain a little bit more here:

Take gaia_test.py for instance, we can refactoring it like:

def setUp():
    marionetteSetUp()
    yoctoSetup()
    deviceCheck()
    timeOutCheck()
    generalAppSetup()
    gaiaCleanUp()
(PS. This is just an example. It doesn't need to be exactly like this)


It's just my thoughts in mind. However, I do want to know how you think and how we can finally get into a better framework altogether. If anything you want to know, I think having a meeting in addition to email communication could be good and clarifying more things.


----------------------------------------------------------------------------

When I was trying to do other framework based on gaia-ui-tests...
For example, the MTBF framework...It goes with gaia-ui-tests/marionette.

However, due to the bad design of abstract functions of gaia-ui-tests/marionette.
It is going to change this line or that line of codes every time.
It should all be wrapped in functions like:
> def setUp():
>      marionetteSetUp()
>      yoctoSetup()
>      deviceCheck()
>      timeOutCheck()
>      generalAppSetup()
>      gaiaCleanUp()

Then when other frameworks are trying to do a new test framework wrapper, it can be modify easily and not affecting by changes from branch to branch.
Also, things like yoctoSetup should be moved to another class, I don't see the reason to put it with gaia_test.py to mess up with other stuffs.

Besides, we should consider not using restart between tests graduately.
Also, I think maybe concept of  behavior driven or data driven tests can be applied to gaia-ui-tests for further benefits.

Comment 1

[Malini Das [:mdas] - Away, not checking bugmail]

I agree with everything above, I just wanted to point out:

(In reply to Walter Chen[:ypwalter][:wachen] from comment #0)
> I mean, for example, Endurance test class or Mtbf test class should be
> another file like EnduranceTestCases.py , and gaia_test.py should import it

We have changed this in later versions of gaiatest/marionette-client: https://github.com/mozilla-b2g/gaia/blob/master/tests/python/gaia-ui-tests/gaiatest/gaia_test.py#L9 which uses the mixin from https://mxr.mozilla.org/mozilla-central/source/testing/marionette/client/marionette/runner/mixins/endurance.py#46

Comment 2

[Zac C (:zac)]

I've just had a look at it and I think it's well abstracted already actually.

Comment 3

[BMO Automation]

Firefox OS is not being worked on