1037680 - Write some tests for MTP

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: MTP/UMS, defect)

Description

[Dave Hylands [:dhylands]]

I don't think that we can integrate any tests into the automated builds since there is no USB support in the emulator, and we don't run any automated tests with real devices.

However, I think that we can write some tests in the form of a script (or set of scripts), that we can use to manually verify that bits of functionality are working.

We can probably write the tests using bash, or perhaps python.

We should verify all of the basic functionality like:
- creating files
- deleting files
- renaming files
- enumerating files
- retrieving file attributes (verify file sizes etc).
- anything else we can think of.

We should endeavor to make sure that we can run these tests on all of the various hosts that we plan on supporting (i.e. windows, linux, mac).

Comment 1

[Dave Hylands [:dhylands]]

Or perhaps, we should implement these as mochitests or xpcshell tests that just don't get run today. Then down the road, when and if we ever get USB support, we can intgrate these easily.

I was playing with mach the other day and it looks pretty straight forward to kick off at least some form of individual tests with a real device (although we typically run the tests on the device and in this case we want to run the tests on the host).

I'll ni jgriffin to see what his thoughts are.

So Jonathon, we want to write some tests which run on the host and talk to a real device. I know we don't currently support this type of thing in our current automated builds, but I think that there are plans to add some of this as far as power management/monitoring tests are concerned. Can you provide any suggestions as to a direction to proceed in so that we can eas integrating things down the road?

Comment 2

[Jonathan Griffin (:jgriffin)]

We do already run some tests against real hardware, like gaia-ui-tests:  https://github.com/mozilla-b2g/gaia/tree/master/tests/python/gaia-ui-tests

We'll soon be adding node-based MarionetteJS tests to our on-device execution as well.

We only run these using Linux hosts and don't have any plans to run them using OSX or Windows hosts.

If you could write tests in one of those frameworks, you'd get them run (on Linux) for "free".

If that isn't practical, we could look at standing up a small run of mochitests or xpcshell tests on-device, although again you'd only get them run on Linux hosts.  xpcshell tests will likely be easier to maintain on-device long-term.  We'd prefer you use an existing framework, if possible, rather than something like a bash script, since a bash script won't know how to report test results in a way that's easy for our dashboards etc to consume.

Comment 3

[Dave Hylands [:dhylands]]

ok - I think using xpcshell makes the most sense then. Even having the tests only running from a linux host (in an automated fashion) would be good.

We can manually run the same tests on other platforms.

Comment 4

[howie [:howie]]

Hi Dave, would you help on the tests? Thank you :)

Comment 5

[Dave Hylands [:dhylands]]

Yeah - I should be help out with that.

Comment 6

[howie [:howie]]

Cool! I'll just change the owner to Dave then, thanks!

Comment 7

[Alphan Chen [:alchen]]

Hi Dave,
I made a document of test cases for MTP. (http://goo.gl/uygLIM)
We can check this test cases before we phase in any patch related to MTP.

Comment 8

[Dave Hylands [:dhylands]]

Not working on FxOS any more, so removing myself as assignee

Comment 9

[BMO Automation]

Firefox OS is not being worked on