Closed Bug 991671 Opened 6 years ago Closed 2 years ago

Provide a simple way to run the python integration tests (aka UI Tests)

Categories

(Firefox OS Graveyard :: Gaia::UI Tests, defect)

x86_64
Linux
defect
Not set

Tracking

(Not tracked)

RESOLVED WONTFIX

People

(Reporter: julienw, Unassigned)

Details

Attachments

(1 file)

It's quite challenging to run the UI Tests for someone that doesn't know them well.

The goal of this bug is to take the MDN documentation [1] and turn part of it to a make goal. The make goal should be able to handle a APP= environment variable to run only the tests for one app.

[1] https://developer.mozilla.org/en-US/Firefox_OS/Platform/Automated_testing/gaia-ui-tests/Gaia_UI_Tests_Run_Tests
Is this a duplicate of bug 987746?
Here I was hoping for a make target in Gaia's Makefile. If I understand correctly bug 987746 would be for the B2G repository, right?
I think bug 987746 would be for mozilla-central, and should be capable of running with locally built or otherwise procured b2g/gaia builds. :ahal can correct me if I'm wrong, I'm not as familiar with mach as I should be.
Flags: needinfo?(ahalberstadt)
Here I'm really looking for a make target because that's what we have for other tests, so the rule of least surprise would hold :)
(In reply to Julien Wajsberg [:julienw] from comment #4)
> Here I'm really looking for a make target because that's what we have for
> other tests, so the rule of least surprise would hold :)

Other gaia tests, but bug 987746 is about aligning with other mozilla tests, which may be even less surprising to contributors. I'll let :ahal respond though, as I'm already out of my depth here. :)
both works :) I bet mach could even use gaia's make ;)
(In reply to Julien Wajsberg [:julienw] from comment #6)
> both works :) I bet mach could even use gaia's make ;)

win-win :D
(In reply to Dave Hunt (:davehunt) from comment #3)
> I think bug 987746 would be for mozilla-central, and should be capable of
> running with locally built or otherwise procured b2g/gaia builds. :ahal can
> correct me if I'm wrong, I'm not as familiar with mach as I should be.

Yes, that's right. The patch in there should work if you have a local B2G build or a local b2g desktop build (though for the latter you need to specify a gaia path).

If you want to create a make target and have the mach script call that, that would work too. Personally I'd rather see a mach-bootstrap script checked into gaia so we can just run them directly with mach and avoid two separate execution paths. I would love it in general if execution paths for running gaia tests converged with the rest of gecko :).
AFAIK mach in Gecko is a way to make running the various tools consistent. I totally see the same goal in Gaia, but that would not be my work as I don't know how to write Python :)
Yeah, it would probably be on me :). I'd be totally fine with a make target and the mach target calling that for now.
Flags: needinfo?(ahalberstadt)
I see that there is a name inconsistency; for example, the Messages app's directory is "sms" (and therefore we use APP=sms to run unit tests for this app) whereas it's "messages" in the UI Test's directory.

Zac, how much work would it be to match the UI Test's directory's names with the app directories?
Flags: needinfo?(zcampbell)
I can't immediately think of a problem. We named the directories after the manifest's name which has a nice correlation with the app in the UI context.
Flags: needinfo?(zcampbell)
Sorry, trying to understand properly :) do you mean you'd rather keep the tree structure as it is now, or that there should be no issue to change it? :)
Flags: needinfo?(zcampbell)
Yes I'd rather keep it as it is now. This doesn't really seem like a P1 bug. Changing the directory is just a risk.
Flags: needinfo?(zcampbell)
ok I'm glad I asked ;)

I think I'll need to have a mapping file then, I'll think about it.
The tests are quite flexible and can run on desktopb2g, several devices and emulator so it's hard to avoid some complexity unfortunately.

We filed to make the address default:
https://bugzilla.mozilla.org/show_bug.cgi?id=1005004
We also made a change to give default timeouts depending upon the device. These two remove two command lines required.

Also wlach is filing to run it with mach, I'm concerned there'll be 4 or more different ways to run the test and actually it will be more complicated:
https://bugzilla.mozilla.org/show_bug.cgi?id=987746
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
To me, mach could simply use anything we'd do here...

The mach solution will be in the B2G repository while here I wanted a purely gaia solution.
Summary: Provide a simple way to run the UI Tests → Provide a simple way to run the python integration tests (aka UI Tests)
I want to have a goal in Gaia.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
QA Whiteboard: [fxosqa-auto-backlog-]
We discussed to have this kind of feature during the past few days. I think I can provide a new make rule here.
Flags: needinfo?(jlorenzo)
Comment on attachment 8565400 [details] [review]
[gaia] JohanLorenzo:bug-991671 > mozilla-b2g:master

Is that the kind of Make rule you had in mind, Julien?
Flags: needinfo?(jlorenzo)
Attachment #8565400 - Flags: feedback?(felash)
Comment on attachment 8565400 [details] [review]
[gaia] JohanLorenzo:bug-991671 > mozilla-b2g:master

I didn't really know about the existing script. If the script works fine I don't really need a dedicated Makefile rule, given the MDN instructions are uptodate.

I don't mind if we have one though.
Attachment #8565400 - Flags: feedback?(felash) → feedback+
Firefox OS is not being worked on
Status: REOPENED → RESOLVED
Closed: 6 years ago2 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.