1111503 - [B2G] run gaia-ui-tests on emulator (locally)

Status: RESOLVED WORKSFORME

(Firefox OS Graveyard :: Emulator, defect)

Description

[Hsin-Yi Tsai (she/her) [:hsinyi]]

Considering the current emulator capability, it is good to take advantage of the emulator to provide better development experience to gaia developers so that they can write tests for telephony network features more easily. 

To achieve the goal, we need to make gaia-ui-tests run on emulator.

Comment 1

[Martijn Wargers (dead)]

What needs to be changed for gaia-ui-tests to make them run on emulator?

Comment 2

[Hsin-Yi Tsai (she/her) [:hsinyi]]

(In reply to Martijn Wargers [:mwargers] (QA) from comment #1)
> What needs to be changed for gaia-ui-tests to make them run on emulator?

I guess very likely no changes needed or very few, but I don't have a solid answer yet.
Gabriele, do you have more to share here? Thank you.

Comment 3

[Julien Wajsberg [:julienw]]

And marionette-js tests as well ;)

Comment 4

[Hsin-Yi Tsai (she/her) [:hsinyi]]

Great news: just saw a live demo. gaia-ui-tests work really good on emulator-kk, except the bad performance which is however tracked in another bug!

Waiting for further test results on marionette-js :)

Comment 5

[Hsin-Yi Tsai (she/her) [:hsinyi]]

(In reply to Hsin-Yi Tsai [:hsinyi] from comment #4)
> Great news: just saw a live demo. gaia-ui-tests work really good on
> emulator-kk, except the bad performance which is however tracked in another
> bug!
> 
> Waiting for further test results on marionette-js :)

Just talked to Al. There's still some work to do for marionette-js. We will evaluate the feature when marionette-js is done. File a bug tracking this issue.

Comment 6

[Martijn Wargers (dead)]

(In reply to Hsin-Yi Tsai [:hsinyi] from comment #5)
> (In reply to Hsin-Yi Tsai [:hsinyi] from comment #4)
> > Great news: just saw a live demo. gaia-ui-tests work really good on
> > emulator-kk, except the bad performance which is however tracked in another
> > bug!

So you mean this is working on emulator, right?
Could we get some documentation on how to run this on emulator, perhaps? https://developer.mozilla.org/en-US/Firefox_OS/Platform/Automated_testing/gaia-ui-tests/Gaia_UI_Tests_Run_Tests

Also, wouldn't it be good to get these gaia-ui tests running on b2g emulator on Treeherder?

Comment 7

[Martijn Wargers (dead)]

(In reply to Martijn Wargers [:mwargers] (QA) from comment #6)
> Also, wouldn't it be good to get these gaia-ui tests running on b2g emulator
> on Treeherder?

Chatted with jgriffin on irc, he said that it is simply that no-one spent the time to green up the tests on that platform.

Comment 8

[Hsin-Yi Tsai (she/her) [:hsinyi]]

(In reply to Martijn Wargers [:mwargers] (QA) from comment #6)
> (In reply to Hsin-Yi Tsai [:hsinyi] from comment #5)
> > (In reply to Hsin-Yi Tsai [:hsinyi] from comment #4)
> > > Great news: just saw a live demo. gaia-ui-tests work really good on
> > > emulator-kk, except the bad performance which is however tracked in another
> > > bug!
> 
> So you mean this is working on emulator, right?
> Could we get some documentation on how to run this on emulator, perhaps?
> https://developer.mozilla.org/en-US/Firefox_OS/Platform/Automated_testing/
> gaia-ui-tests/Gaia_UI_Tests_Run_Tests
> 

Hi Al, 
I believe the steps to run the tests on emulator are the same as those on real devices. Could you kindly confirm? Thank you!

> Also, wouldn't it be good to get these gaia-ui tests running on b2g emulator
> on Treeherder?

Agree, it would be good to enable those tests on treeherder eventually. And the issue is tracked in bug 987045. We need to find someone to help this but I have no idea who is available to help this at the moment.

Comment 9

[Al Tsai [:atsai]]

Yes. This is the same work as the steps on MDN.

Comment 10

[Eric Shepherd [:sheppy]]

(In reply to Al Tsai [:atsai] from comment #9)
> Yes. This is the same work as the steps on MDN.

Does this mean there's no work needed on MDN to cover this? If not, is anyone able to write down for me what needs to be added to MDN to cover how to do this testing in the emulator?

Comment 11

[Al Tsai [:atsai]]

I believe there should be no extra work for MDN. The steps are exactly the same as a physical device. However, I'd like to have some contents mentioned the steps work for emulators as well. I'll add a prototype for the MDN document and I'd like to have your review. :D

Comment 12

[Al Tsai [:atsai]]

https://developer.mozilla.org/en-US/Firefox_OS/Platform/Automated_testing/gaia-ui-tests/Gaia_UI_Tests_Run_Tests

Add some simple texts like "if you are using a emulator... " in Testing on Firefox OS Devices section. Please have someone review it for me and clean the "need review tag". Thanks in advance. :)

Comment 13

[Martijn Wargers (dead)]

Thanks that you're going to do this Al. Such simple info (run gaia-ui tests on emulator is just like on device) can make a huge difference, even though it might seem obvious.

Comment 14

[Eric Shepherd [:sheppy]]

Is anyone able to review the article's accuracy?

Comment 15

[Eric Shepherd [:sheppy]]

I have done a copy edit pass on the page and tidied things up a bit. Calling this document update complete; if it's not, please reset to dev-doc-needed with an explanation.