Automate JS tests for any Android phone or emulator



8 years ago
5 years ago


(Reporter: sayrer, Assigned: cmtalbert)



Firefox Tracking Flags

(blocking2.0 .x+)


(Whiteboard: [softblocker])


(4 attachments)

Comment hidden (empty)

Comment 1

8 years ago
Created attachment 480135 [details]
python script to start and stop emulator

this will find the emulator, start it, wait for it to boot, and unlock the homescreen. Then it checks that the required apks for python are installed.

Next step is to make the emulator step optional and switch to an OptionsParser. That way, the tests can run on an attached device in automated fashion as well. I have tried this by commenting out the emulator stuff.

Now, one crucial broken step here is that the sl4a Android scripting environment is currently broken when running commandline intents. From perusing the stack traces in logcat, I think it will be easy to fix, but will require diving into Eclipse.


8 years ago
OS: Linux → Windows CE


8 years ago
OS: Windows CE → Android
Hardware: x86_64 → ARM
We absolutely must stop finding brokenness in the field that our JS tests would have caught.  This needs to block fennec-beta3.
blocking2.0: --- → ?

Comment 3

8 years ago
I looked at moving the js shell tests over week before last.  I'll add it to my list and see if I can make these remoteable to work with the system we've developed for Android testing.  I don't think it should be too much work here, the JS tests are pretty straight forward.

I'll report back with an ETA once I look at the code in more detail.


8 years ago
blocking2.0: ? → final+

Comment 5

8 years ago
Who should own this? It's currently marked as a blocker, and doesn't sound straightforward.

Comment 6

8 years ago
(In reply to comment #5)
> Who should own this? It's currently marked as a blocker, and doesn't sound
> straightforward.

I'm not sure I'm the best person for it, but I am trying to remote these tests so that we can run them the same way that we run everything else on android via buildbot.  

The first hurdle we had to overcome was getting environment variables to work reliably in the remoting system, which we were able to do in november.

Currently, I'm stuck trying to get a debug build of android to compile, it's breaking in breakpad, and I'm working with ted on that issue.  With an opt build I could not get the js trace test to run at all using the method proposed on the wiki page in comment 4 on a tegra.

Note, that I can get them to run using that method on my Droid.  But, I still can't get them to work on tegra and I was hoping by getting a debug build to function I'd get further along that path.  

The actual remoting code is pretty straightforward and simple, once I can get the js shell to run reliably on the tegra device.  

I'll take this for now as I am working on it, but I do apologize for not realizing this had been made a blocker.  I'll see if I can make some progress here this week.
Assignee: nobody → ctalbert


8 years ago
Whiteboard: [softblocker]

Comment 7

8 years ago
Created attachment 505595 [details] [diff] [review]

So, I've been banging my head against getting this automated so that we can run this on tegras via buildbot and by developers on their phones.  I've all but given up on getting it to run on the tegras.  As my last ditch effort on tegras, I'm trying to build a debug android os to see if I can determine why the js shell refuses to start on that platform.

Since the js shell does work fine on any other phone (galaxy, nexus, droid etc) I'm moving ahead with the necessary refactoring to enable these tests to run via the sutagent & buildbot as well as for developers.  When this refactoring is complete, there will be no functional changes to (it will work the same as it always has) and there will be a that will handle running jit-tests against a device running the SUTagent.

I'll attach a series of WIP patches here to start getting some feedback. Apply them in order:
1. jit-test-whitespace-refactor.diff (whitespace and class changes only)
2. jit-options-refactor.diff
3. jit-test-remote.diff

Comment 8

8 years ago
Created attachment 505596 [details] [diff] [review]

Comment 9

8 years ago
Created attachment 505597 [details] [diff] [review]
jit-test-remote WIP

This isn't finished (obviously).  It's attached so that you can see why we refactor into classes for remoted tests and how those classes get used.

Comment 10

8 years ago
I am a huge fan of automated tests but would we hold the release for this?

This bug is one of 7 automatically changed from blocking2.0:final+ to blocking2.0:.x during the endgame of Firefox 4 for the following reasons:

 - it was marked as a soft blocking issue without a requirement for beta coverage
blocking2.0: final+ → .x+
Hey, we finally fixed this. bug 858621 covers getting the jit-tests running from the test package on tbpl (they're running on Cedar currently).
Last Resolved: 5 years ago
Resolution: --- → DUPLICATE
Duplicate of bug: 858622
You need to log in before you can comment on or make changes to this bug.