The default bug view has changed. See this FAQ.

add make targets to run netwerk xpcshell tests on Android

RESOLVED FIXED in mozilla9

Status

()

Core
XPConnect
RESOLVED FIXED
6 years ago
6 years ago

People

(Reporter: gbrown, Assigned: gbrown)

Tracking

Trunk
mozilla9
All
Android
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [mobile-testing][xpcshell])

Attachments

(1 attachment, 4 obsolete attachments)

(Assignee)

Description

6 years ago
On desktop, make -C netwerk/tests/ xpcshell-tests runs a suite of tests via runxpcshelltests.py. A similar, convenient facility should be available for running xpcshell tests on Android devices.
(Assignee)

Updated

6 years ago
Depends on: 668349
(Assignee)

Updated

6 years ago
Assignee: nobody → gbrown
(Assignee)

Comment 1

6 years ago
Created attachment 548612 [details] [diff] [review]
patch to add make targets for remote xpcshell tests

I tested this with:

cd src/mozilla-central/objdir-droid
make -j9 -s
make package
make -C netwerk/test/ xpcshell-tests-remote
make SOLO_FILE=test_simple.js -C netwerk/test/ check-one-remote

and had no problem.


I also tried:

cd src/mozilla-central/objdir-droid
make -j9 -s
make package
make xpcshell-tests-remote

This executed many tests successfully but eventually one of the tests hung. Do we need to handle remote testing of the global set of xpcshell tests?
Attachment #548612 - Flags: review?(jmaher)
(In reply to comment #1)
> This executed many tests successfully but eventually one of the tests hung.
> Do we need to handle remote testing of the global set of xpcshell tests?

Let's just worry about the network tests here and file a follow up bug to get the manifests for other tests properly annotated. Also, be sure to file a bug for any test that gets marked skip-if or fail-if.
Comment on attachment 548612 [details] [diff] [review]
patch to add make targets for remote xpcshell tests

Review of attachment 548612 [details] [diff] [review]:
-----------------------------------------------------------------

why are we adding targets in rules.mk and testsuite-targets.mk?  It would be nicer to just put stuff in testsuite-targets.mk.  It seems like we have 3 copies of this command we are putting in the .mk files.
(Assignee)

Comment 4

6 years ago
(In reply to comment #3)
> why are we adding targets in rules.mk and testsuite-targets.mk?  It would be
> nicer to just put stuff in testsuite-targets.mk.  It seems like we have 3
> copies of this command we are putting in the .mk files.

I am mirroring the existing make targets. Both rules.mk and testsuite-targets.mk have "xpcshell-tests" targets with comments on each pointing to the other:

rules.mk: # See also testsuite-targets.mk 'xpcshell-tests' target for global execution.

testsuite-targets.mk: # See also config/rules.mk 'xpcshell-tests' target for local execution.

I assume there are "historical reasons" for this arrangement, I don't want to change the "xpcshell-tests" targets, and I want to provide "-remote" equivalents to the "xpcshell-tests" targets. 

I am open to alternatives, but not sure of what to do...
(Assignee)

Updated

6 years ago
Whiteboard: [mobile-testing][xpcshell]
(Assignee)

Comment 5

6 years ago
The split between rules.mk and testsuite-targets.mk is more clear to me now. There are slightly different invocations of the xpcshell-tests target:

1. The "global" invocation, used to run the full set of 1200+ tests, spread across many directories but all referenced by testing/xpcshell/xpcshell.ini. Use "make xpcshell-tests" in $objdir for this. In this case, xpcshelltests.py is called with --manifest=... <master xpcshell.ini>

2. The "local" invocation, used to run a single directory of tests. Use "make -C <directory> xpcshell-tests" in $objdir for this. In this case, xpcshelltests.py is called with a list of directories for arguments, and it searches those directories for manifests.

The remote versions in this patch mirror the existing targets:

make xpcshell-tests
make xpcshell-tests-remote

make -C netwerk/test xpcshell-tests
make -C netwerk/test xpcshell-tests-remote

make SOLO_FILE=test_simple.js -C netwerk/test check-one
make SOLO_FILE=test_simple.js -C netwerk/test check-one-remote
(Assignee)

Comment 6

6 years ago
Created attachment 551790 [details] [diff] [review]
patch to add make targets for remote xpcshell tests

Minor update to avoid the symbolic link -- not needed or used for xpcshell tests.
Attachment #548612 - Attachment is obsolete: true
Attachment #551790 - Flags: review?(jmaher)
Attachment #548612 - Flags: review?(jmaher)

Updated

6 years ago
Attachment #551790 - Flags: review?(jmaher) → review+
(Assignee)

Updated

6 years ago
Keywords: checkin-needed
(Assignee)

Comment 7

6 years ago
http://hg.mozilla.org/integration/mozilla-inbound/rev/f2e8ee893fe4
Whiteboard: [mobile-testing][xpcshell] → [mobile-testing][xpcshell] [inbound]
backed out due to orange:
http://tbpl.mozilla.org/?tree=Mozilla-Inbound&rev=d9456378c12d

config/rules.mk needs to match js/src/config/rules.mk. The simple follow up didn't solve the problem because the rules.mk in this push references packager.mk which does not exist in js/src
Keywords: checkin-needed
Whiteboard: [mobile-testing][xpcshell] [inbound] → [mobile-testing][xpcshell]
(Assignee)

Comment 9

6 years ago
(In reply to Brad Lassey [:blassey][blassey@mozilla.com] from comment #8)
> config/rules.mk needs to match js/src/config/rules.mk. The simple follow up
> didn't solve the problem because the rules.mk in this push references
> packager.mk which does not exist in js/src

I tried a few things and came to the conclusion that including package-name.mk was just a bad idea. The package name was needed so that the --apk argument could be passed to remotexpcshelltests.py, and then that .apk could be pushed to the device. I have made changes now so that rules.mk does not include package-name.mk and does not pass the --apk argument; remotexpcshelltests.py is updated (in bug 668349) to search for an apk in $objdir, if no --apk argument is provided.
why can't you use $(ANDROID_PACKAGE_NAME) without including packager.mk
(Assignee)

Comment 11

6 years ago
Created attachment 552595 [details] [diff] [review]
patch to add make targets for remote xpcshell tests

Updated to sync config/ and js/src/config/ versions of rules.mk. --apk argument no longer specified.
Attachment #551790 - Attachment is obsolete: true
(Assignee)

Comment 12

6 years ago
(In reply to Brad Lassey [:blassey][blassey@mozilla.com] from comment #10)
> why can't you use $(ANDROID_PACKAGE_NAME) without including packager.mk

$(ANDROID_PACKAGE_NAME) is something like "org.mozilla.fennec"; we need the name of the APK file, like "fennec-8.0a1.en-US.eabi-arm.apk".
(Assignee)

Comment 13

6 years ago
Created attachment 552600 [details] [diff] [review]
patch to add make targets for remote xpcshell tests
Attachment #552595 - Attachment is obsolete: true
(Assignee)

Comment 14

6 years ago
Created attachment 553854 [details] [diff] [review]
patch to add make targets for remote xpcshell tests

updated for bitrot
Attachment #552600 - Attachment is obsolete: true
(Assignee)

Comment 15

6 years ago
Comment on attachment 553854 [details] [diff] [review]
patch to add make targets for remote xpcshell tests

r=jmaher
Attachment #553854 - Flags: review+
(Assignee)

Comment 16

6 years ago
Pushed to try -- looks good to me.

http://hg.mozilla.org/try/pushloghtml?changeset=ff5c3f0b0f6a
http://tbpl.mozilla.org/?tree=Try&rev=ff5c3f0b0f6a
(Assignee)

Updated

6 years ago
Keywords: checkin-needed
(Assignee)

Updated

6 years ago
Keywords: checkin-needed
(Assignee)

Comment 17

6 years ago
A group of patches is ready for check-in, ideally all together:

https://bug661282.bugzilla.mozilla.org/attachment.cgi?id=548872
https://bug678385.bugzilla.mozilla.org/attachment.cgi?id=554510
https://bug668349.bugzilla.mozilla.org/attachment.cgi?id=553856
https://bug668349.bugzilla.mozilla.org/attachment.cgi?id=553857
https://bug668351.bugzilla.mozilla.org/attachment.cgi?id=553854
Keywords: checkin-needed
Whiteboard: [mobile-testing][xpcshell] → [mobile-testing][xpcshell] see comment 17 for checkin instructions
The 5 patches in comment 17 have been pushed together to try:
http://tbpl.allizom.org/?tree=Try&usebuildbot=1&rev=95446695ead0

Presuming green, I'll push to inbound shortly afterwards :-)
http://hg.mozilla.org/integration/mozilla-inbound/rev/ea0fb4946b29
Status: NEW → ASSIGNED
Keywords: checkin-needed
Whiteboard: [mobile-testing][xpcshell] see comment 17 for checkin instructions → [mobile-testing][xpcshell]
Target Milestone: --- → mozilla9
http://hg.mozilla.org/mozilla-central/rev/ea0fb4946b29
Status: ASSIGNED → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.