Closed Bug 523402 Opened 10 years ago Closed 10 years ago

when doing a make package-tests, it doesn't include the mobile specific browser chrome tests

Categories

(Testing :: Mochitest, defect)

defect
Not set

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: jmaher, Assigned: jmaher)

References

Details

Attachments

(2 files, 3 obsolete files)

When trying to run browser-chrome mochitests written for fennec, we don't find them in the package-tests file.  This looks to be the case because all of the unittests live in the xulrunner objdir, and our mobile specific tests live int he mobile objdir.  

I tried a make package-tests in the mobile objdir and received errors in the script while it was trying to cd ../../dist/bin/plugins.

My recommendation is to create a cleaner make package-tests that can be run from the mobile objdir.  If we go this route, we will need releng to create another tests binary for us for each build in addition to the normal tests binary.

Ted, you might have ideas here.
I'm surprised that we didn't already have a bug on this, but yeah, as currently implemented "package-tests" can't know about the Fennec objdir, because it's completely separate from the xulrunner objdir. It's the same situation as "make package", really, where the mobile tree jumps through some hoops to package the xulrunner bits.

(Also, I think we should just change Fennec builds to not be a two-pass build, but that belongs in a separate bug.)
Attached patch package mobile tests (obsolete) — Splinter Review
patch to package tests for mobile similar to 'make package-tests' but in the mobile tree.

Run it by doing:
cd $(objdir)/mobile/mobile/installer
make package-mobile-tests
cd $(objdir)/mobile/dist
ls *tests*.bz2

One caveat: if you run this first, then $(objdir)/mobile/make package, make package will delete the $(objdir)/mobile/dist folder and your bz2 file will disappear.

Another thing to note is this will turn the tests into the same testing structure as the xulrunner (firefox) *tests.tar.bz2 file.  So if you untar this in the same directory, it will add the additional mobile browser-chrome tests to mochitest/browser/mobile/*.

I testing this with a fresh clobber build on mozilla-central and 1.9.2 for desktop linux Fennec.  I ran the tests in a virgin directory (make package + make package-mobile-tests only- untar'd to a fresh dir) like so:
cd mochitest
python runtests.py --appname=../fennec/fennec --xre-path=../fennec/xulrunner --browser-chrome --autorun
Assignee: nobody → jmaher
Attachment #407663 - Flags: review?(mark.finkle)
Comment on attachment 407663 [details] [diff] [review]
package mobile tests 

Seems ok to me, but Ted should really have a look
Attachment #407663 - Flags: review?(ted.mielczarek)
Attachment #407663 - Flags: review?(mark.finkle)
Attachment #407663 - Flags: review+
It would be nice if we could run the make package-mobile-tests in the objdir too. This certainly gets us moving in the right direction: running browser-chrome tests!
Blocks: 523758
Attachment #407663 - Flags: review?(ted.mielczarek) → review-
Comment on attachment 407663 [details] [diff] [review]
package mobile tests 

Bleh. Just include $(topsrcdir)/testing/testsuite-targets.mk, and do

package-mobile-tests: stage-mochitest

Then tar up the result? Ideally you could just reuse the package-tests target, but we'd have to do some trickery in there to not try to package all the reftest stuff, etc.
I would like to have it reuse the same logic and code if possible.  I found that what we do for mochitest stage-package (http://mxr.mozilla.org/mozilla-central/source/testing/mochitest/Makefile.in#127), has issues with the /bin and /certs stuff since dist is $(objdir)/mobile vs $(objdir)/xulrunner.
Yeah. I'd take a patch that adds ifdefs or handles failures there silently to make it work.
patch to allow $(objdir)/mobile/mobile: make package-mobile-tests reuse the same logic for package-tests as xulrunner.
Attachment #407663 - Attachment is obsolete: true
Attachment #408536 - Flags: review?(ted.mielczarek)
this is the other part of the patch that uses #ifdef's to fix the 'make package-mobile-tests' from the mobile objdir.
Attachment #408537 - Flags: review?(ted.mielczarek)
Attachment #408537 - Flags: review?(ted.mielczarek) → review-
Comment on attachment 408537 [details] [diff] [review]
package-tests for mozilla-central/1.9.2

+#ifdef WINCE
+DIST_BIN = $(DIST)/bin/xulrunner
+#endif
+
+#ifdef MOZ_PLATFORM_HILDON
+DIST_BIN = $(DIST)/bin/xulrunner
+#endif

This is no good. For one, we already have Firefox WinCE builds, and we'll want to run tests on those. Also, when do the xulrunner bits get stuck in the Fennec dist/bin? You'll want to make sure there isn't an ordering problem between running "make package" and "make package-tests".

I think if you just did
DIST_BIN = $(DIST)/bin, then you could just have the mobile Makefile call stage-mochitest directly, like
package-mobile-tests:
    $(MAKE) stage-mochitest DIST_BIN=$(DIST)/bin/xulrunner
and that would be ok.
Comment on attachment 408536 [details] [diff] [review]
package-tests for the mobile-browser branch

This looks ok, although per my other review comment you might want to change the way this calls stage-mochitest.
Attachment #408536 - Flags: review?(ted.mielczarek) → review+
update for package-mobile-tests for the mobile-browser branch from last review comments.
Attachment #408536 - Attachment is obsolete: true
Attachment #412923 - Flags: review?(ted.mielczarek)
updated patch for package-mobile-tests on mozilla-central and 1.9.2.  Much simpler and tested from fresh clobber builds on both branches.
Attachment #408537 - Attachment is obsolete: true
Attachment #412924 - Flags: review?(ted.mielczarek)
Attachment #412923 - Flags: review?(ted.mielczarek) → review+
Attachment #412924 - Flags: review?(ted.mielczarek) → review+
just verified with a fresh m-c/m-b sync up and clean clobber build that this has no bitrot for patches or functionality.  And the mobile browser-chrome tests still pass!

mfinkle, please check this in when you get a chance.
Blocks: 535922
You need to log in before you can comment on or make changes to this bug.