448069 - Add xulrunner tests

Status: RESOLVED INCOMPLETE

(Toolkit Graveyard :: XULRunner, defect)

Description

[Dave Townsend [:mossop]]

I think it would be worthwhile to create some automated tests for xulrunner. If nothing else it would give us an indication that the release builds are working mostly as expected.

I think a good start would be to just check in a couple of simple xul applications and just run them and check they start up ok.

Comment 1

[Mark Finkle (:mfinkle) (use needinfo?)]

I believe there is already a simple XUL app in the tree and is built with xulrunner. I had been planning on adding an application to run PGO as well.

Comment 2

[Alex Vincent [:WeirdAl]]

(In reply to Mark Finkle (:mfinkle) from comment #1)
> I believe there is already a simple XUL app in the tree and is built with
> xulrunner.

http://mxr.mozilla.org/mozilla-central/source/xulrunner/examples/simple/

What do we need to kickstart this?  I have time and can write both Python code and XUL code to add a test.  I'm thinking something really simple:
(1) Python script to set up the XULRunner app, based on an install-app.py patch (bug 747597)
(2) Modify Python script to set up a dumb HTTP server and launch the XULRunner app
(3) XULRunner app, from its chrome, sends a XMLHttpRequest to the test server and exits
(4) Integrating this into the actively run tests (this is where I would need help).

Comment 3

[Alex Vincent [:WeirdAl]]

Attachment: test for XULRunner on Windows

Comment 4

[Alex Vincent [:WeirdAl]]

Attachment: patch tested on Mac & Windows

Note that this test *fails* on Mac, but not because of the test itself:  the test writes the same JS code as it does on Windows, and to the right place inside the application.  The test failure happens because we can't successfully launch XULRunner on Mac:  it looks for libmozglue.dylib unsuccessfully.

I have not tested this patch on Linux, but it should be fairly close to what it needs to be.

Comment 5

[Alex Vincent [:WeirdAl]]

glandium, conversation in #macdev suggests you might have some insights to offer regarding libmozglue not loading.  Most of the other answers I'm getting there suggest I'll have to fix it myself... which I haven't any good ideas how to do.  (As a starting point, they did mention |man dyld|.)

Comment 6

[Alex Vincent [:WeirdAl]]

(In reply to Alex Vincent [:WeirdAl] from comment #5)
> glandium, conversation in #macdev suggests you might have some insights to
> offer regarding libmozglue not loading.  Most of the other answers I'm
> getting there suggest I'll have to fix it myself... which I haven't any good
> ideas how to do.  (As a starting point, they did mention |man dyld|.)

This has been taken care of in bug 923979.

Comment 7

[Alex Vincent [:WeirdAl]]

Attachment: xulrunner-testlaunch.diff

Comment 8

[Alex Vincent [:WeirdAl]]

Updated to the latest mozilla-central code.

This test is *not* passing on Linux.  The current test code tries to install directly from objdir/dist/, but several files, notably libxul.so, are relative symlinks instead of the .so files we need.  I'm not sure what to do about that yet:  the correct thing to do would be to require |make sdk| before |make check|, but can we support that with our distributed build/test infrastructure?

Comment 9

[Alex Vincent [:WeirdAl]]

gps:  I'm thinking about changing xulrunner/app/Makefile.in in my patch to say the following:

+check::
+	$(MAKE) -C $(DEPTH) sdk
+	@$(PYTHON) $(srcdir)/testLaunch.py $(DIST)/xulrunner-sdk

In other words, using the SDK in the test itself.  Of course, that does mean building the SDK itself, and this might not be a "right" place to do so.

Plus, to make this useful, release engineering would need to build XULRunner with tests enabled, and we're not sure whether all tests that can run should run, or just the check tests, or just the check tests in this directory.

What're your thoughts on this approach?

Comment 10

[Gregory Szorc [:gps]]

Comment on attachment 825142 [details] [diff] [review]
xulrunner-testlaunch.diff

Review of attachment 825142 [details] [diff] [review]:
-----------------------------------------------------------------

Please use the unittest module for running Python tests. This results in a consistent test environment for everything Python. See python/mozbuild/mozbuild/test/test_util.py for an example.

::: xulrunner/app/testLaunch.py
@@ +3,5 @@
> +# Min version of python is 2.7
> +import sys, argparse, os, shutil, tempfile, subprocess, install_app
> +
> +if ((sys.version_info.major < 2) or (sys.version_info.minor < 7)):
> +  raise Exception("You need to use Python version 2.7 or higher")

The build system enforces 2.7.3+, so no need to do this here.

Comment 11

[Gregory Szorc [:gps]]

How CPU intensive is building the sdk? Can it be done in such a way as to not pollute the build output? If we can do this in a way that doesn't add too much infrastructure load and doesn't change the output of builds, I don't see a problem with it.

glandium can probably answer these questions off the top of his head.

Comment 12

[Mike Hommey [:glandium]]

IIRC, make sdk essentially does another package staging and some more file copying, then tars the result. I don't think it affects other things, but i wouldn't put my money on it. That being said, we already do build the sdk as part of the build and my guess is we probably do this before make check is run. So I'd skip creating the sdk in make check, but skip the test nevertheless when the sdk is not there (for people running make check locally without running make sdk first).

Comment 13

[Benjamin Smedberg]

XULRunner has been removed from the Mozilla tree: see https://groups.google.com/forum/#!topic/mozilla.dev.platform/_rFMunG2Bgw for context.

I am closing all the bugs currently in the XULRunner bugzilla component, in preparation for moving this component to the graveyard. If this bug is still valid in a XULRunner-less world, it will need to be moved to a different bugzilla component to be reopened.