Closed Bug 894990 Opened 6 years ago Closed 6 years ago

Expose mochitest test data file URL via SimpleTest.getTestFileURL

Categories

(Testing :: Mochitest, defect)

defect
Not set

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: ochameau, Assigned: ochameau)

References

Details

Attachments

(1 file, 3 obsolete files)

While working on writing an apps related mochitest, I was desperately looking for a way to get the base path for the currently running mochitest.
It is actually very challenging to compute it!
First, I've came across this code:
http://mxr.mozilla.org/mozilla-central/source/dom/tests/browser/browser_webapps_perms_reinstall.js#127
That code ends up being wrong, fails on some environment and ended up being disabled per bug 794920.

Then while trying to expose a safe way to retrieve the base path, I ended up seeing justin's way of doing, that appears to be working well and used on multiple places:
http://mxr.mozilla.org/mozilla-central/source/content/base/test/test_bug338583.html?force=1#253

I'm now suggesting to factorize this little snippet and expose it in SpecialPowers, so that we can easily discover and use this feature!
Note that this code can't work when running tests on Android or B2G, as we don't actually have the tests on device there--we run a HTTP server on a different system and run all the tests over HTTP.
I manually tested modified tests, but here is a try run:
https://tbpl.mozilla.org/?tree=Try&rev=e52d2cf34d17
The app permissions file is currently disabled, but I manually tested it
and some tests fails (since it has been disabled, this test regressed)
 but the modified code pass.

Joel, it looks like you are a potential reviewer for specialpowersAPI,
but feel free to redirect this review to the right person!
Also, should I also right review for each test being modified??
Attachment #777227 - Flags: review?(jmaher)
Ted, Hum, very good point. Would you have any suggestion for a test that needs access to some data file?
For android we push all related files to the device, otherwise we reference them via http.  have you looked at some of the function in chrome-harness.js ?

A chrome test case which accesses files:
mxr.mozilla.org/mozilla-central/source/toolkit/content/tests/chrome/test_righttoleft.xul
Is this doing the same as directoryService.get "CurWorkD" is doing? (bug 838726)
(In reply to Joel Maher (:jmaher) from comment #4)
> For android we push all related files to the device, otherwise we reference
> them via http.  have you looked at some of the function in chrome-harness.js
> ?
> 
> A chrome test case which accesses files:
> mxr.mozilla.org/mozilla-central/source/toolkit/content/tests/chrome/
> test_righttoleft.xul

This file access a data file via http:// so there is no particular issue.
In my case, and existing cases I modified, we need a local file path to either use a file:// uri -or- be able to create a nsIFile instance.

(In reply to Martijn Wargers [:mw22] (QA - IRC nick: mw22) from comment #5)
> Is this doing the same as directoryService.get "CurWorkD" is doing? (bug
> 838726)

Not exactly that, but I think we end up with a very similar scenario, where the file isn't copied to the device.
We do a request to the httpd server that returns the absolute path of the test directory. The httpd server being hosted on a test runner server and the test being run on a mobile device, this path ends up referering to a inexistant file.
It would exists if we push tests file to the device.

I can workaround these issues by doing a binary xhr and copying the required test data file to the tmp folder and returning the path to this file in tmp folder... Do you have any other idea? Otherwise, would you agree on such workaround?
For browser and chrome mochitests, I think we might package the tests into a jar file and push them to the device, but you still probably can't get a file: URI that works there. You'd have to read it out of a chrome:// URL, probably still using XHR.
So, based on your comments and the various disabled tests 
we have on b2g, here is another way of addressing these needs
of data file during tests.
SpecialPowers.getFilePath("mydata.txt") -> /tmp/mydata.txt

This new method should be working on device tests.
So we no longer use test base path. 
Instead we copy the required data file to device /tmp folder,
by downloading it from mochitest http URL.
We end up returning this temporary file copy from /tmp.

I you are fine with this new method I'd like to add a test against it.
Also, we can tune it to avoid doing this copy while running on platform
where mochitest http server runs on the same filesystem.

https://tbpl.mozilla.org/?tree=Try&rev=4a69ef8c491d
Attachment #777227 - Attachment is obsolete: true
Attachment #777227 - Flags: review?(jmaher)
Overall this seems like a sane approach.  The idea behind a chrome.jar for all the tests is that it is placed in the profile as an extra file and we can register chrome://mochitests/chrome/.../filename.xul and access it that way.

That is a system that was designed and implemented, although not really used in a remote case since mobile doesn't support chrome (java front end).  

I am glad you are doing a cleanup of all the files in specialpowers.
So after almost two months fighting with mochitest and b2g,
I ended up using something simplier.
I was unable to get files working on b2g, as content processes used for apps
have extremelly restricted file access (if any!).
So even if I managed to copy the file to the device running the test,
the test itself running in content process wasn't able to access it.

But I still need to access data file next to the test, that aren't shipped in firefox.
So I came up with SpecialPowers.getTestFileURL, that just return the http url
for a test file hosted in tree, with a path relative to the mochitest html file.
Attachment #777838 - Attachment is obsolete: true
Attachment #802289 - Flags: review?(jmaher)
Comment on attachment 802289 [details] [diff] [review]
Expose mochitest data file URL via SpecialPowers.getTestFileURL

Review of attachment 802289 [details] [diff] [review]:
-----------------------------------------------------------------

this is nice and simple.  I am not sure it needs to be in specialpowers, it could live in simpletest.
Attachment #802289 - Flags: review?(jmaher) → review+
I can easily move it to SimpleTest with some help.
Would it be only about adding something like SimpleTest.getTestFileURL in this file?
http://mxr.mozilla.org/mozilla-central/source/testing/mochitest/tests/SimpleTest/SimpleTest.js
Assignee: nobody → poirot.alex
Summary: Expose mochitest file base path via SpecialPowers → Expose mochitest test data file URL via SpecialPowers
yes, that file would be the one.
Blocks: 914604
Attachment #802289 - Attachment is obsolete: true
Attachment #802355 - Flags: review?(jmaher)
Is there any reason you can't just list your file in MOCHITEST_FILES and then XHR it by URL? Everything is installed to /tests/$relativesrcdir, so if you had $srcdir/foo/file.txt you should be able to GET /tests/foo/file.txt.
This method is really a small helper...
This is resilient to any layout change (if the test is moved) and also abstracts how mochitest does actually work internaly.
Summary: Expose mochitest test data file URL via SpecialPowers → Expose mochitest test data file URL via SimpleTest.getTestFileURL
Comment on attachment 802355 [details] [diff] [review]
Use SimpleTest.getTestFileURL instead

Review of attachment 802355 [details] [diff] [review]:
-----------------------------------------------------------------

this is great;  please make sure this runs on try server before pushing.
Attachment #802355 - Flags: review?(jmaher) → review+
Keywords: checkin-needed
https://hg.mozilla.org/integration/fx-team/rev/11f0b6f6618c
Flags: in-testsuite+
Keywords: checkin-needed
Whiteboard: [fixed-in-fx-team]
https://hg.mozilla.org/mozilla-central/rev/11f0b6f6618c
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Whiteboard: [fixed-in-fx-team]
Target Milestone: --- → mozilla26
I wrote a brief note about this helper in mochitest doc:
https://developer.mozilla.org/en-US/docs/Mochitest#Helper_functions
Target Milestone: mozilla26 → ---
You need to log in before you can comment on or make changes to this bug.