Closed Bug 937568 Opened 13 years ago Closed 5 years ago

JS test harness

Categories

(Webtools Graveyard :: DXR, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED WONTFIX

People

(Reporter: espressive, Assigned: vedad)

References

Details

(Whiteboard: js)

We need to integrate CasperJS with either PhantomJS or SlimerJS (or both) so we can have automated UI tests going forward.
Certainly not going to get this done by the time we push the ui branch live. This new "js" whiteboard flag can serve as a handy filter for when somebody with JS skills wants to work on the project.
No longer blocks: 860271
Whiteboard: js
Generalizing this bug to mean "set up some kind of JS tests for DXR". After a long discussion on #webdev, I now have some understanding of the state of the JS test ecosystem, which is currently undergoing a sort of Cambrian explosion, yielding lots of good choices and difficulty choosing among them. Here's what I favor for DXR: * Karma (to launch browsers) * plus Mocha (to organize and report on tests) * plus SlimerJS (as the DOM) * plus Chai (to assert) Karma is just a thing for launching browsers and getting them to run some JS. I favor it because it supports many backends: anything from real browsers to the headless PhantomJS and SlimerJS. Mocha looks like a port of RSpec to JS. It supports setup/teardown on suites and even nested suites, and it pretty much makes people write docstrings. SlimerJS is headless, which makes it ideal for CI, and it's Gecko, which is probably what most of our users are using. Chai is optional, but I hear it makes DOM assertions easier to write and read. We should get all this running within the vagrant box, kicked off by "make test" so Jenkins can see the result.
Summary: [UI] Integrate CasperJS → JS test harness
Woohoo! Glad to see this moving forward. Is anyone assigned to do this? I know it is currently on my name but, was wondering if someone else is going to run with it. Just an FYI, I would love to work on this :)
I'm making good progress understanding all these levels of a testing framework. I experimented and searched around quite a bit. Menaged to set up almost everything that ErikRose mentioned. Karma really has plugins for all sorts of things, including mocha and chai. Only thing that is in question there seems to be SlimerJS. I would suggest going with PhantomJS since it is well supported for karma and has it's own plugin. I don't really know how to integrate it with jenkins, but I figured it'll be done by "make test" and tweaking the makefile. I am ready to work on this as soon as the PR #319 is merged. Also, here's a good article for further reading: http://attackofzach.com/setting-up-a-project-using-karma-with-mocha-and-chai/
Assignee: schalk.neethling.bugs → vedadux
(In reply to Vedad Hadžić [:vedad][:vedadux] from comment #4) > I'm making good progress understanding all these levels of a testing > framework. I experimented and searched around quite a bit. Menaged to set up > almost everything that ErikRose mentioned. > Karma really has plugins for all sorts of things, including mocha and chai. > Only thing that is in question there seems to be SlimerJS. I would suggest > going with PhantomJS since it is well supported for karma and has it's own > plugin. The reason Erik suggested using Slimer is because it is Gecko based and Phantom is Webkit only. Seeing the majority (perhaps even all) of our users are Firefox users, it makes more sense to use SlimerJS. If you need more assistance in that regard, SlimerJS is actually developed at Mozilla (can't remember who is involved again but, will update the bug when I remember/find out) so, you will be able to find someone that can assist there. > Also, here's a good article for further reading: > http://attackofzach.com/setting-up-a-project-using-karma-with-mocha-and-chai/ Thanks!
Ah yes, join the #slimerjs channel on IRC - phew, that was easy ;)
I found the slimer plugin, couldn't find it at first for some reason...
In case the example comes in handy, here's what Brick does for its karma/mocha/chai setup: https://github.com/mozbrick/brick-storage-indexeddb/blob/master/karma.conf.js
I'm setting up something similar, I expect first commits tomorrow. I had some personal business to attend to so I'm late on my planned schedule. Thanks for the example!
Hey there Vedad, I finally have some time to actually work on DXR again and I would love to take this bug up and make it happen. Are you ok with that, or are you still working on it? Also, I believe you have opened a PR, which I will look for, so I may build on top of that or start from scratch. Let me know if you would still want to be involved with this and we can totally work together.
Flags: needinfo?(vedadux)
The semester holidays are comming up, an I was planning to advance on this bug then. I am totally for cooperation, since it had been some time since I worked on it. If my memory serves me corretly I already set up the JS only tests, and had to add the browser feature testing to it.
Flags: needinfo?(vedadux)
(In reply to Vedad Hadžić [:vedad][:vedadux] from comment #11) > The semester holidays are comming up, an I was planning to advance on this > bug then. I am totally for cooperation, since it had been some time since I > worked on it. If my memory serves me corretly I already set up the JS only > tests, and had to add the browser feature testing to it. Do you have your code somewhere public? I am basing the test harness on Karma, Mocha and Chai and then possible adding Sinon for mocking etc.
(In reply to Vedad Hadžić [:vedad][:vedadux] from comment #13) > Yeah, it's on my fork of DXR - https://github.com/vedadux/dxr > https://github.com/vedadux/dxr/commit/ > e4d9cb343db672c640efe578915ff7bb49ba1f6b That's great, with a few tweaks here and there, we should totally merge this in and start adding tests. One thing I do believe we will also need is Sinonjs. Erik, how do you think we should proceed here so that we can get the work Vedad has done in, my tweaks and what not on top, without him losing credit for the work he has done?
Flags: needinfo?(erik)
Hey, Schalk. Let's add things (like Sinonjs) only as we need them. Doing otherwise is what led to use having 30MB of nodejs dependencies in there right now. (We're trying to pare that down: see https://github.com/mozilla/dxr/pull/380.) But if we need it for the first test, so be it. To preserve Vedad's work and be able to collaborate on it, let's push his work to a DXR branch called "js-tests" in the dxr repo. Then you'll both be able to submit PRs against it. If you need help with git, let me know.
Flags: needinfo?(erik)
Sounds good to me. I reckon Vedad, push away. Erik, how do we push to this new branch, will you create it?
Flags: needinfo?(erik)
Okay, I cherry-picked Vedad's work (it was 1 big commit) onto the new js-tests branch in the mozilla repo: https://github.com/mozilla/dxr/tree/js-tests. You can both file PRs against that.
Flags: needinfo?(erik)
(In reply to Erik Rose [:erik][:erikrose] from comment #17) > Okay, I cherry-picked Vedad's work (it was 1 big commit) onto the new > js-tests branch in the mozilla repo: > https://github.com/mozilla/dxr/tree/js-tests. You can both file PRs against > that. Awesome, thanks Erik!
I can now personally vouch for mocha and chai. I've used them in Fathom, whose early commits (say, https://github.com/mozilla/fathom/tree/024876998e2d9567d2550057cbde68c7c5aa2e9f) can serve as an example of a basic JS test harness ("make test" is the entrypoint). Of course, it doesn't need in-browser testing, so DXR has a bit more work to do there.

DXR is no longer available. Searchfox is now replacing it.
See meta bug 1669906 & https://groups.google.com/g/mozilla.dev.platform/c/jDRjrq3l-CY for more details.

Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → WONTFIX
Product: Webtools → Webtools Graveyard
You need to log in before you can comment on or make changes to this bug.