1091984 - Use SlimerJS instead of PhantomJS for Fireplace's UI tests

Status: RESOLVED FIXED

(Marketplace Graveyard :: Code Quality, defect, P4)

Description

[Christopher Van Wiemeersch [:cvan]]

PhantomJS is headless WebKit (and quite an old version actually). (PhantomJS 2 <https://github.com/ariya/phantomjs/wiki/PhantomJS-2> is still in development, but it's based on WebKit 538.1, which is *still* a year old.)

SlimerJS has a PhantomJS-compliant API but runs Firefox. You can use xvfb to make Firefox run headlessly. And, the best part? You can point it to any *stable* version of Firefox (as long as it's supported in SlimerJS' hardcoded version list).

SlimerJS supports web fonts fantastically, allows the engine to be more update, and enables us to test our site in the browser used by Mozilla Open Web Apps' target audience: Firefox.

Luckily, CasperJS lets you swap out engines pretty easily:

Before:

> $(CASPERJS_BIN) test tests/ui/

After:

> SLIMERJSLAUNCHER?='/Applications/Firefox.app/Contents/MacOS/firefox'
> $(CASPERJS_BIN) test tests/ui/ --engine=slimerjs

––

SlimerJS is easy to set up on Macs locally, but I've previously had problems deploying on stackato/digitalocean; Travis I hear already has Firefox installed, and everything is already virtualised and headless.

Here's a script I've used to install SlimerJS on Mac/Linux for personal projects:

    https://github.com/cvan/phantomHAR/blob/master/install-packages.sh

Here are sample `.travis.yml` files:

   https://github.com/andreasgal/j2me.js/blob/master/.travis.yml
   https://github.com/mnem/travisci-webgl-test/blob/8d0da4ba51074f9a329c4a61231d83fd961f00b5/.travis.yml

––

See bug 963152 for Zippy's bug to use SlimerJS.

Comment 1

[Mark Striemer [:mstriemer]]

This is mostly working in https://github.com/mstriemer/fireplace/commit/e924173c320e3655d9dbf0797ceb19b386906146. I've gotten each file to pass individually but when one test fails more failures seem to happen and flaky tests have kept the whole suite from passing together. Still some cleanup work to do I guess.

Comment 2

[Mark Striemer [:mstriemer]]

Apparently I forgot to update the commit message for this so it still claims that some tests still fail, oh well. https://github.com/mozilla/fireplace/commit/c316b085a676b179a181aa5f7d2dac5afa8ae6a0