Created attachment 8535228 [details] Link to Github pull-request: https://github.com/mozilla-b2g/gaia/pull/26722#issuecomment-66664868 Create two jobs to for this. Currently they are pointing to my local branch since it is not merged yet. http://jenkins1.qa.scl3.mozilla.com/job/flame-kk-319.mozilla-central.ui.graphics.smoke.store_reference.preliminary/ This is to store the reference images. What needs to be done here: - The end of flash script fails with this error, and it seems that pip is not installed. 13:59:37 /tmp/build_step_template1390806849779024885.sh: /var/jenkins/2/workspace/flame-kk-319.mozilla-central.ui.graphics.smoke.store_reference.preliminary/mozversion_temp/bin/pip: /var/jenkins/2/workspace/flame-kk-319.mozilla-central.ui.graphics.smoke.store: bad interpreter: No such file or directory - Need to check whether the reference files are saved and stored properly? http://jenkins1.qa.scl3.mozilla.com/job/flame-kk-319.mozilla-central.ui.graphics.smoke.preliminary/ What needs to be done here: - Same pip error as above - Does it retrieve the reference images to the correct location by default? - Does it generate the proper logs?
Also, the second job requires the installation of an add-on to install ImageMagick, which needs apt-get tool.
I've fixed the pip issue, though I can only speculate on the cause. I reduced the job name to flame-kk-319.mozilla-central.ui.graphics.smoke.reference and it started working, perhaps the original was too long? Reference images are appearing in http://jenkins1.qa.scl3.mozilla.com:8080/job/flame-kk-319.mozilla-central.ui.graphics.smoke.reference/ws/tests/python/gaia-ui-tests/reference_images/ and so far they look good to me. The configuration looked generally good, although I've done a few minor cleanups. Feel free to have a look at the job config history to review my changes. A job is currently running so I've yet to see the reference images as artifacts, but it looks like that's been configured correctly. For ImageMagick, have a look at the Custom tool section of the Jenkins configuration at http://jenkins1.qa.scl3.mozilla.com:8080/configure to see the existing tools - you should be able to add a script for obtaining the ImageMagick tool, and then you should only need to configure the test job to require that tool. Feel free to needinfo me if you have further questions.
Just FYI, I made a couple more tweaks. The reference images and screenshots are now directly off the top level in the workspace, which I think it much cleaner than burying them in tests/python/gaia-ui-tests.
thanks! I was spending hours on pip issue and couldn't fix it. I'll study the changes you made and ping you if I have questions.
So it looks like you need sudo privilege to install imagemagick. I added the IMAGEMAGICK entry under Custom Tool section in http://jenkins1.qa.scl3.mozilla.com:8080/configure, and it basically installs from source. I failed to install from binary with the available packages. (It was failing at linking the dependecy libraries) Also I added the custom tool install step in http://jenkins1.qa.scl3.mozilla.com:8080/view/Graphics/job/flame-kk-319.mozilla-central.ui.graphics.smoke/configure job as well. The installation instruction can be found here: http://www.imagemagick.org/script/install-source.php Binary release instruction is here: http://www.imagemagick.org/script/binary-releases.php#unix
Also changed jobs so now it runs all tests, and I removed the reference_images folder path under 'Copy artifacts from another project'
Just wondering, is there way to have the node 28.2 exclusive to graphics? (i.e. other jobs won't run on it) I don't think we need it now, but maybe later when we actually run it often.
(In reply to No-Jun Park [:njpark] from comment #5) > So it looks like you need sudo privilege to install imagemagick. I added > the IMAGEMAGICK entry under Custom Tool section in > http://jenkins1.qa.scl3.mozilla.com:8080/configure, and it basically > installs from source. I failed to install from binary with the available > packages. (It was failing at linking the dependecy libraries) > > Also I added the custom tool install step in > http://jenkins1.qa.scl3.mozilla.com:8080/view/Graphics/job/flame-kk-319. > mozilla-central.ui.graphics.smoke/configure job as well. > > The installation instruction can be found here: > http://www.imagemagick.org/script/install-source.php > > Binary release instruction is here: > http://www.imagemagick.org/script/binary-releases.php#unix I figured it was probably easier to just install imagemagick on all B2G nodes using apt-get. I have now done this. I've triggered a job, let me know if it's working for you now. (In reply to No-Jun Park [:njpark] from comment #7) > Just wondering, is there way to have the node 28.2 exclusive to graphics? > (i.e. other jobs won't run on it) I don't think we need it now, but maybe > later when we actually run it often. It's possible but then if something happens to that node you won't be able to run the image comparison tests. In my opinion it's better to just use the pool of UI nodes.
Component: Marionette → Gaia::UI Tests
Product: Testing → Firefox OS
Pretty sure this is done.
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.