925266 - Set up Travis artifacts for Gaia UI tests HTML report

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::UI Tests, defect)

Description

[Dave Hunt [:davehunt] [he/him] ⌚BST]

We should set up Travis artifacts so we can see the HTML report from Gaia UI tests. This includes useful resources for understanding failures (screenshots, page source, settings).

See http://about.travis-ci.org/blog/2012-12-18-travis-artifacts/ for details.

Comment 1

[Gareth Aye [:gaye] (back from PTO)]

+1

Comment 2

[Zac C (:zac)]

It looks like even I could do this but for the part about secret access keys.

James can you help the keys and I'll experiment with the rest?

Comment 3

[James Lal [:lightsofapollo]]

I just did this for a different project, sure :)

The encryption is easy but I need to get the S3 credentials first... I will try to find the right person on the a-team who can generate some keys for us.

Comment 4

[Dave Hunt [:davehunt] [he/him] ⌚BST]

jgriffin: Do you know who might have these credentials?

Comment 5

[Jonathan Griffin (:jgriffin)]

We haven't used S3 on the a-team account yet; lightsofapollo is discussing this with glob atm.

Comment 6

[Zac C (:zac)]

Attachment: github pr

I wanted this to be in the root (of that job) rather than nestled in a folder so people are more likely to look at it.

James, it is entirely self contained so just one file.

Comment 7

[Zac C (:zac)]

Also strictly speaking this does not depend on bug 948177, it could be merged before that with no ill effects.

Comment 8

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Comment on attachment 8345260 [details] [review]
github pr

Looks good. Would like to see the XML output included too, but r+ with or without this change.

Comment 9

[James Lal [:lightsofapollo]]

Comment on attachment 8345260 [details] [review]
github pr

Zac- I don't think this will work as is.. we still need to run the travis-artifacts script (see after_script in s3uploads).

Comment 10

[Zac C (:zac)]

Oh, maybe I have mis-understood the after_script but I thought this, if merged, would mean the html report would just sit in artifacts/ unused until #948177 is merged?

Comment 11

[Zac C (:zac)]

(In reply to Dave Hunt (:davehunt) from comment #8)

> Looks good. Would like to see the XML output included too, but r+ with or
> without this change.

I just couldn't see the reason to include the XML output if there's no clear need for it. (and easily added when there is a need)

Comment 12

[Dave Hunt [:davehunt] [he/him] ⌚BST]

(In reply to Zac C (:zac) from comment #10)
> Oh, maybe I have mis-understood the after_script but I thought this, if
> merged, would mean the html report would just sit in artifacts/ unused until
> #948177 is merged?

It will need a follow-up pull to add tests/travis_ci/gaia_ui_tests/after_script to actually perform the upload as in James' example.

Comment 13

[Zac C (:zac)]

OK! don't merge this now; I'll re-do it later after https://github.com/mozilla-b2g/gaia/pull/14519 is merged.

Comment 14

[James Lal [:lightsofapollo]]

Comment on attachment 8345260 [details] [review]
github pr

I will do the final upload step later today and mark Zac for review... all we need is to call travis artifacts and output a link url.

Comment 15

[Zac C (:zac)]

Attachment: github pr

Comment 16

[Zac C (:zac)]

Note dave this won't work on pull requests because a pull is outside the repo for which the key was generated.

I understand this means it will work on merge commits only for the time being.

Comment 17

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Comment on attachment 8348671 [details] [review]
github pr

Comments in pull request.

Comment 18

[Zac C (:zac)]

Comment on attachment 8348671 [details] [review]
github pr

updated the PR

Comment 19

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Comment on attachment 8348671 [details] [review]
github pr

Looks good, but I'm not familiar with the various components here. Will appreciate James' review too.

Comment 20

[James Lal [:lightsofapollo]]

Comment on attachment 8348671 [details] [review]
github pr

https://github.com/mozilla-b2g/gaia/commit/83cc63f728489a24256731adf558354bb2012a59

Comment 21

[Zac C (:zac)]

James, this has regressed. I'm not sure when it started.

Can you investigate?


home/travis/.rvm/gems/ruby-1.9.3-p484/gems/faraday-0.9.0.rc7/lib/faraday.rb:189:in `lookup_middleware': :json is not registered on Faraday::Response (Faraday::Error)

from /home/travis/.rvm/gems/ruby-1.9.3-p484/gems/faraday-0.9.0.rc7/lib/faraday/rack_builder.rb:203:in `use_symbol'

from /home/travis/.rvm/gems/ruby-1.9.3-p484/gems/faraday-0.9.0.rc7/lib/faraday/rack_builder.rb:96:in `response'

from /home/travis/.rvm/gems/ruby-1.9.3-p484/gems/travis-artifacts-0.2.2/lib/travis/client.rb:39:in `block in create_connection'

from /home/travis/.rvm/gems/ruby-1.9.3-p484/gems/faraday-0.9.0.rc7/lib/faraday/connection.rb:91:in `initialize'

from /home/travis/.rvm/gems/ruby-1.9.3-p484/gems/faraday-0.9.0.rc7/lib/faraday.rb:70:in `new'

from /home/travis/.rvm/gems/ruby-1.9.3-p484/gems/faraday-0.9.0.rc7/lib/faraday.rb:70:in `new'

from /home/travis/.rvm/gems/ruby-1.9.3-p484/gems/travis-artifacts-0.2.2/lib/travis/client.rb:38:in `create_connection'

from /home/travis/.rvm/gems/ruby-1.9.3-p484/gems/travis-artifacts-0.2.2/lib/travis/client.rb:9:in `initialize'

from /home/travis/.rvm/gems/ruby-1.9.3-p484/gems/travis-artifacts-0.2.2/lib/travis/artifacts/cli.rb:14:in `new'

from /home/travis/.rvm/gems/ruby-1.9.3-p484/gems/travis-artifacts-0.2.2/lib/travis/artifacts/cli.rb:14:in `initialize'

from /home/travis/.rvm/gems/ruby-1.9.3-p484/gems/travis-artifacts-0.2.2/bin/travis-artifacts:4:in `new'

from /home/travis/.rvm/gems/ruby-1.9.3-p484/gems/travis-artifacts-0.2.2/bin/travis-artifacts:4:in `<top (required)>'

from /home/travis/.rvm/gems/ruby-1.9.3-p484/bin/travis-artifacts:23:in `load'

from /home/travis/.rvm/gems/ruby-1.9.3-p484/bin/travis-artifacts:23:in `<main>'

from /home/travis/.rvm/gems/ruby-1.9.3-p484/bin/ruby_executable_hooks:15:in `eval'

from /home/travis/.rvm/gems/ruby-1.9.3-p484/bin/ruby_executable_hooks:15:in `<main>'

Done. Your build exited with 1.

Comment 22

[Dave Hunt [:davehunt] [he/him] ⌚BST]

This bug was resolved, so I think we should open a new bug for the regression rather than reopen this bug.

Comment 23

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Looks like this is working again.