826450 - Deploy B2G update tests to an unagi on WebQA's Jenkins

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: General, defect)

Description

[Jonathan Griffin (:jgriffin)]

As soon as bug 821412 gets into mozilla-b2g18, we can start working on getting them running on an unagi in WebQA's Jenkins instance.  Things that need to happen:

1 - A new unagi, dedicated to these tests, needs to be connected to Jenkins.  Since I don't think we can distinguish multiple unagis on the same host, we'll either need to develop some way to do that, or we'll need this unagi connected to its own Jenkins slave.  Stephen, is there a spare mac mini you can use as a new Jenkins slave for this?

2 - We need to figure out some way of actually getting the tests into the job's workspace.  Currently, the required code is split between gecko and the B2G repo.  We can clone both of these into the workspace in the pattern expected by the tests in the short term; longer-term, making this a distinct Python package will make things easier (bug 825284).

3 - We need to figure out which command-line(s) we need to invoke.  Marshall, is the command-line used in https://github.com/mozilla-b2g/B2G/blob/master/scripts/updates.sh the same as we should use?  Do we need any --testvars or anything?

Comment 1

[Dave Hunt [:davehunt] [he/him] ⌚BST]

#1 - we have an unagi in Haxxor that we could potentially use, although this is having difficulties with tests than require connection, so may not be suitable. I'm sure we can locate another Mac Mini in the short term.

#2 - we can use the Multiple SCM plugin to clone both repositories.

Once I have the answer for #3 I would be happy to start setting this up.

Comment 2

[Marshall Culpepper [:marshall_law]]

> 3 - We need to figure out which command-line(s) we need to invoke. 
> Marshall, is the command-line used in
> https://github.com/mozilla-b2g/B2G/blob/master/scripts/updates.sh the same
> as we should use?  Do we need any --testvars or anything?

We'll want to run two different sets of tests:
- The "self-contained" update tests can be run without the generated testvars, and are very simple. You can simply call the b2g updates.sh script directly, or use testing/marionette/client/marionette/venv_b2g_update_test.sh. Keep in mind that these tests will require a pre-forwarded port, since they are not managed by the smoketest frontend

- The update smoketests require a deeper level of integration and configuration with the CI. The WIP documentation currently has most of the info for this:

https://wiki.mozilla.org/B2G/UpdateTesting

Comment 3

[Jonathan Griffin (:jgriffin)]

Any update on this?  Have we identified a mac mini we can use a Jenkins slave for this, and do we have an unagi for it?

Comment 4

[Stephen Donner [:stephend] Not actively reading bugmail]

(In reply to Jonathan Griffin (:jgriffin) from comment #3)
> Any update on this?  Have we identified a mac mini we can use a Jenkins
> slave for this, and do we have an unagi for it?

We have both (tons of Mac Minis + a few Unagis dedicated to hax0r) -- I can optimistically get those set up by the end of this week (want to focus on test-stability first).

Comment 5

[cmtalbert]

(In reply to Stephen Donner [:stephend] from comment #4)
> (In reply to Jonathan Griffin (:jgriffin) from comment #3)
> > Any update on this?  Have we identified a mac mini we can use a Jenkins
> > slave for this, and do we have an unagi for it?
> 
> We have both (tons of Mac Minis + a few Unagis dedicated to hax0r) -- I can
> optimistically get those set up by the end of this week (want to focus on
> test-stability first).

Wouldn't it make more sense to get the mac mini + new unagi set up now so we can parallelize getting this up and running while we continue making the tests more stable?

Comment 6

[Stephen Donner [:stephend] Not actively reading bugmail]

(In reply to Clint Talbert ( :ctalbert ) Not reading bugmail Dec 24 - Jan 7 from comment #5)
> (In reply to Stephen Donner [:stephend] from comment #4)
> > (In reply to Jonathan Griffin (:jgriffin) from comment #3)
> > > Any update on this?  Have we identified a mac mini we can use a Jenkins
> > > slave for this, and do we have an unagi for it?
> > 
> > We have both (tons of Mac Minis + a few Unagis dedicated to hax0r) -- I can
> > optimistically get those set up by the end of this week (want to focus on
> > test-stability first).
> 
> Wouldn't it make more sense to get the mac mini + new unagi set up now so we
> can parallelize getting this up and running while we continue making the
> tests more stable?

Yes, now that we know we're not going to sprinkle tap()s everywhere to get things nominally passing; sure :-)

I'll need help, and our best bet is :davehunt.  I'll get this set up at my desk, and we can move it to hax0r or wherever it should end up eventually, once that's done.

Comment 7

[Dave Hunt [:davehunt] [he/him] ⌚BST]

I believe this is currently targeted for unagi-4 in Web QA's Jenkins configuration.

Comment 8

[cmtalbert]

So, then is this completed?

Comment 9

[Stephen Donner [:stephend] Not actively reading bugmail]

(In reply to Clint Talbert ( :ctalbert ) from comment #8)
> So, then is this completed?

The initial infrastructure + deployment part (build + update script) is done, but as Marshall set this to block both bug 850686 and bug 850696, it's not completely finished.

Also, there's a recent module-import error: http://qa-selenium.mv.mozilla.com:8080/view/B2G/job/b2g.unagi.update.tests/78/console

Comment 10

[Stephen Donner [:stephend] Not actively reading bugmail]

Faramarz, can you find a new owner for this (I'm making the assumption we still very much want/need this coverage), now that Marshall has moved on?  Thanks!