Deploy B2G update tests to an unagi on WebQA's Jenkins

RESOLVED WONTFIX

Status

Firefox OS
General
RESOLVED WONTFIX
5 years ago
3 years ago

People

(Reporter: jgriffin, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

5 years ago
As soon as bug 821412 gets into mozilla-b2g18, we can start working on getting them running on an unagi in WebQA's Jenkins instance.  Things that need to happen:

1 - A new unagi, dedicated to these tests, needs to be connected to Jenkins.  Since I don't think we can distinguish multiple unagis on the same host, we'll either need to develop some way to do that, or we'll need this unagi connected to its own Jenkins slave.  Stephen, is there a spare mac mini you can use as a new Jenkins slave for this?

2 - We need to figure out some way of actually getting the tests into the job's workspace.  Currently, the required code is split between gecko and the B2G repo.  We can clone both of these into the workspace in the pattern expected by the tests in the short term; longer-term, making this a distinct Python package will make things easier (bug 825284).

3 - We need to figure out which command-line(s) we need to invoke.  Marshall, is the command-line used in https://github.com/mozilla-b2g/B2G/blob/master/scripts/updates.sh the same as we should use?  Do we need any --testvars or anything?
#1 - we have an unagi in Haxxor that we could potentially use, although this is having difficulties with tests than require connection, so may not be suitable. I'm sure we can locate another Mac Mini in the short term.

#2 - we can use the Multiple SCM plugin to clone both repositories.

Once I have the answer for #3 I would be happy to start setting this up.
> 3 - We need to figure out which command-line(s) we need to invoke. 
> Marshall, is the command-line used in
> https://github.com/mozilla-b2g/B2G/blob/master/scripts/updates.sh the same
> as we should use?  Do we need any --testvars or anything?

We'll want to run two different sets of tests:
- The "self-contained" update tests can be run without the generated testvars, and are very simple. You can simply call the b2g updates.sh script directly, or use testing/marionette/client/marionette/venv_b2g_update_test.sh. Keep in mind that these tests will require a pre-forwarded port, since they are not managed by the smoketest frontend

- The update smoketests require a deeper level of integration and configuration with the CI. The WIP documentation currently has most of the info for this:

https://wiki.mozilla.org/B2G/UpdateTesting
(Reporter)

Comment 3

5 years ago
Any update on this?  Have we identified a mac mini we can use a Jenkins slave for this, and do we have an unagi for it?
(In reply to Jonathan Griffin (:jgriffin) from comment #3)
> Any update on this?  Have we identified a mac mini we can use a Jenkins
> slave for this, and do we have an unagi for it?

We have both (tons of Mac Minis + a few Unagis dedicated to hax0r) -- I can optimistically get those set up by the end of this week (want to focus on test-stability first).

Comment 5

5 years ago
(In reply to Stephen Donner [:stephend] from comment #4)
> (In reply to Jonathan Griffin (:jgriffin) from comment #3)
> > Any update on this?  Have we identified a mac mini we can use a Jenkins
> > slave for this, and do we have an unagi for it?
> 
> We have both (tons of Mac Minis + a few Unagis dedicated to hax0r) -- I can
> optimistically get those set up by the end of this week (want to focus on
> test-stability first).

Wouldn't it make more sense to get the mac mini + new unagi set up now so we can parallelize getting this up and running while we continue making the tests more stable?
(In reply to Clint Talbert ( :ctalbert ) Not reading bugmail Dec 24 - Jan 7 from comment #5)
> (In reply to Stephen Donner [:stephend] from comment #4)
> > (In reply to Jonathan Griffin (:jgriffin) from comment #3)
> > > Any update on this?  Have we identified a mac mini we can use a Jenkins
> > > slave for this, and do we have an unagi for it?
> > 
> > We have both (tons of Mac Minis + a few Unagis dedicated to hax0r) -- I can
> > optimistically get those set up by the end of this week (want to focus on
> > test-stability first).
> 
> Wouldn't it make more sense to get the mac mini + new unagi set up now so we
> can parallelize getting this up and running while we continue making the
> tests more stable?

Yes, now that we know we're not going to sprinkle tap()s everywhere to get things nominally passing; sure :-)

I'll need help, and our best bet is :davehunt.  I'll get this set up at my desk, and we can move it to hax0r or wherever it should end up eventually, once that's done.
Depends on: 829613
Depends on: 841935
Assignee: nobody → marshall
I believe this is currently targeted for unagi-4 in Web QA's Jenkins configuration.

Comment 8

5 years ago
So, then is this completed?
(In reply to Clint Talbert ( :ctalbert ) from comment #8)
> So, then is this completed?

The initial infrastructure + deployment part (build + update script) is done, but as Marshall set this to block both bug 850686 and bug 850696, it's not completely finished.

Also, there's a recent module-import error: http://qa-selenium.mv.mozilla.com:8080/view/B2G/job/b2g.unagi.update.tests/78/console
No longer blocks: 850686
Depends on: 850686
Depends on: 870501
Faramarz, can you find a new owner for this (I'm making the assumption we still very much want/need this coverage), now that Marshall has moved on?  Thanks!
Assignee: marshall → nobody
Flags: needinfo?(frashed)
Flags: needinfo?(faramarz)
(Reporter)

Updated

3 years ago
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.