Create a job to run gaia-ui-tests against a Spark build (on Flame)

RESOLVED FIXED

Status

Firefox OS
Gaia::UI Tests
RESOLVED FIXED
3 years ago
3 years ago

People

(Reporter: jlorenzo, Assigned: jlorenzo)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

(Assignee)

Description

3 years ago
As we currently have 2 ways of building the Flame (one without the spark apps and one with), we need to support the second way.

We don't have currently have spark build on Flame. So as a temporary workaround, I propose to add 
> GAIA_DISTRIBUTION_DIR=distros/spark make reset-gaia
right after a flash in our builds. Having the job is a high priority, if we want to have bug 1180624 running.

We'll deal with the download job in a follow up bug.
(Assignee)

Updated

3 years ago
Assignee: nobody → jlorenzo
Created attachment 8629955 [details] [review]
[gaia] JohanLorenzo:bug-1180651 > mozilla-b2g:master
(Assignee)

Comment 2

3 years ago
After taking a look at Jenkins configuration, we can't make our own gaia on Jenkins. Reasons:
1. At the beginning, we just do a sparse checkout on /tests/atoms and /tests/python/gaia-ui-tests.
2. Even with making full copies of the repo, we don't have the dependencies to create a gaia build.

Hence, it's not worth to do this workaround. Let's create a new flame build on TC and grab it with Jenkins.

http://jenkins1.qa.scl3.mozilla.com/view/UI/job/flame-kk-319.mozilla-central.nightly.ui.functional.spark.bitbar/4/console
(Assignee)

Updated

3 years ago
Depends on: 1180736
(Assignee)

Updated

3 years ago
Depends on: 1184935
(Assignee)

Comment 3

3 years ago
Comment on attachment 8629955 [details] [review]
[gaia] JohanLorenzo:bug-1180651 > mozilla-b2g:master

This simple test is just to check that the new apps are present on the home screen. Here's a try run [1].

The spark flag is run in this job[2]. The spark build is downloaded from taskcluster in this job[3]. The configuration of the download job is not very factorized, but I'd prefer to deal with the common parts once we need them (in the aries downlads). So now, we'd be able to make Mermi's tests run in our labs.

Also, please note: I didn't create a spark job due to the issue we currently have with the jobs (bug 1186837, and bug 1186835). 

[1] http://jenkins1.qa.scl3.mozilla.com/view/UI/job/flame-kk-spark-319.mozilla-central.nightly.ui.functional.spark/8/console
[2] http://jenkins1.qa.scl3.mozilla.com/view/UI/job/flame-kk-spark-319.mozilla-central.nightly.ui.functional.spark
[3] http://jenkins1.qa.scl3.mozilla.com/view/UI/job/flame-kk-spark.mozilla-central.nightly.download.tc/
Attachment #8629955 - Flags: review?(martijn.martijn)
Attachment #8629955 - Flags: review?(jdorlus)
Attachment #8629955 - Flags: review?(martijn.martijn) → review+
(In reply to Johan Lorenzo [:jlorenzo] (QA) from comment #3)
> Comment on attachment 8629955 [details] [review]
> [gaia] JohanLorenzo:bug-1180651 > mozilla-b2g:master
> 
> This simple test is just to check that the new apps are present on the home
> screen. Here's a try run [1].

I'm wondering about this test. This list of apps might change, so we might need to change this test often, too, then.
We don't have a similar test for the default apps for the Flame device. Do we need one there? Also do we need a negative test for the Flame device? That there are no other apps (like Spark apps) installed on the device?
(Assignee)

Comment 5

3 years ago
I agree, the list will likely change, the maintenance cost wouldn't be that big, tough. We don't have a test like this for the Flame but I think it's worth it. I've seen some apps disappear right after a flash. We'd be able to catch that.

For the negative test case, I think too much for an end to end test. I'd see it more in at an integration level.

Updated

3 years ago
Attachment #8629955 - Flags: review?(jdorlus) → review+
(Assignee)

Comment 6

3 years ago
Merged in master at https://github.com/mozilla-b2g/gaia/commit/a576c2f32d54f0e65c003a31146e661d9993fdb7
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
(Assignee)

Comment 7

3 years ago
I deleted the job [1] as the spark jobs are on Task Cluster and bitbar has no way to get downloads from them, at the moment. It will be solved in bug 1189354.

[1] http://jenkins1.qa.scl3.mozilla.com/view/UI/job/flame-kk-319.mozilla-central.nightly.ui.functional.spark.bitbar/
You need to log in before you can comment on or make changes to this bug.