973512 - The "marionette_js_stable_check" action might exceed the limit time on Travis.

Status: RESOLVED FIXED

(Testing Graveyard :: JSMarionette, defect)

Description

[Evan Tseng [:evanxd]]

The "marionette_js_stable_check" action might exceed the limit time(50 mins) on Travis.
We do not need to build the profile for each time doing "make test-integration" in the "marionette_js_stable_check" action.

Do not build a profile with doing "make test-integration" if there is already a profile.
It will save much time, and the action might not exceed 50 minutes.

Comment 1

[Evan Tseng [:evanxd]]

Hi Yuren,

How do you think about that
we avoid building a profile, if there is profile there?

Comment 2

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

Discussed with Evan offline.

Comment 3

[Evan Tseng [:evanxd]]

This bug is related with Bug 969215.
If Bug 969215 is fixed, then we should remove the patch of Bug 973512 from the code base.

Comment 4

[EJ Chen [:eragonj][:小龍哥][Contributor mode on](inactive)]

Took this bug !

Comment 5

[Evan Tseng [:evanxd]]

And thanks for the EJ's help. :)

Comment 6

[EJ Chen [:eragonj][:小龍哥][Contributor mode on](inactive)]

https://travis-ci.org/mozilla-b2g/gaia/builds/19168856
  
Yean, with this patch, we just only build once and run tests 30 times and it really accelerates the process for marionette tests !

Comment 7

[EJ Chen [:eragonj][:小龍哥][Contributor mode on](inactive)]

Attachment: patch on master

Hi Evan & Yurenju,

can you guys help me take a look at this patch !? 

Big thanks :P

Comment 8

[Evan Tseng [:evanxd]]

Comment on attachment 8378170 [details] [review]
patch on master

Hi Gareth,

Could you give feedback for the patch.
Thanks. :)

Comment 9

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

Comment on attachment 8378170 [details] [review]
patch on master

r=yurenju if you use |PROFILE_FOLDER=profile-test make| to generate and use |make test-integration-test| to run tests without building profile.

please make sure travis-ci is green with your last change

Comment 10

[Evan Tseng [:evanxd]]

Hi EJ,

The Bug 973876 is already landed.

Comment 11

[EJ Chen [:eragonj][:小龍哥][Contributor mode on](inactive)]

Some related information : http://docs.travis-ci.com/user/build-configuration/#Build-Timeouts

Comment 12

[Evan Tseng [:evanxd]]

Comment on attachment 8378170 [details] [review]
patch on master

I'm fine with it, if yuren r+ the patch.

Comment 13

[Evan Tseng [:evanxd]]

(In reply to EJ Chen [:eragonj][:小龍哥] from comment #11)
> Some related information :
> http://docs.travis-ci.com/user/build-configuration/#Build-Timeouts

The 50 mins timeout is a big issue we need to consider,
or some job might have timeout issue.

Comment 14

[EJ Chen [:eragonj][:小龍哥][Contributor mode on](inactive)]

Hi Yurenju, after discussing with Evan, we decided to break this job into 5 different jobs so that we can make sure we won't hit 50 minutes timeout issue. (http://docs.travis-ci.com/user/build-configuration/#Build-Timeouts).

What do you think :) ?

Comment 15

[EJ Chen [:eragonj][:小龍哥][Contributor mode on](inactive)]

We think if we break this job into 5 different jobs would dispatch the loads and make each test not to exceed 50 mins timeout. And I made a test on bubble-tea branch here (https://travis-ci.org/mozilla-b2g/gaia/builds/19551056)

Comment 16

[Evan Tseng [:evanxd]]

(In reply to EJ Chen [:eragonj][:小龍哥] from comment #15)
> We think if we break this job into 5 different jobs would dispatch the loads
> and make each test not to exceed 50 mins timeout. And I made a test on
> bubble-tea branch here
> (https://travis-ci.org/mozilla-b2g/gaia/builds/19551056)

I think we could do the way first.
I filed a follow up bug 976448.
Once we have the reporter, I think we should remove the 30 times things.

Comment 17

[EJ Chen [:eragonj][:小龍哥][Contributor mode on](inactive)]

(In reply to Evan Tseng [:evanxd][:愛聞插低] from comment #16)
> (In reply to EJ Chen [:eragonj][:小龍哥] from comment #15)
> > We think if we break this job into 5 different jobs would dispatch the loads
> > and make each test not to exceed 50 mins timeout. And I made a test on
> > bubble-tea branch here
> > (https://travis-ci.org/mozilla-b2g/gaia/builds/19551056)
> 
> I think we could do the way first.
> I filed a follow up bug 976448.
> Once we have the reporter, I think we should remove the 30 times things.

Yeah, based on our off-line discussions with Yurenju, let's land this patch first !

Comment 18

[EJ Chen [:eragonj][:小龍哥][Contributor mode on](inactive)]

Thanks all, 

this patch just got landed on Gaia/master : 

https://github.com/mozilla-b2g/gaia/commit/38e78223d4d5f8fa0f092bbc7b6ef5e8fd201098