Closed Bug 1228632 Opened 9 years ago Closed 9 years ago

Add all missing Linux64 test jobs

Categories

(Taskcluster :: Services, defect)

Product:
Taskcluster
Component:
Services
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED
Milestone:
mozilla46

People

(Reporter: armenzg, Assigned: armenzg)

References

Details

Attachments

(6 files)

wip - scheduling changes
33.74 KB, patch
dustin
: feedback+
Details | Diff | Splinter Review
MozReview Request: Patch 1 - Bug 1228632 - Add all missing Linux64 jobs
40 bytes, text/x-review-board-request
ahal
: review+
Details
MozReview Request: Bug 1228632 (Patch 1) - Follow up patch
40 bytes, text/x-review-board-request
Details
MozReview Request: Address concerns
40 bytes, text/x-review-board-request
ahal
: review+
Details
MozReview Request: Inherit from the right base classes
40 bytes, text/x-review-board-request
ahal
: review+
Details
Fix mochitest-other and jittests definitions
1.47 KB, patch
jmaher
: review+
Details | Diff | Splinter Review 
I'm making all code changes to match up what we run on Buildbot.
That way we can have more oranges to look at :)
Nothing of this should be run side by side on integration branches iiuc.
Attachment #8694223 - Flags: feedback?(dustin)
Comment on attachment 8694223 [details] [diff] [review]
wip - scheduling changes

Review of attachment 8694223 [details] [diff] [review]:
-----------------------------------------------------------------

::: testing/taskcluster/tasks/branches/base_job_flags.yml
@@ +99,5 @@
>      - android-api-11
>      - android-partner-sample1
>      - android-b2gdroid
>      - linux
> +    - linux64_tc

Why this change?

::: testing/taskcluster/tasks/tests/fx_linux64_mochitest_dt.yml
@@ +14,5 @@
>          loopbackVideo: true
>          loopbackAudio: true
>    extra:
>      chunks:
> +      total: 9

Is this because the 2 chunks were too long?  Or just to get a better resolution of passing/failing chunks?
Attachment #8694223 - Flags: feedback?(dustin) → feedback+
Comment on attachment 8694223 [details] [diff] [review]
wip - scheduling changes

Review of attachment 8694223 [details] [diff] [review]:
-----------------------------------------------------------------

Thanks dustin!
I will be asking ahal to review the final patch.

::: testing/taskcluster/tasks/branches/base_job_flags.yml
@@ +99,5 @@
>      - android-api-11
>      - android-partner-sample1
>      - android-b2gdroid
>      - linux
> +    - linux64_tc

By doing this I avoided running Buildbot linux64 jobs.
For some reason, Buildapi has been awful at cancelling jobs.
This won't be part of the final patch.

::: testing/taskcluster/tasks/tests/fx_linux64_mochitest_dt.yml
@@ +14,5 @@
>          loopbackVideo: true
>          loopbackAudio: true
>    extra:
>      chunks:
> +      total: 9

Unfortunately, I've been needing to chunk more than anticipated.
I was getting 60 minutes time outs.
Patch 1 - Bug 1228632 - Add all missing Linux64 jobs
Comment on attachment 8694897 [details]
MozReview Request: Patch 1 - Bug 1228632 - Add all missing Linux64 jobs

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/26933/diff/1-2/
Attachment #8694897 - Flags: review?(ahalberstadt)
This patch does not make any changes to scheduling.
To highlight, we split off fx_unittest_base.yml to various .yml files including a generic one which can be used by scripts like luciddream.

I have scheduled a try run just to be sure of what I'm talking about.
Comment on attachment 8694897 [details]
MozReview Request: Patch 1 - Bug 1228632 - Add all missing Linux64 jobs

https://reviewboard.mozilla.org/r/26933/#review24467

This looks great! Thanks for testing out all these extra suites, good stuff.

::: testing/taskcluster/tasks/tests/fx_linux64_cppunit.yml:14
(Diff revision 1)
> +        # TODO move linux_unittest.py to a platform specific config

nit: remove this comment

::: testing/taskcluster/tasks/tests/fx_linux64_marionette.yml:18
(Diff revision 1)
> +    description: Marionette run

There's always been confusion about what marionette is, let's put 'Marionette unittest run' here.

::: testing/taskcluster/tasks/tests/fx_linux64_mochitest_dt.yml:18
(Diff revision 1)
> -      total: 2
> +      total: 15

How come you needed so many other chunks? Should we spend some time figuring out why it's so much slower before enabling it?

::: testing/taskcluster/tasks/tests/fx_linux64_mochitest_other.yml:8
(Diff revision 1)
> +    name: '[TC] Linux64 mochitest-gl M(gl)'

s/gl/oth

Though I thought we were splitting a11y and chrome out of "oth".

::: testing/taskcluster/tasks/tests/fx_linux64_mochitest_plain.yml:18
(Diff revision 1)
> -      total: 5
> +      total: 20

Ditto as for devtools. Why is this so much slower?

::: testing/taskcluster/tasks/tests/fx_linux64_web-platform-tests-reftests-e10s.yml:27
(Diff revision 1)
> +      groupSymbol: tc-W

Should be tc-W-e10s

Some general comments:

1. You use the format fx_linux64_<suite>-<flavor>.yml in half the configs, and fx_linux64_<suite>_<flavor>.yml in the other half. These should be consistent.
2. For all the suites that are chunked way higher.. I think we should make sure this is acceptable before scheduling them. Maybe we should look into beefing up their AWS instances first if there are no other obvious performance problems with the image. Either way, I think for now we should land the configs with the chunks they currently have in buildbot, even though they time out. That way bumping up the chunks will have to be a concious decision when they get scheduled.
3. I called the base config "unittest" to make it clear that it was related to the desktop_unittest.py mozharness script. Maybe a better naming scheme would be 'fx_test_base.yml', 'fx_desktop_unittest.yml' and 'fx_desktop_generic.yml'
Attachment #8694897 - Flags: review?(ahalberstadt)
a11y/chrome split out of m-oth was backed out when we hit builder limits on linux64 :(  I could try it again
(In reply to Joel Maher (:jmaher) from comment #21)
> a11y/chrome split out of m-oth was backed out when we hit builder limits on
> linux64 :(  I could try it again

We could separate it in here.
If you show me the patch I can make it happen.
Attachment #8694897 - Flags: review?(ahalberstadt)
Comment on attachment 8694897 [details]
MozReview Request: Patch 1 - Bug 1228632 - Add all missing Linux64 jobs

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/26933/diff/1-2/
Bug 1228632 (Patch 1) - Follow up patch
Address concerns
https://reviewboard.mozilla.org/r/26933/#review24467

> How come you needed so many other chunks? Should we spend some time figuring out why it's so much slower before enabling it?

We will have to re-evaluate this once we get dt to be green.
I wanted to make sure that all dt *debug* jobs would run under 30 minutes.

I filed a bug to follow up on this:
https://bugzilla.mozilla.org/show_bug.cgi?id=1230330

We're currently seeing a lot of time outs in those suites.
For reference:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=042f329cbbb1

> s/gl/oth
> 
> Though I thought we were splitting a11y and chrome out of "oth".

This was a typo.

> Ditto as for devtools. Why is this so much slower?

I actually need even more chunks than 20 to make sure they finish in less than an hour :S

Again, we need to determine the source of the timeouts and slowness.
Attachment #8695533 - Flags: review?(ahalberstadt)
Comment on attachment 8695533 [details]
MozReview Request: Bug 1228632 (Patch 1) - Follow up patch

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/27145/diff/1-2/
Comment on attachment 8695534 [details]
MozReview Request: Address concerns

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/27147/diff/1-2/
Attachment #8695534 - Flags: review?(ahalberstadt)
ahal, I believe I have addressed all the issues, however, mozreview has not cleared the issues when I pushed:
https://reviewboard.mozilla.org/r/26933/#issue-summary

Let me know if there is anything else you would like me to fix.
Comment on attachment 8694897 [details]
MozReview Request: Patch 1 - Bug 1228632 - Add all missing Linux64 jobs

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/26933/diff/2-3/
Comment on attachment 8695533 [details]
MozReview Request: Bug 1228632 (Patch 1) - Follow up patch

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/27145/diff/1-2/
Comment on attachment 8695534 [details]
MozReview Request: Address concerns

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/27147/diff/1-2/
Inherit from the right base classes
Attachment #8695582 - Flags: review?(ahalberstadt)
Comment on attachment 8695533 [details]
MozReview Request: Bug 1228632 (Patch 1) - Follow up patch

https://reviewboard.mozilla.org/r/27145/#review24611

::: testing/taskcluster/tasks/branches/base_job_flags.yml:36
(Diff revision 2)
> -    jittests: /jittest.*/
> +    jittests: /jittests.*/

Are you sure the 's' is what's wanted? It looked right as is before.
Attachment #8695533 - Flags: review?(ahalberstadt)
Attachment #8695534 - Flags: review?(ahalberstadt) → review+
Attachment #8695582 - Flags: review?(ahalberstadt) → review+
Comment on attachment 8695582 [details]
MozReview Request: Inherit from the right base classes

https://reviewboard.mozilla.org/r/27149/#review24615

Thanks!

The issues don't get resolved automatically, you have to go into the review and mark them fixed yourself.
Fyi, for future reference, you might want to consider amending review changes (via hg commit --amend, or hg amend if you have evolve installed) instead of pushing new commits. That way, there's still only a single review request, which makes it a bit less confusing to figure out what the state of the patch is.
Comment on attachment 8694897 [details]
MozReview Request: Patch 1 - Bug 1228632 - Add all missing Linux64 jobs

https://reviewboard.mozilla.org/r/26933/#review24617
Attachment #8694897 - Flags: review?(ahalberstadt) → review+
https://reviewboard.mozilla.org/r/27145/#review24611

> Are you sure the 's' is what's wanted? It looked right as is before.

You're right. I will fix it on push.
https://hg.mozilla.org/integration/mozilla-inbound/rev/fe26fc0a0dc6939e610eba6ba6c81821f10740a4
Bug 1228632 - Add TaskCluster task test definitions for Linux64 (it won't schedule). r=ahal
I'm seeing tc-m(gl) tests on inbound that definitely weren't running prior to this landed. Are you sure about that "it won't schedule" thing?
Flags: needinfo?(armenzg)
Those were definitely running before as tier 2 jobs. Is it possible this patch inadvertently bumped them to tier 1?
https://hg.mozilla.org/mozilla-central/rev/fe26fc0a0dc6
Status: NEW → RESOLVED
Closed: 9 years ago
status-firefox45: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla45
https://hg.mozilla.org/integration/mozilla-inbound/rev/f481f8ed45308ee9a4dd449ad158387886e3f3c3
Bug 1228632 - Remove inadvertedly added Linux64 debug mochitest-gl. DONTBUILD. r=me
@sheriffs, my previous patch added Linux64 debug mochitest-gl by mistake. I've landed a change on inbound to disable its scheduling. I would have also added it to mozilla-central but didn't want to do so without consulting with you first.

Let me know if you would like me to land it for you.
Flags: needinfo?(armenzg)
A couple of changes we missed.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---

        Attachment #8698148 -
        Flags: review?(ahalberstadt)

    
    

    
  
this is run with --interactive, results are not really useful.

    
    

    
  
My apologies for the automatic pushes. They should not coming here.

gardnt: I tried once again with --interactive and all the results are coming back green. I'm not going to use it for now until we determine why.

    
    

    
  
Comment on attachment 8698148 [details] [diff] [review]
Fix mochitest-other and jittests definitions

I forgot the PTO email. Passing to jmaher.

        Attachment #8698148 -
        Flags: review?(ahalberstadt) → review?(jmaher)

    
    

    
  
Comment on attachment 8698148 [details] [diff] [review]
Fix mochitest-other and jittests definitions

Review of attachment 8698148 [details] [diff] [review]:
-----------------------------------------------------------------

::: testing/taskcluster/tasks/tests/fx_linux64_mochitest_other.yml
@@ +14,5 @@
>          loopbackAudio: true
>    extra:
>      suite:
>        name: mochitest
> +      flavor: chrome,a11y

I don't understand this specific line.

        Attachment #8698148 -
        Flags: review?(jmaher)

    
    

    
  
(In reply to Joel Maher (:jmaher) from comment #56)
> Comment on attachment 8698148 [details] [diff] [review]
> Fix mochitest-other and jittests definitions
> 
> Review of attachment 8698148 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> ::: testing/taskcluster/tasks/tests/fx_linux64_mochitest_other.yml
> @@ +14,5 @@
> >          loopbackAudio: true
> >    extra:
> >      suite:
> >        name: mochitest
> > +      flavor: chrome,a11y
> 
> I don't understand this specific line.

Those are the values passed to this:
> --mochitest-suite=chrome,a11y

    
    

    
  
Comment on attachment 8698148 [details] [diff] [review]
Fix mochitest-other and jittests definitions

Review of attachment 8698148 [details] [diff] [review]:
-----------------------------------------------------------------

Passing the review back to ahal.

        Attachment #8698148 -
        Flags: review?(ahalberstadt)

    
    

    
  
Comment on attachment 8698148 [details] [diff] [review]
Fix mochitest-other and jittests definitions

Review of attachment 8698148 [details] [diff] [review]:
-----------------------------------------------------------------

jmaher volunteered for it again :)

        Attachment #8698148 -
        Flags: review?(ahalberstadt) → review?(jmaher)

    
    

    
  
Comment on attachment 8698148 [details] [diff] [review]
Fix mochitest-other and jittests definitions

Review of attachment 8698148 [details] [diff] [review]:
-----------------------------------------------------------------

thanks for explaining this!

        Attachment #8698148 -
        Flags: review?(jmaher) → review+

    
    

    
  
https://hg.mozilla.org/integration/mozilla-inbound/rev/925963c25a513ee5014eb29ec4dfb260c14116ad
Bug 1228632 - Fix Linux 64 mochitest-other definition for TC jobs. DONTBUILD. r=jmaher

    
    

    
  
https://hg.mozilla.org/mozilla-central/rev/925963c25a51
Status: REOPENED → RESOLVED
Closed: 9 years ago → 9 years ago

          status-firefox46:
          --- → fixed
Resolution: --- → FIXED

    
    

    
  
Moving closed bugs across to new Bugzilla product "TaskCluster".

          status-firefox45:
          fixed → ---

          status-firefox46:
          fixed → ---
Component: TaskCluster → Integration
Product: Testing → Taskcluster
Target Milestone: mozilla45 → mozilla46
Component: Integration → Services

          You need to log in
          before you can comment on or make changes to this bug.
        






  

    
  







  

    

      
Attachment

      

      
      
      
      
    

    

      

        

          

            
General

            

              Creator: 



            

            
Created: 

            
Updated: 

            
Size: