Closed Bug 1253312 Opened 4 years ago Closed 3 years ago

TC Linux 32 debug builds Tier-1

Categories

(Infrastructure & Operations :: CIDuty, task)

task
Not set

Tracking

(firefox53 fixed)

RESOLVED FIXED
Tracking Status
firefox53 --- fixed

People

(Reporter: selenamarie, Assigned: aselagea)

References

Details

Attachments

(6 files, 1 obsolete file)

This is a planning bug for tracking work related to getting Linux 32 builds Tier-1.
We're ready to push linux32 debug to tier 1. 

Can buildduty please disable the linux32 debug builds and tests in buildbot? You can then make the corresponding changes to treeherder to make the TC linux32 debug jobs show up as tier 1.
Component: Task Configuration → Buildduty
Product: Taskcluster → Release Engineering
QA Contact: bugspam.Callek
Duplicate of this bug: 1330302
See also notes from bug 1330302

we now have all of the linux32 tests running in taskcluster (currently tier-2, although we can configure this in-tree to be tier-1).

For all trunk branches, we are ready to disable linux32-debug (not opt or pgo) from buildbot scheduling.
Assignee: nobody → aselagea
Attachment #8825859 - Flags: review?(kmoir)
Attached file diff_builds.txt
Attached patch bug_1253312_disable_tests.patch (obsolete) — Splinter Review
Attachment #8825861 - Flags: review?(kmoir)
Attached file diff_tests.txt
While doing some testing on my test master, I noticed that buildbot linux32 debug tests are missing for some platforms (e.g. ash) - when I do the diff, they're missing from both old and new file.

I even printed the content of BRANCHES['ash']['platforms']['linux']['ubuntu32_vm']['debug_unittest_suites'] and it's empty. But in TH, we seem to have linux32 debug tests running on buildbot:
e.g. https://treeherder.mozilla.org/#/jobs?repo=ash&revision=de7bf784e619d7fa12b47750faaf7c4c22fa95b8

Am I missing something here?
perhaps something needs to be adjusted in project_branches.py, the platform is included in it

'ash': {
        'enable_talos': False,
        'lock_platforms': True,
        'merge_builds': False,
        'periodic_start_hours': [9, 21],
        'pgo_strategy': 'periodic',
        'platforms': {
            'linux': {
                'slave_platforms': ['ubuntu32_vm'],
            },
            'linux-debug': {
                'slave_platforms': ['ubuntu32_vm'],
            },
Thanks for pointing me to that file! I updated the patch to remove all "linux-debug" entries for the trunk project branches. However, that doesn't affect the difference, as no extra jobs were removed - it seems like mozilla/project_branches.py is not considered at all when computing the total number of jobs. 

I even tried to match "master/master_config.json" file for a linux buildbot master, but got the same results.
Attachment #8825861 - Attachment is obsolete: true
Attachment #8825861 - Flags: review?(kmoir)
It looks like there are changes on default that didn't merge to production with respect to the ash branch.  So when you test on your master you are testing on default which has the changes.  The masters run on the production branches which do not have the changes so the old builders are still triggered and appear on th.

I think this is related to
https://hg.mozilla.org/build/buildbot-configs/rev/fb273fd16706
and 
https://bugzilla.mozilla.org/show_bug.cgi?id=1328915

I just merged default to production so a reconfig should fix this up shortly on our masters.
I verified that master don't show those builders anymore, so you can ask for review on your patches.
Attachment #8825859 - Flags: review?(kmoir) → review+
Comment on attachment 8826187 [details] [diff] [review]
bug_1253312_disable_tests_v2.patch

Thanks for checking Kim!

Maybe I didn't express myself well but my question was not related to the builders - https://bugzilla.mozilla.org/attachment.cgi?id=8825860 shows several "leak" entries (which stand for linux debug builders) being removed, so that looks good.

I was worried about the tests diff on some branches (like "ash"). If I create a test master and try to obtain the initial list of tests (e.g. "old" file), I only get the following thing for "ash": 

(disable-l32dbg-test)[aselagea@dev-master2.bb.releng.use1.mozilla.com disable-l32dbg-test]$ cat old | grep "ash " | grep "Ubuntu"
Ubuntu HW 12.04 x64 ash talos g4 ScriptFactory
Ubuntu HW 12.04 x64 ash pgo talos g4 ScriptFactory
Ubuntu HW 12.04 x64 ash talos g4-e10s ScriptFactory
Ubuntu HW 12.04 x64 ash pgo talos g4-e10s ScriptFactory

So only the talos tests which run on HW. Yet Treeherder shows that we have Ubuntu VM tests running on Linux debug on "ash": https://treeherder.mozilla.org/#/jobs?repo=ash&revision=f88b08dee5c92679632f4066c0b66f375c3f3f70

Applying *this* patch will not change anything (the end result is the desired one, as we no longer want those tests), so asking for a review on this patch too :-).
 
As for the talos tests, they shouldn't be there either: https://dxr.mozilla.org/build-central/source/buildbot-configs/mozilla/project_branches.py#22. That is being treated in bug 1326471 though.
Attachment #8826187 - Flags: review?(kmoir)
Attachment #8826187 - Flags: review?(kmoir) → review+
We don't have those Ubuntu VM tests running on Linux debug on "ash" since I merged default -> production for the buildbot-configs repo on Friday.  See subsequent pushes like this

https://treeherder.mozilla.org/#/jobs?repo=ash&revision=54a50878ea092a81362ea7ede3c68f308450e262
Attachment #8825859 - Flags: checked-in+
Attachment #8826187 - Flags: checked-in+
From what I could see, we only need to switch Linux32 debug builders to tier 1 in TC. The corresponding tests will automatically be switched (excepting those which already have a tier level explicitly set).
Attachment #8827445 - Flags: review?(kmoir)
Attachment #8827445 - Flags: review?(kmoir) → review+
Pushed by kmoir@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/90f04d07ad34
TC Linux 32 debug builds Tier-1 r=kmoir
Attachment #8827445 - Flags: checked-in+
(In reply to Alin Selagea [:aselagea][:buildduty] from comment #15)
> From what I could see, we only need to switch Linux32 debug builders to tier
> 1 in TC. The corresponding tests will automatically be switched (excepting
> those which already have a tier level explicitly set).

Perhaps an update to https://dxr.mozilla.org/mozilla-central/rev/3e275d37a06236981bff399b7d7aa0646be3fee7/taskcluster/taskgraph/transforms/tests.py#412 ?
Thanks Geoff! I uploaded a new patch to add linux32/debug in the list of tier 1 platforms as well.
Attachment #8827824 - Flags: review?(gbrown)
https://hg.mozilla.org/mozilla-central/rev/90f04d07ad34
Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Attachment #8827824 - Flags: review?(gbrown) → review+
Pushed by kmoir@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/b8cf89eff546
TC Linux 32 debug builds Tier-1 r=gbrown
FWIW, this change got blown away by the patch Callek landed directly to m-c for bug 1331986.
Product: Release Engineering → Infrastructure & Operations
You need to log in before you can comment on or make changes to this bug.