Closed Bug 1563402 Opened 3 months ago Closed 3 months ago

Remove GCP shippable builds

Categories

(Firefox Build System :: Task Configuration, task)

task
Not set

Tracking

(firefox69 fixed)

RESOLVED FIXED
mozilla69
Tracking Status
firefox69 --- fixed

People

(Reporter: mshal, Assigned: mshal)

References

(Blocks 1 open bug)

Details

Attachments

(1 file)

I believe the linux-gcp-shippable/opt and linux64-gcp-shippable/opt tier-3 builds are the last automation builds that still use 1-tier PGO (this ends up getting enabled by mozharness because these tasks include the taskcluster_nightly.py config). I'd like to move these over to 3-tier PGO, which I think can happen one of two ways:

  1. Make these into 3-tier by reusing the profile data from the existing linux-shippable builds. This means linux-gcp-shippable/opt would depend on generate-profile-linux-shippable/opt and instrumented-build-linux-shippable/opt, both running in AWS. This is straightforward and keeps the jobs PGO'd, but doesn't vet a full PGO pipeline in GCP.

  2. Make these into 3-tier by creating new generate-profile-gcp and instrumented-build-gcp tasks. I'm not sure how difficult these would be to get to run in GCP, but presumably since we already do 1-tier PGO there successfully it should work. This means the GCP stack is fully independent from the AWS stack, in case we want to swap out entirely. It does result in some duplicated work, but maybe that's what we want to do with these jobs?

:grenade, it looks like you added the gcp jobs - do you have thoughts here? Or know who would?

Flags: needinfo?(rthijssen)

It might make sense to move the generate-profile task to GCP, but probably not the instrumented-build. These jobs exist to benchmark the performance of GCP workers, and I suspect that the different builds don't vary enough for it to be worthwhile to run the instrumented-build.

:coop is probably the best person to answer this.

Flags: needinfo?(rthijssen) → needinfo?(coop)

(In reply to Tom Prince [:tomprince] from comment #1)

It might make sense to move the generate-profile task to GCP, but probably not the instrumented-build. These jobs exist to benchmark the performance of GCP workers, and I suspect that the different builds don't vary enough for it to be worthwhile to run the instrumented-build.

:coop is probably the best person to answer this.

I support option #1, i.e. re-using existing AWS profile data.

The GCP builds exist at tier 3 right now because we're waiting on the performance characteristics of GCP instances to improve to match AWS. We shouldn't invest too much time in doing new or interesting things in GCP, or indeed standing up unique workloads there, until baseline performance warrants it. Performance might be there by the end of Q3.

Flags: needinfo?(coop)

I was chatting with :coop on IRC, and another option would be just removing the shippable-gcp buids, as they aren't providing much value currently.

Removing them works for me. If necessary, turning them back on and using the PGO profile from existing builds should be straightforward.

Summary: Convert Linux GCP shippable builds to use 3-tier PGO → Remove Linux GCP shippable builds
Attachment #9076277 - Attachment description: Bug 1563402 - Remove Linux GCP shippable builds; r?tomprince → Bug 1563402 - Remove GCP shippable builds; r?tomprince
Summary: Remove Linux GCP shippable builds → Remove GCP shippable builds
Pushed by mshal@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/88d424f4bde2
Remove GCP shippable builds; r=tomprince
Status: NEW → RESOLVED
Closed: 3 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla69
You need to log in before you can comment on or make changes to this bug.