1712629 - Finish Treeherder migration - Prototype and Taskcluster-staging setup

Status: RESOLVED FIXED

(Cloud Services :: Operations: Releng, task)

Description

[Sarah Clements [:sclements]]

Prototype's db can be based off of stage and all secrets, cron jobs and workers should be the same as stage.

Taskcluster-staging should be set up differently since it won't be used as often:

• Can be a new, smaller sized database since we'll only store a smaller subset of data

• PROJECTS_TO_INGEST should be added, with the value: autoland,try,ci-admin,ci-configuration,taskgraph, taskgraph-try,ci-admin-try,ci-configuration-try (:bhearsum, please let Jeremy know if you want to consume from different repos than these).

• PULSE_TASK_SOURCES needs to use the same secrets that are stored in the heroku deployment for treeherder-taskcluster-staging.

• The only cron job needs to be: ./manage.py cycle_data --days 14 from:treeherder and it can skip all of the worker_log_parser* workers.

Comment 1

[Jeremy Orem [:oremj]]

These are at https://prototype.treeherder.nonprod.cloudops.mozgcp.net/ and https://tc-staging.treeherder.nonprod.cloudops.mozgcp.net/. These are set to deploy off the "latest" docker image tag for now.

Comment 2

[Sarah Clements [:sclements]]

Thanks! Is the latest docker image pointing to the master or production branch? Can you set prototype to deploy from a specific 'prototype' branch so that we can push to it. Sorry, I forgot to set this up but I've created it now and added some protections to limit non-TH admins pushes to the perf team.

Also, tc-staging deployment should automatically deploy from treeherder-prod.

Can you please make these changes when you're back from PTO?

Comment 3

[Sarah Clements [:sclements]]

To clarify, tc-staging should be updated whenever new staging code is promoted to production. It should always have most updated treeherder-prod code. Is there a way to automate that?

Comment 4

[Jeremy Orem [:oremj]]

Alright, I've changed the pipelines as follows:

• prototype will automatically deploy when someone pushes to the prototype branch on the treeherder github repo.
• tc-staging will automatically deploy directly after production using the same image tag as production.

Comment 5

[Sarah Clements [:sclements]]

Awesome, thanks!