1850748 - Add documentation about how to ingest performance data locally

Status: NEW

(Tree Management :: Perfherder, task)

Description

[Greg Mierzwinski [:sparky]]

This bug is for adding some documentation somewhere in the treeherder readthedocs page that explains how to ingest performance data locally for testing (instead of always needing to use staging).

Here are the steps I follow for this:

Setup:
Create a pulse guardian account and run:

export PULSE_URL=amqp://USER:[e-mail]:5671/?ssl=1
yarn install
docker-compose up --build

Run each of these commands in multiple windows:
docker-compose up --build
docker-compose run -e PROJECTS_TO_INGEST=autoland backend celery -A treeherder worker --concurrency 1

Run the db viewer:
dbeaver-ce
Connect to the following while up --build is running

Serverhost: localhost
Port: 3306
Database: treeherder
Username: root
No password

Run in separate window while running above to do ingestion:

Ingest push:

docker-compose exec backend ./manage.py ingest push -p autoland -r 1ee42a54a431acdd6cbe43b49de0237fe67eddd9

Ingest all the tasks, and run celery to trigger the log parsing, and performance data ingestion:

docker-compose exec backend ./manage.py ingest push -p autoland -r 1ee42a54a431acdd6cbe43b49de0237fe67eddd9 -a --enable-eager-celery

For ingesting multple pushes:
docker-compose exec backend ./manage.py ingest push -p autoland --last-n-pushes 100

We can format/modify these instructions to be much clearer.