It crossed my mind, it might be handy to have a simple, very small list of key functionality to run through (or if we get fancy later some automated equivalent) to test a local instance of treeherder before opening PR's. For example testing pinning, repo switching, filters, etc, or whatever has historically proven fragile. We might want one for people who are staff and have full functionality access, or a subset which works for anyone. We could possibly post this bullet point list on the Wiki as a link in the Contributing section. Not that it would be mandatory, but it may help encourage that mindset during development and prior to commits.
camd has an etherpad going of potential tests, so cross referencing them here https://etherpad.mozilla.org/treeherder-test-checklist
We might similarly want to run through these smoke tests on an instance eg. stage, before a push to production.
Camd has migrated the wip test list from etherpad to moztrap https://moztrap.mozilla.org/runtests/environment/6497
We've been using these tests for a while prior to deployments to prod, they're on Moztrap per above, and will evolve there and some subset perhaps as automated tests. So marking this bug Fixed for now.