1054977 - (tbpl-eol) [Meta] Tracker for TBPL EOL

Status: RESOLVED FIXED

(Tree Management Graveyard :: TBPL, defect, P2)

Description

[Ed Morley [:emorley]]

This is still a fair way out, since even after we switch to using treeherder on a daily basis (bug 1030636), the odd workflow will still need it running (eg bug 1050477). However creating this now to keep track.

Comment 1

[Ed Morley [:emorley]]

To clarify how these bugs related to one and another:

This bug is the overall "can we switch of TBPL yet?" bug. When this bug is resolved fixed, TBPL will no longer be running.

It depends on:
  1) Bug 1030636 (alias treeherder-blockers): the absolute minimum for gecko/firefox sheriffs to switch over to treeherder (a Q3 goal).
  2) Bug 1059400 (alias treeherder-need): regressions compared TBPL, that are either followups from #1 for sheriffs, or else necessary for any other uses of TBPL to switch over (eg non-sheriff gecko devs, or those working on other projects eg Thunderbird, addon-sdk).
  3) Other non-regression requirements that will be added directly as dependants of this bug. eg: updating links in docs/hghooks/other tools to point to Treeherder, or setting up redirects on the TBPL domain, finding a new home for mcMerge (which just happens to run from the same box) etc.

Comment 2

[Ed Morley [:emorley]]

Repos that will need updates, since they either link to TBPL or else use its API:

https://github.com/dminor/ouija
https://github.com/catlee/reportor
https://github.com/jmaher/alert_manager
https://github.com/klahnakoski/datazilla-alerts
https://github.com/gkoberger/BugzillaJS
https://github.com/boklm/tor-browser-try
https://github.com/evanxd/tbpl-hou
https://github.com/jhford/try-server-hook
https://github.com/mstange/analyze-tryserver-profiles
https://github.com/mozilla/moz-git-tools
https://github.com/JohnLZeller/release-kickoff
https://github.com/mozilla/build-relengdocs
https://github.com/bbondy/codefirefox

Comment 3

[Ed Morley [:emorley]]

Also:

https://hg.mozilla.org/automation/orangefactor/
https://hg.mozilla.org/hgcustom/version-control-tools/file/default/hgext/bzpost/
https://hg.mozilla.org/hgcustom/version-control-tools/file/default/pylib/mozautomation/
https://wiki.mozilla.org/Buildbot/Talos/Profiling
http://status.mozilla.com/

Comment 4

[Ed Morley [:emorley]]

> https://github.com/dminor/ouija
Bug 1071152.

> https://github.com/catlee/reportor
Bug 1085527 + bug 1085492.

> https://github.com/jmaher/alert_manager
Bug 1085561.

> https://github.com/klahnakoski/datazilla-alerts
Bug 1085704.

> https://github.com/gkoberger/BugzillaJS
https://github.com/gkoberger/BugzillaJS/issues/101

> https://github.com/boklm/tor-browser-try
https://github.com/boklm/tor-browser-try/pull/1

> https://github.com/evanxd/tbpl-hou
This is a TBPL-specific addon for giving notifications when people's Try jobs are complete. IMO this should be implemented natively in treeherder-UI.

> https://github.com/jhford/try-server-hook
https://github.com/jhford/try-server-hook/commit/641f5d54c7db2650ea08cfe8928d7edc068349b8

> https://github.com/mstange/analyze-tryserver-profiles
https://github.com/mstange/analyze-tryserver-profiles/issues/2

> https://github.com/mozilla/moz-git-tools
https://github.com/mozilla/moz-git-tools/pull/24

> https://github.com/JohnLZeller/release-kickoff
https://github.com/JohnLZeller/release-kickoff/pull/1

> https://hg.mozilla.org/automation/orangefactor/
Bug 1085728 + bug 1085730. (Plus also the longer term bug 870616, but that can wait post TBPL EOL).

> https://hg.mozilla.org/hgcustom/version-control-tools/file/default/hgext/bzpost/
Bug 1085719.

> https://hg.mozilla.org/hgcustom/version-control-tools/file/default/pylib/mozautomation/
Bug 1085723 + bug 1085724.

> https://wiki.mozilla.org/Buildbot/Talos/Profiling
I think this may be redundant now there's: https://github.com/mstange/analyze-tryserver-profiles

> http://status.mozilla.com/
Bug 1072189 removed TBPL from status.m.o since to make room for treeherder's entry (finite number of services supported at any one time).

TODO (more docs related, so will need a bit more work):
> https://github.com/mozilla/build-relengdocs
> https://github.com/bbondy/codefirefox

Comment 5

[Ed Morley [:emorley]]

https://hg.mozilla.org/webtools/tbpl/rev/056344599c25
https://hg.mozilla.org/webtools/tbpl/rev/ec1850f9e430

Comment 6

[Ed Morley [:emorley]]

TBPL will be EOLed at the end of the month. 

An announcement was emailed on 8th March (to dev-tree-management, dev-platform, dev-b2g, dev-gaia, firefox-dev, auto-tools):
https://groups.google.com/d/msg/mozilla.dev.tree-management/PeE_nfhhJkQ/aDi4rRm-CGYJ

There have been no objections, so looks like we're good to proceed when the time comes.

Rough plan:
0) Ensure bug 1050477 (mcMerge migration) is complete.
1) Add 302 entries to the in-repo .htaccess to redirect users to Treeherder (https://hg.mozilla.org/webtools/tbpl/file/default/.htaccess).
2) Ask for the TBPL import-buildbot-data.py cron to be stopped (for both prod and dev). At this point TBPL won't be consuming any resources beyond the redirects.
3) Announce TBPL EOL to dev.tree-management.
4) Presuming those 302s work ok, convert them to 301s after a day or two (http://getluky.net/2010/12/14/301-redirects-cannot-be-undon/)
5) <wait N days / a week, in case there are any complaints>
6) Ask for the redirects to be added to Zeus and the tbpl.m.o and tbpl-dev.allizom.o domains to point to that rather than the current hosting.
7) Kill everything with fire \o/
 - Remove entries from Nagios
 - Remove flows (eg to elasticsearch/Bugzilla/...)
 - Remove random puppet entries
 - Remove web/backend/Chief bits from generic
 - Decom DB nodes
 - Remove from New Relic
8) Misc cleanup
 - Update TBPL wiki page
 - Update readme in TBPL repo
 - Double check for any wiki/MDN/other links to TBPL (though shouldn't be anything left after bug 1082602)
 - Revert commits to TBPL repo that did things like disable bug suggestions/lengthen refresh times (so anyone using TBPL code, eg locally to compare against treeherder, gets a more usable Vagrant install)
 - Update TBPL EOL entries on Treeherder page + A-team goal/ongoing task pages

Think that's everything...

Comment 7

[Ed Morley [:emorley]]

> 0) Ensure bug 1050477 (mcMerge migration) is complete.

mcMerge is in it's new home (bug 1050477), and Treeherder links to it have been updated (on staging, needs deploying).

> 1) Add 302 entries to the in-repo .htaccess to redirect users to Treeherder
> (https://hg.mozilla.org/webtools/tbpl/file/default/.htaccess).

This has been landed, and is currently on tbpl-dev, ready to be deployed. Looks to be working well - see bug 1145713 for more info. 

So after a prod push of Treeherder (to pick up the mcMerge link updates + the new infra menu; will happen in the next few days) and a prod push of TBPL (when we're ready for the redirects to be live) and we'll be good to start the switch-off \o/

I'm on PTO now until after the EOL date (back 2nd April), but somehow suspect I might pop by for a few hours to do the deed, given all the blood sweat and tears of the last year.. :-)

Comment 8

[Ed Morley [:emorley]]

(In reply to Ed Morley (Away 23rd March -> 1st April) [:edmorley] from comment #6)
> 0) Ensure bug 1050477 (mcMerge migration) is complete.

Done & deployed link updates.

> 1) Add 302 entries to the in-repo .htaccess to redirect users to Treeherder
> (https://hg.mozilla.org/webtools/tbpl/file/default/.htaccess).

Deployed now (bug 1145713).

> 2) Ask for the TBPL import-buildbot-data.py cron to be stopped (for both
> prod and dev). At this point TBPL won't be consuming any resources beyond
> the redirects.

Bug 1148920.

> 3) Announce TBPL EOL to dev.tree-management.

https://groups.google.com/forum/#!topic/mozilla.dev.tree-management/U4itH2b711Q

> 4) Presuming those 302s work ok, convert them to 301s after a day or two
> (http://getluky.net/2010/12/14/301-redirects-cannot-be-undon/)

Bug 1148915.

Comment 9

[Ed Morley [:emorley]]

(In reply to Ed Morley (Away 23rd March -> 1st April) [:edmorley] from comment #6)
> 6) Ask for the redirects to be added to Zeus and the tbpl.m.o and
> tbpl-dev.allizom.o domains to point to that rather than the current hosting.
> 7) Kill everything with fire \o/
>  - Remove entries from Nagios
>  - Remove random puppet entries
>  - Remove web/backend/Chief bits from generic

Bug 1149753 covers all of the above.

>  - Remove flows (eg to elasticsearch/Bugzilla/...)

Bug 1149758.

>  - Decom DB nodes

Bug 1149564.

>  - Remove from New Relic

I'll do this once the generic cluster parts are removed.

> 8) Misc cleanup
>  - Update TBPL wiki page
>  - Update readme in TBPL repo
>  - Double check for any wiki/MDN/other links to TBPL (though shouldn't be
> anything left after bug 1082602)
>  - Revert commits to TBPL repo that did things like disable bug
> suggestions/lengthen refresh times (so anyone using TBPL code, eg locally to
> compare against treeherder, gets a more usable Vagrant install)
>  - Update TBPL EOL entries on Treeherder page + A-team goal/ongoing task
> pages

I've set a reminder for me to do this when I'm back for vacation.

For now: EOL itself is complete \o/

Comment 10

[Ed Morley [:emorley]]

(In reply to Ed Morley [:emorley] from comment #6)
> 8) Misc cleanup
>  - Update TBPL wiki page

https://wiki.mozilla.org/index.php?title=Sheriffing%2FTBPL&diff=1066242&oldid=1043657

>  - Update readme in TBPL repo

https://hg.mozilla.org/webtools/tbpl/rev/8aff676182f9

>  - Revert commits to TBPL repo that did things like disable bug
> suggestions/lengthen refresh times (so anyone using TBPL code, eg locally to
> compare against treeherder, gets a more usable Vagrant install)

https://hg.mozilla.org/webtools/tbpl/rev/134bdfcf7e2d

>  - Update TBPL EOL entries on Treeherder page + A-team goal/ongoing task
> pages

Done.