A recent buildbot upgrade changed alwaysRun steps to instead be almostAlwaysRunExceptWhenStopBuildUsed. This means that when builds or tests are stopped via the 'Stop Build' button, which can happen manually, via cancellator, or self-serve, the machines don't run the reboot step at the end of the job. This is particularly bad for test jobs, since extra processes are often left running that interfere with future tests. I recommend changing cancellator and self-serve to not stop running tests for the moment. I believe stopping running builds is mostly safe.
Do you know what change affected that?
(In reply to comment #1) > Do you know what change affected that? https://github.com/buildbot/buildbot/commit/e0d4533f962a51c60456a5a4fd1a5b4d6dee3c1d Curse the guy who committed that!
Test jobs are now killable again