842844 - jenkins job hangs during build for services-server-core, serices-server-reg and services-node-assignemnt

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: WebOps: Other, task, P3)

Description

[Ryan Kelly [:rfkelly]]

The following three jenkins jobs are currently failing to build/run correctly:

  services-server-core
  services-server-reg
  services-node-assignment

They appear to hang indefinitely in the early phases of the build process, hogging resources until they are manually aborted.  Example of an aborted build:

  https://ci.mozilla.org/job/services-node-assignment/1583/


I have tried to debug the issue but with no real success.  Some observations:

 * Console output appears to stall indefinitely at "Started by timer\n[EnvInject] - Loading node environment variables."
 * Deleting the job and re-creating it doesn't help, the new job still fails in the same way
 * Removing all build and post-build steps doesn't help, the now-empty job still fails in the same way
 * Renaming the job appears to fix the problem, e.g. moving "services-server-core" to "services-server-core-old".  But that's pretty yuck, and renaming it back to the original name makes the problem re-appear.
 * Setting the job to "Abort the build if it's stuck" doesn't help and doesn't actually cause the job to be aborted.


So I'm a bit lost as to what's going on.  Any advice or suggestions?

In the meantime I have disabled periodic building of these jobs, so they should not tie up server resources for now.

Comment 1

[Brandon Burton [:solarce]]

Is this still an issue? Please let me know and if it is we'll triage for someone to investigate

Thanks

Comment 2

[Ryan Kelly [:rfkelly]]

Thanks for bumping this, I've just checked and the jobs now seem to be completing fine.