STR: 1. https://build.mozilla.org/buildapi/self-serve/mozilla-central/rev/1edfae12015b 2. type "andr" in the builds section's search box 3. click rebuild for the Android build Expected: Retrigger it Actual: 502 Proxy Error Proxy Error The proxy server received an invalid response from an upstream server. The proxy server could not handle the request POST /buildapi/self-serve/mozilla-central/build. Reason: Error reading from remote server Apache/2.2.3 (Red Hat) Server at cruncher.build.mozilla.org Port 80
This is also happening for me when I try to trigger any additional reftest jobs (any platform*) on my tryserver push: https://build.mozilla.org/buildapi/self-serve/try/rev/e1e4c67862d8 * (only linux debug reftest & linux opt reftest are available, since I TryChooser'd those in particular)
I restarted buildapi on cruncher, by impersonating catlee. There's all sorts of things we could be doing better here, many of which are filed already.
Didn't quite survive for 24 hours, which is considerably worse than the usual (though it's not up to the point of nagios bitching about the age of pending and running, the way it was yesterday).
I've restarted both parts this time (cruncher and the agent on pm03). The last thing that agent.log logged was: 2011-04-10 11:43:30,963 cancelling build by email@example.com of 3058530 2011-04-10 11:43:30,970 build is complete, nothing to do [snipped the channel shutdown] and it failed to do this (from access.log): 2011-04-10 15:58:45,704 INFO [buildapi.access] [worker 19] firstname.lastname@example.org accessing POST /self-serve/try/build 2011-04-10 15:58:45,861 INFO [buildapi.access] [worker 19] email@example.com rebuild_build try 3059570 0 philor successfully did a rebuild after that.
Hitting this again
Looks like it's working at the moment. Did someone restart without commenting ?
We have steps in our Releng Sheriff notes on how to kick the buildapi when needed so this bug can be closed now and if there are proxy errors in the future that require a kick, ping *-buildduty in #build or #developers