1382439 - unable to retrigger jobs via self-serve

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: CIDuty, task)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

on try server, I pushed 3 pushes to compare talos:
https://treeherder.mozilla.org/#/jobs?repo=try&author=[e-mail]&fromchange=d12fad3e0ad30c59f042d79708cf257297b248f8&filter-searchStr=talos

the --rebuild didn't work for the buildbot jobs, then manually retriggering didn't seem to work as well.  I would do the retrigger and see the 'retrigger request sent', then nothing happens, even hours later.

I did this on autoland as well with the same results:
https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=22e24b1ada9431c21c7f1f8a950d68ec94842c83&filter-searchStr=win%20talos

this was working earlier this week, even earlier today (at least on autoland/inbond/beta).

Comment 1

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

I've been retriggering jobs all day. Can you try signing out of treeherder and signing back in and see if it works better?

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

I did retriggers many hours ago (about 7) and jobs never started, I would conclude that this was broken all day today.  As a note, this was for linux/osx/windows talos jobs where I was not able to make retriggers work.

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

this still isn't working for me after logging out/in.  I am on firefox nightly- maybe that is the problem.

Comment 4

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

I haven't been able to retrigger those talos jobs, either, now that I've tried.

Comment 5

[Ed Morley [:emorley]]

<emorley> jmaher: by retrigger, do you mean the standard retrigger button (as opposed to the fancy backfill or "add missing jobs" features)? Also, this is for buildbot jobs right?
<KWierso> emorley: correct
<KWierso> jmaher: tried, but I also can't retrigger those talos jobs, either
<emorley> Treeherder is making the POST to buildapi, so this is a buildapi/buildbot problem
<catlee> https://secure.pub.build.mozilla.org/buildapi/self-serve/jobs
<catlee> {'body': {'msg': 'there was an error processing your request; please check logs on buildbot-master72.bb.releng.usw2.mozilla.com', 'errors': True}, 'request_id': 3365289}

Comment 6

[Nick Thomas [:nthomas] (UTC+12)]

We rotated some credentials but selfserve wasn't automatically restarted to pick that up. catlee restarted it on bm70, and I got bm71/72/73/85.