957627 - uploading file for signing: 400 Invalid token

Status: RESOLVED FIXED

(Infrastructure & Operations Graveyard :: CIDuty, task)

Description

[Carsten Book [:Tomcat]]

hit tbpl a few moments ago

2014-01-08 09:25:08,690 - 60573158673390f6896d181698326c76d9a83f22: uploading for signing
2014-01-08 09:25:09,473 - 60573158673390f6896d181698326c76d9a83f22: error uploading file for signing: 400 Invalid token
2014-01-08 09:25:10,475 - 60573158673390f6896d181698326c76d9a83f22: processing ../../dist//firefox-29.0a1.en-US.linux-x86_64.checksums on https://signing4.srv.releng.scl3.mozilla.com:9110
2014-01-08 09:25:11,115 - 60573158673390f6896d181698326c76d9a83f22: uploading for signing
2014-01-08 09:25:11,849 - 60573158673390f6896d181698326c76d9a83f22: error uploading file for signing: 400 Invalid token

and then it finally gives up with

make[1]: *** [checksum] Error 1
make: *** [upload] Error 2make[1]: Leaving directory `/builds/slave/m-in-l64-000000000000000000000/build/obj-firefox/browser/installer'


example logs:
https://tbpl.mozilla.org/php/getParsedLog.php?id=32698575&tree=Mozilla-Inbound
https://tbpl.mozilla.org/php/getParsedLog.php?id=32698374&tree=Mozilla-Inbound

Comment 1

[Armen [:armenzg]]

I had to restart redis 3 times in the last hour.
It should be a transient issue.

<bhearsum> i've never seen "invalid nonce" like that before...
<bhearsum> but i see that most of these are "invalid token", which i suspect may be related to the insane number of redis restarts that just happened
* mak|afk is now known as mak
<bhearsum> (signing servers use redis to share the valid tokens amongst the different servers)

Comment 2

[Armen [:armenzg]]

<bhearsum> i see signing happening fine now though, fwiw