1170821 - queue: Azure queue naming collisions

Status: RESOLVED FIXED

(Taskcluster :: Services, defect, P2)

Description

[Jonas Finnemann Jensen (:jonasfj)]

taskcluster-queue uses azure queue storage, one queue for each provisionerId/workerType.

Due to super obscure azure queue storage naming conventions we tried to base64 decode identifiers followed by a base32 encoding, see: 
https://github.com/taskcluster/taskcluster-queue/blob/a51f68aae994d7d13b7e55b524b99b3fc17cf645/queue/queueservice.js#L292-L303
And decodeUrlSafeBase64 in the same file.

However, it turns out that this doesn't work. We have collisions.

As nothing is critically broken right now, and this only affects workerTypes that almost the same name...

We critically have to fix this... And try to do a gradual migration, by returning URLs for multiple queues...

Comment 1

[Jonas Finnemann Jensen (:jonasfj)]

Attachment: GIthub PR

See comments on PR.
This also fixes: bug 1162943, otherwise the legacy queues would just sit around forever, now we get clean up too.

Of course remind me to configure an scheduled job on heroku :)

Comment 2

[Greg Arndt [:garndt]]

Comment on attachment 8616316 [details] [review]
GIthub PR

Looks good.  Had some comments in the PR but nothing major.  I am curious how this will handle multiple priority queues as it's right now hard coded to have two queues (new one, and legacy), but maybe there is something I overlooked about this.

Comment 3

[Greg Arndt [:garndt]]

oh, and I also just took a quick look at how the new queuenames are being constructed and tested it out locally, I assume it's a sane idea and works fine.

Comment 4

[Jonas Finnemann Jensen (:jonasfj)]

Pushed fixed will merge and deploy this later today when an interrupt is less annoying.

I doubt this will cause problems, but we never know...

Comment 5

[Jonas Finnemann Jensen (:jonasfj)]

Rolled out... Next step is to remove the legacy queue names...

We can do so in a week or so, when all messages in them have been consumer or expired.

Comment 6

[Jonas Finnemann Jensen (:jonasfj)]

Attachment: github PR remove legacy queue names

Remove legacy queues... Should just work (tm).

Comment 7

[Jonas Finnemann Jensen (:jonasfj)]

Deployed and working -- I hope..