Consider alternatives to LOG(current_load)/LOG(capacity) for selecting least-loaded node

RESOLVED WONTFIX

Status

Cloud Services
Server: Token
RESOLVED WONTFIX
4 years ago
2 years ago

People

(Reporter: rfkelly, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

4 years ago
Our use of LOG() when selecting the least-loaded node can change the order of node selection, as in this example from current tokenserver stage db:


mysql> select id, node, LOG(current_load) / LOG(capacity), current_load/capacity from nodes order by LOG(current_load) / LOG(capacity);
+----+-------------------------------------------+-----------------------------------+-----------------------+
| id | node                                      | LOG(current_load) / LOG(capacity) | current_load/capacity |
+----+-------------------------------------------+-----------------------------------+-----------------------+
|  3 | https://sync-2-us-east-1.stage.mozaws.net |                 0.100343331887994 |                0.0020 |
|  4 | https://sync-3-us-east-1.stage.mozaws.net |                 0.118553382442218 |                0.0003 |
|  2 | https://sync-1-us-east-1.stage.mozaws.net |                 0.159040418239887 |                0.0030 |
+----+-------------------------------------------+-----------------------------------+-----------------------+
3 rows in set (0.00 sec)


The least-loaded node here is sync-3, by a large margin, but the LOG() causes sync-2 to be selected.  Perhaps there's an alternate function we can use to get the increased float precision of LOG() without this shuffling?
(Reporter)

Updated

4 years ago
QA Contact: rfkelly
Without seeing the raw numbers, it's a little hard to comment here.

I can see some ways that a setup might produce weird numbers here, but

a) if the numbers required to generate this are wacky it may not matter in practice. Most denominators will be pretty similar.
b) the occasional misassignment of isolated users is acceptable. If a lot of users need to be assigned before things balance out, that's more of a concern.
(Reporter)

Comment 2

4 years ago
> if the numbers required to generate this are wacky it may not matter in practice.
> Most denominators will be pretty similar.

The reason for this in stage, is that sync-3 is a big hefty box while the other two are much smaller.  So its "capacity" figure is much larger, skewing the denominators as you suggest.

I don't think it's a huge concern, but worth spending an hour or two at some stage to see if there's a better option.
(Reporter)

Comment 3

4 years ago
A quick glance at MySQL's math functions doesn't reveal anything too promising.  Multiplying both sides by PI() might work, but geez...I think I'll just leave it as-is for now :-)
(Reporter)

Comment 4

2 years ago
meh
Status: NEW → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.