1262647 - chunkify returns an empty list for the last chunk

Status: RESOLVED FIXED

(Release Engineering :: Release Automation, defect)

Description

[Rail Aliiev [:rail]]

Hit this in staging where tried to chunkify locales and got [] for the last chunk

see https://gist.github.com/rail/74437aaa63835af095d1b4c3ba7d1f30

Comment 1

[Rail Aliiev [:rail]]

chunkify([x for x in range(35)], 10, 10) is one of the examples

Comment 2

[Jordan Lund (:jlund)]

in staging we tried 35 locales with 10 chunks. But I don't think this is limited to a small subset of total locales against 10 chunks.

afaict we are missing locales after we run chunkify for a variety of total locales: https://gist.github.com/lundjordan/db7dec7885b3d10b6cf9400c3b8268ae

oddly, 89 locales with 10 chunks works and that is what we have been using in our betas in production.. :)

Comment 3

[Jordan Lund (:jlund)]

debugging the 35 locales 10 chunk case, it seems that chunkify does the right thing for the first few chunks..

it takes the 35locales/10chunks remainder (5) and distributes one locale on each of the first 5 chunks ('big_chunk's).

so it knows to put 4 locales in first 5 chunks, and 3 locales in the remaining. However, after the 6 chunk (one chunk greater than the first 5 big_chunks), things start to fall apart and we lose locales to holes in index/splicing.

my head hurts however trying to figure out why..

Comment 4

[Jordan Lund (:jlund)]

Attachment: condition off whether or not this_chunk is one of the big_chunks or not

Comment 5

[Rail Aliiev [:rail]]

Attachment: recursive approach

jinx, I have something to show too! :)

Comment 6

[Rail Aliiev [:rail]]

remote:   https://hg.mozilla.org/build/puppet/rev/3db9321512c7
remote:   https://hg.mozilla.org/build/puppet/rev/a5fab2acd92e

Comment 7

[Rail Aliiev [:rail]]

restarted releaserunner on bm83 and bm85