Together with :travis we might have figured out why pings are send super delayed: With Glean v32.2.0 we moved wait/retry functionality into the Rust core. The rate limiter itself throttles to no more than 15 pings in a 60s interval. If throttled it tells the uploader to `wait`. We leverage the default behavior of the WorkManager and on wait return a `Result.retry()` which causes it to follow the default backoff policy: It waits for 10s, then runs the upload worker again. If that happens again it then waits for 20s, runs again, 40s, runs again, ... with an exponential backoff. Now with that many pending pings a single upload worker will always send a burst of pings in a loop, before it returns `wait`. With every wait it suspends the task for longer and longer. If the workmanager task isn't killed the backoff might increase for a long long time, therefore never catching up to the pending pings (especially as each backgrounding/foregrounding creates new pings)
Bug 1665041 Comment 14 Edit History
Note: The actual edited comment in the bug view page will always show the original commenter’s name and original timestamp.
Together with :travis we might have figured out why pings are send delayed: With Glean v32.2.0 we moved wait/retry functionality into the Rust core. The rate limiter itself throttles to no more than 15 pings in a 60s interval. If throttled it tells the uploader to `wait`. We leverage the default behavior of the WorkManager and on wait return a `Result.retry()` which causes it to follow the default backoff policy: It waits for 10s, then runs the upload worker again. If that happens again it then waits for 20s, runs again, 40s, runs again, ... with an exponential backoff. Now with that many pending pings a single upload worker will always send a burst of pings in a loop, before it returns `wait`. With every wait it suspends the task for longer and longer. If the workmanager task isn't killed the backoff might increase for a long long time, therefore never catching up to the pending pings (especially as each backgrounding/foregrounding creates new pings)