756653 - Don't fallback to DB for missing quota entries

Status: RESOLVED FIXED

(Cloud Services Graveyard :: Server: Sync, defect)

Description

[Toby Elliott [:telliott]]

In talking to atoll, we realized that it's damaging to fallback to the db if there's a missing quota entry. In a cache-clear event, this is the main source that hurts the database, and, for the most part we don't really care.

Let's just treat them as 0. Most users won't matter - they're way below quota - and everyone will eventually catch up and exceed their actual value since quotas always run ahead of real values. In the event of a cache-clear event, we'll run a loop to rebuild the users in a background process.

We should still continue to recalculate quota if they hit the cap-1MEG, and we may need to do a fallback if they do a specific quota request, but for normal reads and writes, let's just turn off fallback.

Comment 1

[Ryan Kelly [:rfkelly]]

Attachment: sync1 patch to avoid hitting to db during quota check

Patch to implement this for sync1.1.  I've implemented it so that storage.get_size_left() will not hit the database unless the "recalculate" flag is explicitly provided, while storage.get_total_size() will retain the current behavior of hitting the DB when the cache is empty.

Quota checks call get_size_left() and hence will not fallback to the DB.  Explicit calls to view the quota call get_total_size() and hence will retain their current behavior.

Comment 2

[Ryan Kelly [:rfkelly]]

Attachment: sync2 patch to avoid hitting to db during quota check

And here's the same logic implemented for sync2.0/aitc.

Comment 3

[Toby Elliott [:telliott]]

Comment on attachment 627105 [details] [diff] [review]
sync1 patch to avoid hitting to db during quota check

Perfect. I had a moment of panic when it looked like you'd just added a function, since it suggested that we were going straight to the db for quota checks, but I eventually traced the correct path. This has the advantage of making that a lot clearer, too.

Comment 4

[Ryan Kelly [:rfkelly]]

Attachment: sync2 patch to avoid hitting to db during quota check

Revised sync2 patch for clarity

Comment 5

[Ryan Kelly [:rfkelly]]

http://hg.mozilla.org/services/server-storage/rev/4837ec4bc672

https://github.com/mozilla-services/server-syncstorage/commit/9655860a8ef660c00cf313bc9983a702a065c7e3

Comment 6

[James Bonacci [:jbonacci]]

Will need some QA before this gets shipped/deployed.

Comment 7

[James Bonacci [:jbonacci]]

Will verify this against Sync 1.1 and Sync 2.0