1664466 - Please assist QA with creating an over quota bookmark collection.

Status: RESOLVED INVALID

(Cloud Services Graveyard :: Operations: Sync, task)

Description

[Rachel Tublitz [:rachel]]

In order to help us test the upcoming sync quota changes we need to provide QA with a collection of bookmarks > 2GB (let's go with 2.25 GB). Can you help create one server side so that the QA team can connect to the account and sync it down to desktop to import into new profiles?

Timing-wise, we need this before the sync quota changes merge and get deployed....however the ETA there for getting them deployed to staging is next Wednesday, so as long as we have this early next week, we should be good.

Comment 1

[Bob Micheletto [:bobm]]

One option would be to write a script that populates a Netscape Bookmark File Formatted bookmarks.html file with example.com bookmarks and makes heavy use of base64 embedded ICON flags. That file could be imported using the standard procedure and synced to a staging server in the usual fashion.

However, I think with a little of assistance from sync-dev we could do one step better and programmatically generate the JSON blobs that we could just insert directly into the DB. It is possible to insert garbage, either directly from SQL or through the load testing script but undecryptable data tends to break Sync clients horribly.

:lina would you be able to assist us with a procedure for generating Sync book mark blobs suitable for scripting? I'm not sure where I can grab a key from a given profile, and once I have it how to generate a bookmark blob.

Comment 2

[Bob Micheletto [:bobm]]

(In reply to Bob Micheletto [:bobm] from comment #1)

One option would be to write a script that populates a Netscape Bookmark File Formatted bookmarks.html file with example.com bookmarks and makes heavy use of base64 embedded ICON flags.

Turns out Sync bookmark records do not store image data. However, we could probably use the description field to hold some ballast data. Thanks to :rnewman for pointing me to the source in slack.

Comment 3

[Lina Butler (ex-Mozilla)]

Sure, we can help with that! There's a command-line tool that Mark wrote in Rust for syncing bookmarks...we could certainly extend it to write the encrypted blobs out to a file, instead of uploading it, and you could insert that into the database server-side.

I'm also curious to know more about how you'd like to test this! Is the intent to make a local profile with ~2.25 GB worth of bookmarks in the Places database, and try to sync that? Or are you thinking of populating a collection for a test user on the server with ~2.25 GB worth of encrypted blobs, and downloading that? When does the quota kick in—are we allowing users to have over 2 GB stored on the server (so it can be downloaded to a new profile, they just can't upload anything), or will they encounter an error right away (can't read or write; the data becomes inaccessible)?

If the new behavior is to fail the sync right away, on the first read for a collection that's over quota...you can totally get away with inserting garbage into the database, since the client shouldn't ever see it. If you want to let clients read, but not write, that's trickier; we have to make a well-formed structure, with folders.

But syncing a client with that many bookmarks is likely to take ages. If we assume each bookmark record is going to be ~1 KB (folders will be bigger—each folder can only contain ~14k children before we bump into the limit on the size of a record...but maybe we can ignore that for fake records?), that's over 2 million bookmarks. At 1k bookmarks per download batch, that would take forever to download, and then merge—and could lock up Firefox, I'm not sure we've ever tested with real Places databases that big! 😅

But it all depends on what exactly you'd like to test! Would you mind giving me a little more background?

Comment 4

[JR Conlin [:jrconlin,:jconlin]]

Right now, the plan is for the quota to kick in once the user has stored over 2GB of data on the server. We'd need to have a profile hit that mark, then try to sync again and ideally fail. It would be nice to have it hit that mark "organically", but yeah, pumping 2GB of data will take a while. I'm able to generate 2GB of "small" records fairly easily as an HTML file, but yeah, firefox is very much Not Happy trying to load that in. I could prefill the database for the user with GBs of garbage data, but I'm not sure how sync client would deal with that.

The data would basically be test ballast. We don't really care what's in there, just that there's enough to trigger the quota system, and not cause the client to abort out for some reason. If there's a way for me to generate that and feed it into the db without using the client, that would be fantastic, since I can fill data far faster than the client can.

Comment 5

[Rachel Tublitz [:rachel]]

Another thing that might make this easier is if we just tweak the quota value for staging down to something smaller that we can more easily test. JR: assuming that the quota is easily configurable via ENV var or something like that, how about if we just bump it down to verify client behaviour?

I don't want to waste a lot of folks time here generating these giant collections that are major edge cases for our users; let's verify that we get the "over quota" failure using a smaller value that we can confirm is able to be imported into staging prior to release?

If you can confirm that this is do-able let's just close this out for now. We want to confirm client behaviour over quota, that's the main bit here. If we can set the quota lower to allow that, I think that's enough for the time being.

Comment 6

[JR Conlin [:jrconlin,:jconlin]]

We can definitely reduce the quota trigger if need be.

Comment 7

[Rachel Tublitz [:rachel]]

Great, then I'm going to close this out.