1465129 - Collect some telemetry data related to the storage.local "JSONFile to IDBBackend" data migration failures

Status: VERIFIED FIXED

(WebExtensions :: Storage, enhancement, P1)

Description

[Luca Greco [:rpl] [:luca] [:lgreco]]

This issue is being created as a follow up for Bug 1406181, in particular from Bug 1406181 comment 193:

> In the last update push to mozreview, I've changed the 
> behavior expected when "the data has been successfully 
> read from the old JSONFile backend but it failed to be
> saved in the new IDB backend" to keep using the JSONFile 
> backend and retry to migrate to the new backend on the 
> next extension load (as Andrew and I have been discussed
> and agreed on last week).

as discussed with aswan, this "data migration failure" scenario should be very unlikely, but we may want to collect some telemetry data to learn how often it is happening in the wild (and maybe also how many of the installed extensions are still using the old JSONFile backend while the IDBBackend is enabled).

Comment 1

[Luca Greco [:rpl] [:luca] [:lgreco]]

Attachment: request-data-review-bug-1465129.md

Comment 2

[Luca Greco [:rpl] [:luca] [:lgreco]]

Attachment: Bug 1465129 - Collect telemetry data related to the storage.local "JSONFile to IDBBackend" data migrations.

Review commit: https://reviewboard.mozilla.org/r/251824/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/251824/

Comment 3

[Luca Greco [:rpl] [:luca] [:lgreco]]

Comment on attachment 8986485 [details]
request-data-review-bug-1465129.md

Hi :chutten,
This data review request is related to a new telemetry histogram related to the data migration from the storage.local JSONFile to the IndexedDB backend.

Besides the data collected into this new histogram, I'd also like to collect some telemetry data on a per-addon basis (e.g. by including a new "storageLocalIDBBackend" boolean property into the environment's "addons.activeAddons.<addon id>" objects that are part of the main ping, and possibly also some information related to the last migration error that prevented the extension to migrate successfully), is it better for you if I also include those into the same data review request or into a separate one?

Comment 4

[Luca Greco [:rpl] [:luca] [:lgreco]]

Comment on attachment 8986498 [details]
Bug 1465129 - Collect telemetry data related to the storage.local "JSONFile to IDBBackend" data migrations.

Adding the r? also on the patch, for a feedback on the change to the Histograms.json file, in particular related to the chosen type ("categorical") and the taxonomy of the "category" values

(The patch currently defines only two categories: "success" and "failure", but there is a third one that I'm thinking about, which would be related to a "successfull data migration from a corrupted file", 'cause in that case we are migrating "successfully" to the new backend but the old data is not going to be available to the user, and it would be great to know the incidence also of this third scenario).

Comment 5

[Chris H-C :chutten]

Comment on attachment 8986498 [details]
Bug 1465129 - Collect telemetry data related to the storage.local "JSONFile to IDBBackend" data migrations.

https://reviewboard.mozilla.org/r/251824/#review258306

f+

Comment 6

[Chris H-C :chutten]

(In reply to Luca Greco [:rpl] from comment #3)
> Comment on attachment 8986485 [details]
> request-data-review-bug-1465129.md
> 
> Hi :chutten,
> This data review request is related to a new telemetry histogram related to
> the data migration from the storage.local JSONFile to the IndexedDB backend.
> 
> Besides the data collected into this new histogram, I'd also like to collect
> some telemetry data on a per-addon basis (e.g. by including a new
> "storageLocalIDBBackend" boolean property into the environment's
> "addons.activeAddons.<addon id>" objects that are part of the main ping, and
> possibly also some information related to the last migration error that
> prevented the extension to migrate successfully), is it better for you if I
> also include those into the same data review request or into a separate one?

Drop it into a second one, it's different enough. (for instance, there is no built-in expiry mechanism)

Comment 7

[Chris H-C :chutten]

Comment on attachment 8986485 [details]
request-data-review-bug-1465129.md

DATA COLLECTION REVIEW RESPONSE:

    Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate? 

Histograms.json

    Is there a control mechanism that allows the user to turn the data collection on and off?

Standard Telemetry mechanisms apply.

    If the request is for permanent data collection, is there someone who will monitor the data over time?**

N/A, expires in 67

    Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under? **

Type 1, technical

    Is the data collection request for default-on or default-off?

Default on.

    Does the instrumentation include the addition of any new identifiers (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)?

No.

    Is the data collection covered by the existing Firefox privacy notice?

Yes.

    Does there need to be a check-in in the future to determine whether to renew the data?

Yes. :lgreco, you are responsible for removing this collection when it is no longer useful.

---
Result: datareview+

Comment 8

[Shane Caraveo (:mixedpuppy)]

Comment on attachment 8986498 [details]
Bug 1465129 - Collect telemetry data related to the storage.local "JSONFile to IDBBackend" data migrations.

https://reviewboard.mozilla.org/r/251824/#review258304

A reasonable failure test case would be nice, but given the reason for not adding the existing failure test I'll pass on that.

Comment 9

[Chris H-C :chutten]

Comment on attachment 8986485 [details]
request-data-review-bug-1465129.md

Whoops, missed setting the flag.

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/35dd3b1bd10b
Collect telemetry data related to the storage.local "JSONFile to IDBBackend" data migrations. r=mixedpuppy

Comment 11

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/35dd3b1bd10b

Comment 12

[CosminB]

Attachment: Bug1465129.png

This issue is verified as fixed on Firefox 63.0a1 (20180627222831) and Firefox 62.0b3 (20180625141512) under Win 7 64-bit and Mac OS X 10.13.3.

Only the success scenario can be manually verified easily, the histogram WEBEXT_STORAGE_LOCAL_IDB_MIGRATE_RESULT_COUNT is the one used for counting.

Please see the attached screenshot.