Remove LMDB-to-safe-mode migration from Glean
Categories
(Data Platform and Tools :: Glean: SDK, defect, P2)
Tracking
(Not tracked)
People
(Reporter: janerik, Assigned: janerik)
References
(Blocks 2 open bugs)
Details
Attachments
(3 files)
As per the proposal we plan to fully remove LMDB mode and its migration code from Glean.
This requires looking at Firefox/Focus for iOS and the version uptake to ensure clients have been migrated.
|
Assignee | |
Comment 1•2 years ago
|
||
By Q4 we should look at the rollout.
Let's use the bugzilla reminder bot functionality!
|
|
Assignee | |
Updated•2 years ago
|
|
||
Comment 2•2 years ago
|
||
(In reply to Jan-Erik Rediger [:janerik] from comment #1)
By Q4 we should look at the rollout.
Let's use the bugzilla reminder bot functionality!
I think you put a past date? In any case: Any news here?
|
|
Assignee | |
Comment 3•2 years ago
|
||
(In reply to Jens Stutte [:jstutte] from comment #2)
(In reply to Jan-Erik Rediger [:janerik] from comment #1)
By Q4 we should look at the rollout.
Let's use the bugzilla reminder bot functionality!I think you put a past date? In any case: Any news here?
D'oh!
I'll put in on my list for this week.
|
|
Assignee | |
Comment 4•2 years ago
|
||
As per https://sql.telemetry.mozilla.org/queries/86949/source we're at:
- 91.7% for Firefox iOS
- 92.7% for Focus iOS
- 93.2% for Klar iOS
This is (approx.) distinct clients on a Glean v51 or higher, the one that comes with the LMDB->safe-mode migration.
That's less than I hoped.
We currently have an ongoing data incident around Firefox iOS. While we're still investigating the exact cause we don't want to switch out the storage.
I can line up the changes necessary for Glean to do so though.
|
||
Comment 5•2 years ago
|
||
|
|
||
Comment 6•2 years ago
|
||
Can you map what "Glean v51" means in terms of released versions? Just to see, if those are old, unsupported versions, by chance.
|
|
Assignee | |
Comment 7•2 years ago
|
||
Glean v51.0.1 landed in Firefox iOS v104.1.
|
|
||
Comment 8•2 years ago
|
||
(In reply to Jan-Erik Rediger [:janerik] from comment #7)
Glean v51.0.1 landed in Firefox iOS v104.1.
Thanks, I just realized that was bug 1772148. That's just a few releases ago, so I am not too surprised to see older versions still around. What would be the consequence if we remove the upgrade path from LMDB say in 109 and someone upgrades directly from 103 to 109 ? We might just loose some telemetry from the former version? That sounds not too bad, IMHO, but I am not the expert, of course.
|
||
Comment 9•2 years ago
|
||
(In reply to Jens Stutte [:jstutte] from comment #8)
(In reply to Jan-Erik Rediger [:janerik] from comment #7)
Glean v51.0.1 landed in Firefox iOS v104.1.
Thanks, I just realized that was bug 1772148. That's just a few releases ago, so I am not too surprised to see older versions still around. What would be the consequence if we remove the upgrade path from LMDB say in 109 and someone upgrades directly from 103 to 109 ? We might just loose some telemetry from the former version? That sounds not too bad, IMHO, but I am not the expert, of course.
FWIW, intentionally deciding to drop support for these older versions (= potentially resetting their data if they upgrade) would require Firefox iOS, Focus/Klar iOS PM approval.
|
|
Assignee | |
Comment 10•2 years ago
|
||
As discussed in the document:
If a client skips from 103 to 109 it will lose its old data, including the client id. This client will show up as a new one and we will lose the "old" one. This has effects on measured retention. Seeing that for 8% is non-negligible[^1]. I have a meeting with iOS folks later anyway, I'll bring this up.
[^1]: But that's of course only if we would switch today. We will have a couple more weeks/months before this lands in Firefox iOS anyway, so that number will only go down.
|
|
||
Comment 11•2 years ago
|
||
(In reply to Alessio Placitelli [:Dexter] from comment #9)
FWIW, intentionally deciding to drop support for these older versions (= potentially resetting their data if they upgrade) would require Firefox iOS, Focus/Klar iOS PM approval.
We are now at:
- 92.69% for Firefox iOS
- 94.16% for Focus iOS
- 93.07% for Klar iOS (slightly lower?)
Given that we will never reach 100% - is there a known threshold we want to reach?
|
|
Assignee | |
Comment 12•2 years ago
•
|
||
I was preoccupied with an incident on iOS that already caused data churn. The release that hopefully fixes it went out yesterday.
I will talk to product and get their approval for this next.
fwiw, this will need some work on rkv anyway to rip out the LMDB parts. That work can get started in parallel.
|
|
||
Comment 13•2 years ago
|
||
(In reply to Jan-Erik Rediger [:janerik] from comment #12)
I was preoccupied with an incident on iOS that already caused data churn. The release that hopefully fixes it went out yesterday.
I will talk to product and get their approval for this next.fwiw, this will need some work on rkv anyway to rip out the LMDB parts. That work can get started in parallel.
Yeah, this will not happen before January (but that's close).
|
|
||
Updated•2 years ago
|
|
|
Assignee | |
Comment 15•1 year ago
|
||
No.
Numbers are of course getting higher now.
I remember there was some talk about improving rkv back then, but nothing yet materialized from that.
I can prepare a decision brief for product folks to sign off on, but won't get to that before next week.
Regardless the rkv work still needs to happen as well and afaik that hasn't started either.
|
||
Comment 16•5 months ago
|
||
|
|
Assignee | |
Comment 17•5 months ago
|
||
Updated numbers as of today:
- Firefox iOS: 98.9%
- Focus iOS: 98.2%
- Klar iOS: 97.3% (there's barely 1k clients though)
I'll prepare comms.
|
|
Assignee | |
Comment 18•2 months ago
|
||
|
|
Assignee | |
Comment 19•1 month ago
|
||
Description
•