For the next iteration of onboarding, we will focus on highlighting features. We will have a control (no onboarding), "expanded" firstrun (from the current 48), and feature-based firstrun (this bug).
Hi all, I will take this feature as a QA. Here is the Test plan based on which the testing is made: https://wiki.mozilla.org/QA/Fennec/Feature-focused_first-run_experiment
Chenxia, Where would I find cohorts for this via re:dash? I assume looking for experiment "onboarding3-%". https://sql.telemetry.mozilla.org/queries/598/source I checked, there is no onboarding3-a, is that intentionally? is it because onboarding3-a is the one that doesn't have an onboarding slide. If so, we would still need to identify those though, no? I don't think doing a "not like" will be giving us the results we want for A, especially since no slide was supposed to go out to only a little amount of users. https://sql.telemetry.mozilla.org/queries/329/source
Hmm, running a few queries on my own, this does seem to be the case. I checked the code, and we should be sending 3-a in the experiments json, but for some reason that's not showing up. I also checked the submission dates, and the most recent submissions are from yesterday, so that should also be fine. One thing that seemed odd to me was that some of the json arrays have only a single non-onboarding item - the experiments array should always have an onboarding item, unless the telemetry ping is being sent before (which could be the case, but is still odd). https://sql.telemetry.mozilla.org/queries/603/source I'm still able to see the "no first run" case, so I'm going to make sure that the experiments json array is being formed correctly, and that the onboarding pref is being written correctly.