Closed Bug 1282968 Opened 5 years ago Closed 5 years ago

Redash android_events_v1.experiments missing onboarding3-a

Categories

(Firefox for Android :: General, defect)

ARM
Android
defect
Not set
normal

Tracking

()

RESOLVED WONTFIX
Tracking Status
firefox49 --- affected
firefox50 --- affected

People

(Reporter: liuche, Assigned: liuche)

References

(Blocks 1 open bug)

Details

Attachments

(2 files)

From the discussion in bug 1268641, Barbara was running some Redash queries and wasn't getting any onboarding3-a events so I was investigating:
https://bugzilla.mozilla.org/show_bug.cgi?id=1268641#c2

Locally I'm seeing onboarding3-a in core pings uploaded to my gzipserver (see attachment), but they don't seem to be present in Redash. mcomella mentions that the telemetry payloads are processed before being dumped into Redash, so I'm wondering if the 'experiments' json array is being truncated or mangled somehow.

For what I tried:
Running some queries myself, the 'experiments' json array seemed suspiciously short [1] because I expected every json array to at least include an onboarding event [2], so I set up the local gzip server [3] and was able to see the onboarding3-a experiment. This is code that merged to m-c 5/17 and there haven't been changes to the telemetry code, so this should be present in the Redash database.

[1] https://sql.telemetry.mozilla.org/queries/603/source
[2] https://dxr.mozilla.org/mozilla-central/source/mobile/android/base/java/org/mozilla/gecko/telemetry/pingbuilders/TelemetryCorePingBuilder.java#91
[3] https://wiki.mozilla.org/Mobile/Fennec/Android/Java_telemetry
Georg, I'm seeing some inconsistencies between a local core ping payload and queries from in Redash - do you know about how the core ping data is processed and dumped into Redash, and if that might explain the discrepancies I'm seeing in the 'experiments' json array?
Flags: needinfo?(gfritzsche)
Flags: needinfo?(kparlante)
Talked to mfinkle about this.

Core ping includes all possible experiments, not just the active ones, and the json array is pulled directly from the ping [1]. However, the active experiments shown in android_events_v1 are the active experiments (e.g., ones where 1) a session is active and 2) an event occurs that is tagged with that session).

Looks like all we need to do for this bug is send an event instead of just starting and stopping the session.

[1] https://github.com/mozilla-services/switchboard-experiments
Flags: needinfo?(kparlante)
Flags: needinfo?(gfritzsche)
Assignee: nobody → liuche
(In reply to Chenxia Liu [:liuche] from comment #2)
> Talked to mfinkle about this.
> 
> Core ping includes all possible experiments, not just the active ones, and
> the json array is pulled directly from the ping [1]. However, the active
> experiments shown in android_events_v1 are the active experiments (e.g.,
> ones where 1) a session is active and 2) an event occurs that is tagged with
> that session).
> 
> Looks like all we need to do for this bug is send an event instead of just
> starting and stopping the session.
> 
> [1] https://github.com/mozilla-services/switchboard-experiments

Thanks for looking into this Chenxia, is this you who can setup to send the event?
Flags: needinfo?(liuche)
Oppps I guess you did already looking at comment 4, nevermind
Comment on attachment 8767320 [details]
Bug 1282968 - Redash android_events_v1.experiments missing onboarding3-a.

https://reviewboard.mozilla.org/r/61702/#review59274

Given your analysis, this looks like it'd do the trick.

That being said:
> However, the active experiments shown in android_events_v1 are the active experiments (e.g., ones where 1) a session is active

Makes sense.
> and 2) an event occurs that is tagged with that session).

This seems unexpected to me – I'd expect that we could identify active experiments by client ID. Should we modify the behavior? Is this an issue with the data itself (e.g. in Spark) or with the translation to re:dash?

At the very least, we should consider:

* Adding a comment to the code to explain why these telemetry events are there so they're not removed later – you can avoid comment duplication by having each experiment call out to a single method, which has the comment instead.
* Updating the [re:dash guide](https://wiki.mozilla.org/Mobile/Metrics/Redash#Data_Tables_.2F_Schemas) to explain this behavior
* Announcing this behavior more broadly (e.g. mailing list, things I learned this week Friday meeting)
Attachment #8767320 - Flags: review?(michael.l.comella) → review+
Flags: needinfo?(liuche)
Barbara: actually, talking to mcomella, I had the realization that the reason the onboarding isn't showing up is because the query is for events, and we don't send any onboarding3-a events. Could you use mobile_clients_v1 instead? That's the opt-out client data. I see 3-a present on the query I'm running below. I think that's a better solution to adding the event probe, and that means we don't need to uplift anything.

https://sql.telemetry.mozilla.org/queries/603/source
Flags: needinfo?(bbermes)
Actually, good catch, this seems to work! Thanks.
Flags: needinfo?(bbermes)
We don't actually need to add onboarding to android_events_v1 because they are already present in mobile_clients_v1.
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.