Closed Bug 1519022 Opened 3 years ago Closed 3 years ago

Telemetry event data is not shown in archived telemetry

Categories

(Toolkit :: Telemetry, defect, P1)

Desktop
All
defect
Points:
2

Tracking

()

VERIFIED FIXED
mozilla66
Tracking Status
firefox64 --- wontfix
firefox65 --- wontfix
firefox66 --- verified

People

(Reporter: danibodea, Assigned: janerik)

References

Details

Attachments

(7 files)

[Description]:

  • When the user publishes a recipe, opens the browser with the correct preconditions and confirms the experiment action has been made, the actions made can be seen in the about:telemetry page (Events section, dynamic events shown). Furthermore, if the user restarts the browser again, he will not find the previously shown data in the about:telemetry page (Archived ping data, Events section is not being displayed).

[Affected versions]:

  • Nightly v66.0a1
  • Beta v65.0b9
  • Release v64.0.2

[Affected platforms]:

  • Tested in Windows, most probably all

[Steps to reproduce]:

  1. Create, approve and run a recipe.
  2. Open the browser with the correct preconditions.
  3. Go to about:telemetry page and notice that the experiment actions are documented.
  4. Restart the browser.

[Expected result]:

  • The experiment actions can be checked in the archive of about:telemetry page.

[Actual result]:

  • The preference experiment actions are not archived.
Component: Normandy Client → Telemetry
Product: Firefox → Toolkit
Summary: Telemetry data is not archived → Telemetry event data is not shown in archived telemetry

When you are checking the archived results, which archived ping are you checking? There should be an "event" ping with reason "shutdown".

Unfortunately rich views of non-"main" pings aren't supported by about:telemetry at this time so there will not be any "Events" section in the display. You will instead have to look at the Raw Payload to see the sent events (and the "reason").

Flags: needinfo?(daniel.bodea)

I was looking for the archived results here:

Ping data source: Archived ping data
Ping Type: event
Ping: <date and time of the event at hand>
There is no "Event" section/tab displayed on the left side menu.

If I go to the Raw Payload section/tab, I should see something like:
{
"reason": "shutdown",
"processStartTimestamp": 1547192340000,
"sessionId": "6f53ec85-bfe5-412c-96ee-8aaa1bd7b3fe",
"subsessionId": "92bb36db-58e7-495d-ab3a-5c546d0413a5",
"lostEventsCount": 0,
"events": {
"dynamic": [
[
3703,
"normandy",
"enroll",
"preference_study",
"EXP_NAME1",
{
"experimentType": "exp",
"branch": "nondefault"
}
]
]
}
}

At least now I know what and where should be. I thought this could be OK, but it does not always work. I could not find a reproduction of why it appeared once and not again (tried a few times). Ideas?

Flags: needinfo?(daniel.bodea) → needinfo?(chutten)

What do you mean by "does not always work"? What part isn't reliably working?

Is it the "Event" section that doesn't show for Archived event pings? (that isn't expected to work since we haven't written the code to render events for archived pings. We can file that bug if you would find it helpful)

Or is it that the normandy enroll events don't always show in the event ping? (Event pings are sent fairly frequently, so it could be that an event ping with reason "periodic" was sent earlier and had the events you're looking for).

Flags: needinfo?(chutten) → needinfo?(daniel.bodea)

(In reply to Chris H-C :chutten from comment #3)

What do you mean by "does not always work"? What part isn't reliably working?

I meant to say that the Raw Payload section from Archived ping data does not always show the Normandy event like the one in comment 2.

Is it the "Event" section that doesn't show for Archived event pings? (that isn't expected to work since we haven't written the code to render events for archived pings. We can file that bug if you would find it helpful)

I am having a hard time checking the archived Normandy events and this would really help us with our tests regarding Normandy and other features also. Implementing the Event section for the Archived ping data source would really help, in fact, this bug could be used for this purpose.

Or is it that the normandy enroll events don't always show in the event ping? (Event pings are sent fairly frequently, so it could be that an event ping with reason "periodic" was sent earlier and had the events you're looking for).
If there was an experiment to which the client was enrolled, the event would appear in the "Current Ping data" source, but then, if the browser was restarted, I should be seeing the same event in the Raw Payload section from the Archived ping data, right?

Flags: needinfo?(daniel.bodea)
Flags: needinfo?(chutten)

(In reply to Bodea Daniel [:danibodea] from comment #4)

If there was an experiment to which the client was enrolled, the event would appear in the "Current Ping data" source, but then, if the browser was restarted, I should be seeing the same event in the Raw Payload section from the Archived ping data, right?

That is correct. But finding the correct ping that would have that event might be tricky depending on whether or not an event ping was sent between checking it in the "Current Ping data" source and when the "event" ping with reason "shutdown" was sent.

Generally speaking we make no guarantee that a particular event shows up in a particular ping. We do guarantee (or at least try to) that each event ends up in a ping, at some point. So you may need to check a couple of them (this can be most quickly done by filtering to "event" pings in the Archived ping data dropdown, selecting Raw Payload, then using the "<<" and ">>" arrows to move through the history of archived event pings)

When doing this is there still a problem where the expected events do not show?

Flags: needinfo?(chutten)

Yes. Even then, some events could not be found.

Alrighty, we'll leave this for triage.

I'm a bit confused by comment 3: in version 60.0a1 20180208220029 the about:telemetry#events-tab is shown for archived ping data, when looking after a Shield-Normandy pref-experiment telemetry data. I'm adding an archived profile to exemplify that somehow Events were present in archived pings. (in archived ping data look for "You are looking at the 1/14/19, 11:09:11 AM, main ping.")

One hypothesis might be that Normandy was still a system add-on in Fx60 and not a Firefox component, so it might be that Shield-Normandy made funky stuff that would keep the Events in archived pings?

Flags: needinfo?(chutten)

Ah, I might be able to explain. "main" pings should not contain events after Firefox 62 because that is when the code for the event-specific "event" ping landed in bug 1460595.

If you are looking at an older build, events are carried in "main" pings which may have an Events section.

I'm sorry for the confusion.

Flags: needinfo?(chutten)

To avoid confusions, this bug is intended to address the implementation of the Events section of the Archive ping data, as the bug's title and description suggest.

Assignee: nobody → jrediger
Points: --- → 2
Priority: -- → P1
Pushed by jrediger@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/5b7087dac6b9
Display rich data for archived event pings r=chutten
Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla66

I confirm that the event pings are now properly displayed in the Event section from Archived data ping data, Ping type: event, dynamic. This is confirmed on Windows 10, Mac OS 10.14.2 and Ubuntu 16.04.

Status: RESOLVED → VERIFIED

It appears that the issue is not properly fixed OR it is corrupted by the fix of bug 1520210.

While verifying the fix of bug 1520210, I have found that the Events section (from the Archived Ping Data, ping type: event) is not being displayed in all situations. On my profile, I have 5 different events, each one with a different shutdown event in Raw Payload, but only 2 of them get displayed in a "Events" section (as requested to be implemented in this bug). Please see the following attachment for more details.

Also, The Events section would not appear if Ping Type: main is selected. I believe it should.

Flags: needinfo?(jrediger)
See Also: → 1520210

It appears that if only Normandy events/pings are being displayed (in Raw Payload), then an Event section is not being created, but if another type of event/ping is displayed (in Raw Payload), then an Event section is being created and displayed as intended.

Thanks, I'll take another look into why the events are not showing in some cases correctly.

to answer your second question:

Also, The Events section would not appear if Ping Type: main is selected. I believe it should.

The main ping does not have events anymore, that's why it is not shown anymore (the display of "current ping" is a bit misleading as it shows all current data, which later gets split across multiple different pings, e.g. main and event ping)

Keeping the ni? so I don't forget to take a look

Can you give me the full ping JSON of the ping where the Events section is not shown? (Use the "Raw JSON" button on the bottom)

Flags: needinfo?(jrediger) → needinfo?(daniel.bodea)

the first of 4 consecutive full ping json files; this one has Event section.

Flags: needinfo?(daniel.bodea)

the second of 4 consecutive full ping json files; this one has Event section.

the third of 4 consecutive full ping json files; this DOES NOT have Event section.

the forth of 4 consecutive full ping json files; this one DOES NOT have Event section.

Blocks: 1525521
You need to log in before you can comment on or make changes to this bug.