1649533 - Create new dashboard for What's New Panel message

Status: RESOLVED FIXED

(Firefox :: Messaging System, task, P1)

Description

[Nan Jiang [:nanj]]

See https://jira.mozilla.com/browse/DS-802 for more details.

Comment 1

[Su-Young Hong]

Hey Nan,

I took at look at the WNPanel data, and I had some questions. So the WNPanel impression (badge on the giftbox) seems to be identified by a bucket_id / message_id of WHATS_NEW_BADGE_<ver>, like:

WHATS_NEW_BADGE_76
WHATS_NEW_BADGE_72
WHATS_NEW_BADGE_75
WHATS_NEW_BADGE_74
WHATS_NEW_BADGE_78

Those seem pretty straight-foward. But there's some impressions that have the "WHATS_NEW_BADGE" string in them that don't fit that format: WHATS_NEW_BADGE_MOBILE_71. Meridel mentioned that we run WNPanels tied both to releases, as well as not tied to releases. Is there a list of all WNPanel messages we've run (both tied to releases as well as custom ones), with the corresponding impression IDs?

Also, for the "open message" impression pings (when the message is displayed due to user clicking the badge), it seems to be formatted as WHATS_NEW_<a>,WHATS_NEW<b>,<etc>, but it's unclear which of these are tied to which what's new badges. Like I'm seeing the following data:

WHATS_NEW_LOCKWISE_PASSWORDS_76,WHATS_NEW_PIP_76,WHATS_NEW_VULNERABLE_PASSWORDS_76
	11,135,988	
WHATS_NEW_FINGERPRINTER_COUNTER_ALT,WHATS_NEW_PERMISSION_PROMPT_72,WHATS_NEW_PIP_72
	10,898,290	
WHATS_NEW_AWESOMEBAR_75,WHATS_NEW_FINGERPRINTER_COUNTER_ALT,WHATS_NEW_PIP_72
	6,097,566	
WHATS_NEW_AWESOMEBAR_75,WHATS_NEW_FINGERPRINTER_COUNTER_72,WHATS_NEW_PIP_72
	4,960,354	
WHATS_NEW_FINGERPRINTER_COUNTER_72,WHATS_NEW_PERMISSION_PROMPT_72,WHATS_NEW_PIP_72
	2,914,750

How do we know which of these impressions correspond to which badge impressions?

Is there a full list we'd need to cross reference to be sure?

Comment 2

[Andrei Oprea [:andreio]]

I think I can help answer that. We used the convention to keep the Firefox version in the name of the message for when it was launched. For example the *_76 messages went out in 76.
Sometimes we kept older messages from the previous release around (WHATS_NEW_AWESOMEBAR_75 with the *_72) that's why you see the mix.

The other exception (from what you posted) is WHATS_NEW_FINGERPRINTER_COUNTER_ALT which was a variant of WHATS_NEW_FINGERPRINTER_COUNTER_72 (the panel only showed one or the other) that's why you have 2 combinations of triplets.

The MOBILE messages are different from the rest because we shipped mid release. All others were shipped the day of the release.

Short version is: max([message_id_number]) should be a good indication of when those were seen. Eg 75, COUNTER_ALT, 72 went out in 75.
The Panel meta bug https://bugzilla.mozilla.org/show_bug.cgi?id=1597705 links to all the individual message deployments (in the tasks section).
Let me know if there are any others that don't fit the pattern and I can help to identify them.

Comment 3

[Nan Jiang [:nanj]]

Thanks for your inputs, Andrei!

For the other two questions:

How do we know which of these impressions correspond to which badge impressions?

There are two different ways for the users to access the whats-new-panel, and they will report impressions slightly differently.

• Through the whats new badge on the browser toolbar, the message_id&bucket_id would be just the badge id, such as "WHATS_NEW_BADGE_72". In addition, if one clicks on that badge, it also reports an impression ping for the panel as a comma-separated string of all the messages in the panel, e.g. WHATS_NEW_FINGERPRINTER_COUNTER_72,WHATS_NEW_PERMISSION_PROMPT_72,WHATS_NEW_PIP_72
• Through the "What's New" from the hamburger menu, in this case, it only reports a panel impression, e.g. WHATS_NEW_FINGERPRINTER_COUNTER_72,WHATS_NEW_PERMISSION_PROMPT_72,WHATS_NEW_PIP_72

Is there a full list we'd need to cross reference to be sure?

You can reference this doc to see all the messages back to Firefox 72.

Comment 4

[Nan Jiang [:nanj]]

The dashboard was created here.