1825549 - The client is enrolled in a malformed notification experiment and the “message-malformed” event is not emitted in the Glean viewer when "text" is blank or missing

Status: VERIFIED FIXED

(Firefox for Android :: Experimentation and Telemetry, defect)

Description

[Alice Varro [avarro], Ecosystem QA]

[Affected versions]:
-Firefox Beta 112 (local build set to Stage) (Build #2015940741), 4219b85b96+ GV: 112.0-20230313153916 AS: 97.2.0

• Firefox Nightly 113.0a1 (local build set to Stage) (Build #2015941677), 274454fada GV: 113.0a1 - 20230328163437, AS: 97.2.0

[Affected Platforms]:

• Android 13 - Samsung Galaxy Tab A8.
• Android 9 - Samsung Galaxy S8.
• Android 5.1 - Pixel XL API 22 (emulated device).

[Prerequisites]:

• Have a Nimbus experiment that contains a malformed JSON.(e.g.: Stage experiment).

[Steps to reproduce]:

1. Open the app using a telemetry command (e.g for Firefox Beta.: adb shell am start -n org.mozilla.firefox_beta/mozilla.telemetry.glean.debug.GleanDebugActivity --ez logPings true --es sendPing metrics --es debugViewTag <name>) and attempt to naturally enroll in the experiment from the prerequisites.
2. Close the app and open it again with the telemetry command (e.g for Firefox Beta.: adb shell am start -n org.mozilla.firefox_beta/mozilla.telemetry.glean.debug.GleanDebugActivity --ez logPings true --es sendPing metrics --es debugViewTag <name>).
3. Navigate to the Nimbus Experiments menu and verify the experiment’s branch checkmark.
4. Navigate to the Glean viewer (https://debug-ping-preview.firebaseapp.com/pings/<name>) and search the “message-malformed” ping through the events received.

[Expected result]:

• The client is enrolled in the malformed experiment and the “message-malformed” event is displayed in the Glean viewer.

[Actual result]:

• The client is enrolled in the malformed experiment and the “message-malformed” event is not displayed in the Glean viewer.

[Notes]:

• The issue is reproducible both when “text” is not set, and when “text”:”” is set. Whether “style”:”NOTIFICATION” is set does not affect the result.
• Attaching a screen recording of the issue here.

Comment 1

[James Hugman [:jhugman] [@jhugman]]

[Expected result] should read:

The client is enrolled in the experiment, but the message-malformed event is displayed in the GleanViewer.

Suggest changing the title of this to: "message-malformed" event not emitted when text is blank or missing.

Comment 2

[Alice Varro [avarro], Ecosystem QA]

Attachment: Empty Notification

Hey, thanks for pinging me!

I changed the Expected result according to your suggestion. However, an empty notification is displayed when the “text” is missing and we think that this should be taken into consideration also.
Attaching a screenshot of the issue:

Comment 3

[James Hugman [:jhugman] [@jhugman]]

Agreed, the message should not be displayed, but we can't unenroll from the experiment based on the malformed message.

Comment 4

[Alice Varro [avarro], Ecosystem QA]

We did not expect to be unenrolled from the experiment, we expected to not be able to enroll in a malformed experiment at all in the first place, but it's possible that we misunderstood that.

Comment 5

[James Hugman [:jhugman] [@jhugman]]

Aha. Sorry for me not being clear: the expected behaviour would be that the experiment is enrolled, but the message is rejected as malformed.

Current behaviour (IIUC) is that the experiment is enrolled, but the message is displayed, and the message_malformed is not emitted.

Comment 6

[BMO Github Automation]

Attachment: [mozilla-mobile/firefox-android] Bug 1825549 Disallow messages with blank text fields (#1520)

Comment 7

[BMO Github Automation]

Authored by https://github.com/jhugman
https://github.com/mozilla-mobile/firefox-android/commit/7571f2ec9c35f0208609a86305a0cebd66e62dcf
[main] Bug 1825549 Disallow messages with blank text fields

Comment 8

[Alice Varro [avarro], Ecosystem QA]

I have verified this issue using Firefox Nightly 113.0a1 (Build set to Stage) (Build #2015942845), 16a80621ad+, GV: 113.0a1 - 30230403215207, AS: 97.4.0 using Pixel XL API 22 (Android 5.1) and I can confirm the following:

• The empty notification no longer appears if the text is missing or the fields are blank.
• A "malformed" event is displayed in the Glean viewer when the client enrolled in the experiment.