1482859 - Update Simple Snippet Template to block snippet after user action.

Status: REOPENED

(Firefox :: Messaging System, enhancement, P3)

Description

[Giorgos Logiotatidis [:giorgos]]

Update Simple Snippet Template to make it so

 - after the user has taken action, i.e. has clicked the link or button
 -> the snippet or any snippets on the same campaign don't appear again 

We can use blocking mechanism to achieve this result.

Also include another variable named "do_not_autoblock" of type "checkbox" that defaults to "false". When the variable is set to "true" the above mechanism must get over-ridden and the snippets will continue to appear even after action has been taken.

This behavior must be the default for all the upcoming snippet templates (video, fundraising etc)

Comment 1

[Giorgos Logiotatidis [:giorgos]]

Lisa can you please confirm that I've correctly captured the requirements for this feature?

Comment 2

[Lisa Wright [Mat Leave]]

@giorgos - Confirming all looks correct. 

To recap: We're looking to provide a better user experience for our snippets, and currently if someone clicks on our CTA prompt and takes the action we're asking them to take, they may still get served the same snippet again - and that doesn't make much sense. Instead, we'd like snippets to no longer appear for a user if they click the CTA link or button (similar to if someone blocks a snippet now).

And yes, this would be something we want to implement on all AS snippet templates.

Thanks!

Comment 3

[Lisa Wright [Mat Leave]]

I do, however, want to point back to a previous Issue that Jean filed a while back that explains an important nuance that needs to be considered here >> https://github.com/mozmeao/snippets-service/issues/368

Goal for AS templates, when a user

    - hits X on first scene
    - clicks on hyperlink or button on first scene (if Simple snippet or Video template)
    - clicks on "dismiss" button on second scene

They will not see any snippets ever again with the same Campaign field name (in the admin tool)

Because we have templates that include second scenes however, we don't want to consider the click on the button on the first scene for those as fully "taking the action" to make it that they don't see snippets under the same campaign name ever again. This is because the first scene button is for users to continue the experience in the snippet (it's not an actual conversion).

For example, the Newsletter Sign Up template, we want to stop showing these snippets to users that actually converted on the second scene button (not the first scene button because first button is for them to continue to see the next scene).

The templates that need logic update so that a click on hyperlink or button on first scene is NOT looped into the "do not show users these snippets under the same campaign name" group would be:
-AS FxA sign up / sign in
-AS Newsletter Sign Up
-AS Send to Device

Apologies for the late addition of this information.

Comment 4

[Giorgos Logiotatidis [:giorgos]]

Kate NI? you to make sure that you see this template requirement for Fx63

Comment 5

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/activity-stream/pull/4409

Comment 6

[[github robot]]

Commit pushed to master at https://github.com/mozilla/activity-stream

https://github.com/mozilla/activity-stream/commit/f0afce2b9cfb72b4d2aa5349bc675ccb704d79d3
Fix Bug 1482859 - Block snippet messages after CTA was clicked (#4409)

Comment 7

[Andrei Oprea [:andreio]]

The PR that landed fixes the issue for SimpleSnippet but I want to keep the bug open until we land all templates (video, newsletter etc) because they all require this feature.

Comment 8

[[github robot]]

Commit pushed to master at https://github.com/mozilla/activity-stream

https://github.com/mozilla/activity-stream/commit/8f9e4d089d2680e0142875d8442008cd955de98a
Bug 1482859 - Fix do not autoblock option for RichText

Comment 9

[Ed Lee :Mardak]

https://hg.mozilla.org/mozilla-central/rev/dc549720ed1f

Comment 10

[Andrei Oprea [:andreio]]

This feature has been added to all snippets we currently shipped 
* fxa bug 1492085
* newsletter bug 1492086
* eoy bug 1492089
* send to device bug 1492088
* and for simple snippet (added in this bug)

Comment 11

[Marius Coman [:mcoman], Ecosystem QA]

I have verified this issue with the latest "Firefox Beta" (64.0b4 Build ID 20181025233934) and latest "Firefox Nightly" (65.0a1 Build ID - 20181029100347) builds installed, on Windows 10 x64, Arch Linux and Mac 10.13.3, and I can confirm that the snippets are blocked in the AS Router, if the "X" button is clicked of if a link/button is clicked on first the scene of a snippet.

Comment 12

[Giorgos Logiotatidis [:giorgos]]

Marius can you please share documentation on how this can be tested?

Comment 13

[Giorgos Logiotatidis [:giorgos]]

(In reply to Marius Coman [:mcoman], Experiments QA from comment #11)
> I can confirm that the snippets are blocked in the AS Router,
> if the "X" button is clicked of if a link/button is clicked on first the
> scene of a snippet.

Comment #3 points out that we don't want Snippets to get blocked if a link / button is clicked on the first scene. Only on the second scene we get a conversion and that's when we want to block.

Comment 14

[Marius Coman [:mcoman], Ecosystem QA]

Hi Giorgos, we don't have a specific documentation, however, I will try to provide the necessary steps in order to test the snippets from the AS Router page:

1. Navigate to the "about:config" page and set the "browser.newtabpage.activity-stream.asrouter.devtoolsEnabled" pref to "true";
2. Navigate to the "about:newtab#asrouter" page.
3. Press the "Show" button next to the desired snippet in order to trigger it.

I hope this helps, but If you will need extra information please don't hesitate to ping me.

Comment 15

[Giorgos Logiotatidis [:giorgos]]

Hi Marius,

I want to understand and test myself the /blocking/ of snippets. 

In comment #11 you confirm that Snippets get blocked after clicking X. I want to know to you verified that the snippet got blocked and how to view the list of blocked snippets.

Thanks

Comment 16

[Giorgos Logiotatidis [:giorgos]]

(In reply to Giorgos Logiotatidis [:giorgos] from comment #13)
> (In reply to Marius Coman [:mcoman], Experiments QA from comment #11)
> > I can confirm that the snippets are blocked in the AS Router,
> > if the "X" button is clicked of if a link/button is clicked on first the
> > scene of a snippet.
> 
> Comment #3 points out that we don't want Snippets to get blocked if a link /
> button is clicked on the first scene. Only on the second scene we get a
> conversion and that's when we want to block.

Also Marius can you please confirm that this is broken based on c#3?

Comment 17

[Marius Coman [:mcoman], Ecosystem QA]

We have verified this only with the test snippets from the AS Router page because we don't have access to an active snippets campaign in order to test them. 
If you want to test if the snippets from the AS Router page are blocked you will need to trigger one by pressing its "Show" button and after the snippet's "x" button is clicked you can observe that the "Show" button from that snippet is no longer displayed and the "Unblock" button is displayed instead.

Also, I can confirm that the snippet is blocked if the "x" button is clicked in the snippet's first scene and considering the comment #3 this issue is still reproducible.

Comment 18

[Giorgos Logiotatidis [:giorgos]]

(In reply to Marius Coman [:mcoman], Experiments QA from comment #17)
> We have verified this only with the test snippets from the AS Router page
> because we don't have access to an active snippets campaign in order to test
> them. 

I can help you with real snippets content for testing. Please file an issue with your request at https://github.com/mozilla/snippets-service/


> If you want to test if the snippets from the AS Router page are blocked you
> will need to trigger one by pressing its "Show" button and after the
> snippet's "x" button is clicked you can observe that the "Show" button from
> that snippet is no longer displayed and the "Unblock" button is displayed
> instead.

OK that answers my question on how to test whether a snippet is blocked or not
 
> Also, I can confirm that the snippet is blocked if the "x" button is clicked
> in the snippet's first scene and considering the comment #3 this issue is
> still reproducible.


Not sure what you mean here. The question is:

 - Is the snippet blocked if a button or link /besides/ the X button, is clicked on the first scene? If so, then this is a bug.

Comment 19

[Giorgos Logiotatidis [:giorgos]]

For newsletter snippet verified that:

 - X button blocks the snippet
 - Going from scene 1 to scene 2 does not block the snippet
 - Pressing Dismiss on Scene 2 does not block the snippet
 - Successfully subscribing to a newsletter blocks the snippet
 - Subscription errors do not block the snippet
 - When do_no_autoblock is true snippet is not blocked on success.

Tested on 64.0b7

Thus this works fine for the newsletter snippet. 

I suggest that we create a testing playbook with similar tests for all snippets and test using that. This will also help the dev and QA teams to be in sync when things change, e.g. when we add variables that change logic such as do_not_autoblock.

From the snippets side I'm happy to provide code / docs / insights to build a solid testing infra and make this as simple as possible. 

Marius what do you think?

Comment 20

[Marius Coman [:mcoman], Ecosystem QA]

Hi Giorgos, for now, we have the following variants:

1. If a button/link is clicked on a snippet that has only one scene (e.g. the "SIMPLE_TEST_1", "SIMPLE_TEST_TALL" from the AS Router page) that snippet is blocked.
2. If a button/link is clicked on the first scene of a snippet that has two scenes (e.g. NEWSLETTER_TEST_1 ), the snippet is not blocked, also if the "Dismiss" button from the second scene is clicked, the snipped is not blocked either. The snippet is blocked only if the "X" is clicked regardless on which scene, or if the newsletter sign-up is completed.

I also have a question regarding the snippets with only one scene, because I think I misunderstood something there. If a button/link (except "X") is clicked on this type of snippets, the snippets should be blocked or not?

Also, I think that a testing playbook with general tests for all snippets is a great idea and it will be very useful for the QA part. If you want to discuss more on this subject we can also move the conversation on Slack.

Comment 21

[Giorgos Logiotatidis [:giorgos]]

(In reply to Marius Coman [:mcoman], Experiments QA from comment #20)
> I also have a question regarding the snippets with only one scene, because I
> think I misunderstood something there. If a button/link (except "X") is
> clicked on this type of snippets, the snippets should be blocked or not?

Yes they should be blocked. Your first variant is correct. Thanks!