1433241 - QA document/process for ASR snippets templates

Status: RESOLVED FIXED

(Firefox :: Messaging System, enhancement, P3)

Comment 1

[Kate Hudson :k88hudson]

This is something we would like to prioritize and work with Giorgos on

Comment 2

[Kate Hudson :k88hudson]

I've added the QA document as the User Story to this bug, which covers detailed instruction for setting up devtools, testing legacy snippets, and testing variables, visuals, and functionality for each template.

I still need to add instructions for testing telemetry.

Comment 3

[Kate Hudson :k88hudson]

A couple of questions as I was going through each template/variable:

- Is there a live test endpoint that always has data we can use for testing with the *legacy* snippets system that I can add to this doc?
- For the Firefox Accounts page, is there an sign-up page/some test data that is functioning we can use for testing? The test data I have seems to show up as an error
- Should source_url still be https://snippets.mozilla.com/show/{id}? Does that still work with the current system?


Thanks!

Comment 4

[Kate Hudson :k88hudson]

Just noting that while functionality should be up to spec there is still some ongoing discussion around some design details (such as button colours, new lines) that needs to be resolved in a meeting tomorrow (Wed Nov 21). I will provide an update when these issue shave been resolved.

Please let me or Andrei know if anything needs clarification/more detail. Thanks!

Comment 5

[Ciprian Muresan [:cmuresan], Ecosystem QA]

I'll take a look asap and add the scenarios to our TestRail suite.

Comment 6

[Giorgos Logiotatidis [:giorgos]]

(In reply to Kate Hudson :k88hudson from comment #3)
> A couple of questions as I was going through each template/variable:
> 
> - Is there a live test endpoint that always has data we can use for testing
> with the *legacy* snippets system that I can add to this doc?

No. We can send you links to snippet previews to check how things should look and feel now.

> - For the Firefox Accounts page, is there an sign-up page/some test data
> that is functioning we can use for testing? The test data I have seems to
> show up as an error

I don't understand what you mean here. I know that the FxA template lands you to FxA accounts page, but from there it's the FxA service, right?

> - Should source_url still be https://snippets.mozilla.com/show/{id}? Does
> that still work with the current system?

Not sure that has even been used, it's a Basket field.  

Deferring to Jean and Lisa to find out:

Jean, Lisa for the newsletter and the send to device templates we send to Basket a field named `source_url` which points back to the snippets service. Do you know whether that's used in any metrics / lead tracking? If so can we change the url from ` https://snippets.mozilla.com/show/{id}` to ` https://snippets.mozilla.com/`?

Comment 7

[Jean Collings]

CC'ing Jon to see if he can help answer the source_url question as he built those templates. I'm also tagging George in case he may know.

Jon any concerns or thoughts on if we change 'source_url' in the newsletter and send to device templates from https://snippets.mozilla.com/show/{id}` to ` https://snippets.mozilla.com/`?

Comment 8

[George Kaberere [:gkaberere]]

Unfortunately, I do not know anything about this.

Comment 9

[Jonathan Petto [:jpetto]]

From a technical standpoint, source_url is just passed on to basket and is associated with that user/subscription in Salesforce. AFAIK, it's only used for segmentation purposes.

I'm not exactly sure what the question is above.

If we're asking if we can change the value of source_url, the answer is yes, absolutely. It would only impact user segmentation. So, if we used the exact same value for every snippet, all snippet-based subscriptions would fall into the same bucket (as far as source_url is concerned). TBH, I'm not really familiar with *how* source_url is used after being stored.

However, if we're asking if the {id} part of `https://snippets.mozilla.com/show/{id}` is still translated into the ID of the actual snippet being seen (e.g. resulting in a value like `https://snippets.mozilla.com/show/12345`), the answer is also yes (at least based on the test URL provided by Jean [0] - which has a source_url value of 'https://snippets.mozilla.com/show/9630').

Hope that helps!


[0] https://snippets-admin.mozilla.org/show/uuid/95a5317c-c91e-4837-bedb-91cfc4e927ad/

Comment 10

[Marius Coman [:mcoman], Ecosystem QA]

Hi Kate, we went through the provided documentation for the "ASR Snippets" and we added all the scenarios from there to our test suite.
Could you please take a look over the test cases in order to verify that we covered all the scenarios?
You can find the test cases here: https://goo.gl/6cZqWZ.

Comment 11

[Giorgos Logiotatidis [:giorgos]]

Closing the loop here on `source_url`:

 - Please keep as it is today, that's `https://snippets.mozilla.com/show/{id}` 

I filed https://github.com/mozmeao/snippets-service/issues/865 to keep track of re-adding the view in snippets service.