1502849 - Integration test for search counts across sessions

Status: RESOLVED FIXED

(Toolkit :: Telemetry, enhancement, P1)

Description

[Raphael Aurich [:raphael] UTC+01:00]

Planned test behavior:

Session S1, subsession 1:
- Open browser
- Open new tab
- Perform search (awesome bar or search bar)
- Restart browser in new session

Session S2, subsession 1:
- Outcome 1
  - Received a main ping P1 for previous session 
  - Ping base contents:
    - clientId should be set
    - sessionId should be set
    - subsessionId should be set
    - previousSessionId should not be set
    - previousSubsessionId should not be set
    - subSessionCounter should be 1
    - profileSubSessionCounter should be 1
    - reason should be "shutdown"
  - Other ping contents:
    - SEARCH_COUNTS values should match performed search action
- Install addon
  - Verify that there should be no listing for tab scalar as we started a new
    session

Session S2, subsession 2:
- Outcome 2
  - Received a main ping P2 for previous subsession 
  - Ping base contents:
    - clientId should be set to the same value
    - sessionId should be set to a new value
    - subsessionId should be set to a new value
    - previousSessionId should be set to P1s sessionId value
    - previousSubsessionId should be set to P1s subsessionId value
    - subSessionCounter should be 1
    - profileSubSessionCounter should be 2
    - reason should be "environment-change"
   - Other ping contents:
     - SEARCH_COUNTS values should not be in P2
- Perform Search 
- Restart Browser

Session S3, subsession 1:
- Outcome 3
  - Received a main ping P3 for session 2, subsession 1
  - Ping base contents:
    - clientId should be set to the same value
    - sessionId should be set to P2s sessionId value
    - subsessionId should be set to a new value
    - previousSessionId should be set to P2s sessionId value
    - previousSubsessionId should be set to P2s subsessionId value
    - subSessionCounter should be 2
    - profileSubSessionCounter should be 3
    - reason should be "shutdown"
  - Other ping contents:
    - SEARCH_COUNTS values should be set per above search

Comment 1

[Raphael Aurich [:raphael] UTC+01:00]

Update: Check "browser.engagement.tab_open_event_count" scalar in "environment-change" ping after installing the addon.

Comment 2

[Raphael Aurich [:raphael] UTC+01:00]

Attachment: Bug 1502849 - Create reusable ping filters for telemetry-tests-client harness; r?dexter,davehunt

Comment 3

[Raphael Aurich [:raphael] UTC+01:00]

Attachment: Bug 1502849 - Add new telemetry client test for SEARCH_COUNTS across sessions; r?dexter,gfritzsche

Depends on D12288

Comment 4

[Raphael Aurich [:raphael] UTC+01:00]

Attachment: Bug 1502849 - Remove unused addon from telemetry-tests-client resources; r?dexter

Depends on D12289

Comment 5

[Raphael Aurich [:raphael] UTC+01:00]

Attachment: Bug 1502849 - Add SEARCH_COUNTS telemetry client test to telemetry suite; r?dexter

Depends on D12420

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e786fe6c9aa1
Create reusable ping filters for telemetry-tests-client harness; r=Dexter,davehunt
https://hg.mozilla.org/integration/autoland/rev/a92340f21853
Add new telemetry client test for SEARCH_COUNTS across sessions; r=Dexter,gfritzsche
https://hg.mozilla.org/integration/autoland/rev/8c3c5e856c83
Remove unused addon from telemetry-tests-client resources; r=Dexter

Comment 7

[Raphael Aurich [:raphael] UTC+01:00]

We will hold off from landing D12421 until we've sorted out the intermittent errors in firefox_puppeteer with closing tabs.

Comment 8

[Daniel Varga [:dvarga]]

https://hg.mozilla.org/mozilla-central/rev/e786fe6c9aa1
https://hg.mozilla.org/mozilla-central/rev/a92340f21853
https://hg.mozilla.org/mozilla-central/rev/8c3c5e856c83