Closed Bug 1340535 Opened 4 years ago Closed 3 years ago

Create a QA test-plan sending shutdown pings with the PingSender

Categories

(Toolkit :: Telemetry, defect, P1)

defect
Points:
2

Tracking

()

RESOLVED FIXED
Tracking Status
firefox54 --- affected

People

(Reporter: Dexter, Assigned: Abe_LV)

References

Details

(Whiteboard: [measurement:client])

We need to write a proper QA test plan for sending the shutdown ping with the PingSender when the browser shuts down.

Some of the things we should cover:

- A shutdown ping must be sent, either trough the PingSender or through the normal Telemetry flow.
- If a ping is successfully sent through the PingSender, it must be pruned from the pending pings directory (if not, we should list the exceptional cases).
Points: --- → 2
Depends on: 1336360
Priority: -- → P1
Whiteboard: [measurement:client]
Blocks: 1343277
Mark, Chris can you think of any other risk/test cases from comment 1? Can you think of anything we missed?
Assignee: nobody → amasresha
Flags: needinfo?(mreid)
Flags: needinfo?(chutten)
Do we need to put any time controls on this? (e.g. A shutdown ping must be sent by pingSender within X seconds of shutdown)

Should we put any integrity controls on this? (The shutdown ping sent by pingSender must be well-formed, compressed, etc. In any exceptional cases when more than one ping is sent, they must be identical in content...)
Flags: needinfo?(chutten)
We should make sure that the submission URL from pingSender vs. normal Telemetry are the same (ie. things external to the submitted payload).

pingSender should include the standard HTTP "Date" header.

Do we care if pingSender sends the same value for the DNT header?
Flags: needinfo?(mreid)
(In reply to Chris H-C :chutten from comment #3)
> Do we need to put any time controls on this? (e.g. A shutdown ping must be
> sent by pingSender within X seconds of shutdown)

I would personally stay away from time sensitive checks even for manual QA.
Moreover, the pingsender is not required to send pings within X seconds from the browser shutdown: if it does, good, if it doesn't that's still good!

> Should we put any integrity controls on this? (The shutdown ping sent by
> pingSender must be well-formed, compressed, etc. In any exceptional cases
> when more than one ping is sent, they must be identical in content...)

Yes, that's already mentioned in the test cases from comment 1 (schema validation,
identical content for duplicate pings, etc.).

Compression is not covered as it will be taken care of in a separate bug. We will update the test cases accordingly after it lands.

(In reply to Mark Reid [:mreid] from comment #4)
> We should make sure that the submission URL from pingSender vs. normal
> Telemetry are the same (ie. things external to the submitted payload).

Good point!

> pingSender should include the standard HTTP "Date" header.

Good point #2! Also, the pingsender wasn't sending the date header.
I just added it today!

> Do we care if pingSender sends the same value for the DNT header?

I don't think so.
@Abe, given the discussion in the previous comments, could we add the following:

- We should make sure that the submission URL from ping sender vs. normal Telemetry are the same.
- We should make sure that the submission from the ping sender have the following headers:

User-Agent: pingsender/1.0
X-PingSender-Version: 1.0
Date: <as described in https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Date )

@Georg, could you take one look at this as well?
Flags: needinfo?(gfritzsche)
Flags: needinfo?(amasresha)
Updated the test cases accordingly. The first one (URL should be identical) is already included in the existing test cases, but the second one (headers verification) is now updated. Thanks for the feedback.
Flags: needinfo?(amasresha)
We should be good to go with the test plan, we'll execute it once the shutdown pignsender change lands.
Status: NEW → RESOLVED
Closed: 3 years ago
Flags: needinfo?(gfritzsche)
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.