1867548 - Add a telemetry for 4-cc-number case

Status: RESOLVED FIXED

(Toolkit :: Form Autofill, defect)

Comment 1

[Kagami Rosylight [:saschanaz] (they/them)]

Attachment: Bug 1867548 - Collect telemetry for multiple cc-number cases r=#credential-management-reviewers

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/58cf44486fdf
Collect telemetry for multiple cc-number cases r=credential-management-reviewers,dimi

Comment 3

[Iulian Moraru]

Backed out for causing bc failures on browser_creditCard_telemetry.js.

• Push with failures

• Failure log

• Backout link

[task 2023-12-07T11:03:19.470Z] 11:03:19     INFO - TEST-PASS | browser/extensions/formautofill/test/browser/creditCard/browser_creditCard_telemetry.js | Expected to have 1 event/s with the name "formDetected" - 1 == 1 - 
[task 2023-12-07T11:03:19.472Z] 11:03:19     INFO - Buffered messages finished
[task 2023-12-07T11:03:19.490Z] 11:03:19     INFO - TEST-UNEXPECTED-FAIL | browser/extensions/formautofill/test/browser/creditCard/browser_creditCard_telemetry.js | {"cc_exp_year":"autocomplete","cc_type":"autocomplete","cc_number_multi_parts":"1","cc_exp_month":"autocomplete","cc_name":"autocomplete","cc_number":"autocomplete","cc_exp":"undetected"} deepEqual {"cc_name":"autocomplete","cc_number":"autocomplete","cc_type":"autocomplete","cc_exp":"undetected","cc_exp_month":"autocomplete","cc_exp_year":"autocomplete"} - {"filename":"chrome://mochitests/content/browser/browser/extensions/formautofill/test/browser/creditCard/browser_creditCard_telemetry.js","name":"assertGleanTelemetry/<","sourceId":820,"lineNumber":50,"columnNumber":14,"sourceLine":"","asyncCause":null,"asyncCaller":null,"caller":{"filename":"chrome://mochitests/content/browser/browser/extensions/formautofill/test/browser/creditCard/browser_creditCard_telemetry.js","name":"assertGleanTelemetry","sourceId":820,"lineNumber":35,"columnNumber":10,"sourceLine":"","asyncCause":null,"asyncCaller":null,"caller":{"filename":"chrome://mochitests/content/browser/browser/extensions/formautofill/test/browser/creditCard/browser_creditCard_telemetry.js","name":"test_popup_opened","sourceId":820,"lineNumber":243,"columnNumber":9,"sourceLine":"","asyncCause":null,"asyncCaller":{"filename":"chrome://mochikit/content/browser-test.js","name":"handleTask","sourceId":556,"lineNumber":1138,"columnNumber":26,"sourceLine":"","asyncCause":null,"asyncCaller":null,"caller":{"filename":"chrome://mochikit/content/browser-test.js","name":"_runTaskBasedTest","sourceId":556,"lineNumber":1210,"columnNumber":18,"sourceLine":"","asyncCause":null,"asyncCaller":{"filename":"chrome://mochikit/content/browser-test.js","name":"Tester_execTest","sourceId":556,"lineNumber":1352,"columnNumber":14,"sourceLine":"","asyncCause":null,"asyncCaller":null,"caller":{"filename":"chrome://mochikit/content/browser-test.js","name":"nextTest/<","sourceId":556,"lineNumber":1127,"columnNumber":14,"sourceLine":"","asyncCause":null,"asyncCaller":null,"caller":{"filename":"chrome://mochikit/content/tests/SimpleTest/SimpleTest.js","name":"SimpleTest.waitForFocus/<","sourceId":581,"lineNumber":1058,"columnNumber":13,"sourceLine":"","asyncCause":null,"asyncCaller":null,"caller":null,"formattedStack":"SimpleTest.waitForFocus/<@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:1058:13\n","nativeSavedFrame":{}},"formattedStack":"nextTest/<@chrome://mochikit/content/browser-test.js:1127:14\nSimpleTest.waitForFocus/<@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:1058:13\n","nativeSavedFrame":{}},"formattedStack":"async*Tester_execTest@chrome://mochikit/content/browser-test.js:1352:14\nnextTest/<@chrome://mochikit/content/browser-test.js:1127:14\nSimpleTest.waitForFocus/<@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:1058:13\n","nativeSavedFrame":{}},"caller":null,"formattedStack":"_runTaskBasedTest@chrome://mochikit/content/browser-test.js:1210:18\nasync*Tester_execTest@chrome://mochikit/content/browser-test.js:1352:14\nnextTest/<@chrome://mochikit/content/browser-test.js:1127:14\nSimpleTest.waitForFocus/<@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:1058:13\n","nativeSavedFrame":{}},"formattedStack":"async*handleTask@chrome://mochikit/content/browser-test.js:1138:26\n_runTaskBasedTest@chrome://mochikit/content/browser-test.js:1210:18\nasync*Tester_execTest@chrome://mochikit/content/browser-test.js:1352:14\nnextTest/<@chrome://mochikit/content/browser-test.js:1127:14\nSimpleTest.waitForFocus/<@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:1058:13\n","nativeSavedFrame":{}},"caller":null,"formattedStack":"test_popup_opened@chrome://mochitests/content/browser/browser/extensions/formautofill/test/browser/creditCard/browser_creditCard_telemetry.js:243:9\nasync*handleTask@chrome://mochikit/content/browser-test.js:1138:26\n_runTaskBasedTest@chrome://mochikit/content/browser-test.js:1210:18\nasync*Tester_execTest@chrome://mochikit/content/browser-test.js:1352:14\nnextTest/<@chrome://mochikit/content/browser-test.js:1127:14\nSimpleTest.waitForFocus/<@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:1058:13\n","nativeSavedFrame":{}},"formattedStack":"assertGleanTelemetry@chrome://mochitests/content/browser/browser/extensions/formautofill/test/browser/creditCard/browser_creditCard_telemetry.js:35:10\ntest_popup_opened@chrome://mochitests/content/browser/browser/extensions/formautofill/test/browser/creditCard/browser_creditCard_telemetry.js:243:9\nasync*handleTask@chrome://mochikit/content/browser-test.js:1138:26\n_runTaskBasedTest@chrome://mochikit/content/browser-test.js:1210:18\nasync*Tester_execTest@chrome://mochikit/content/browser-test.js:1352:14\nnextTest/<@chrome://mochikit/content/browser-test.js:1127:14\nSimpleTest.waitForFocus/<@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:1058:13\n","nativeSavedFrame":{}},"formattedStack":"assertGleanTelemetry/<@chrome://mochitests/content/browser/browser/extensions/formautofill/test/browser/creditCard/browser_creditCard_telemetry.js:50:14\nassertGleanTelemetry@chrome://mochitests/content/browser/browser/extensions/formautofill/test/browser/creditCard/browser_creditCard_telemetry.js:35:10\ntest_popup_opened@chrome://mochitests/content/browser/browser/extensions/formautofill/test/browser/creditCard/browser_creditCard_telemetry.js:243:9\nasync*handleTask@chrome://mochikit/content/browser-test.js:1138:26\n_runTaskBasedTest@chrome://mochikit/content/browser-test.js:1210:18\nasync*Tester_execTest@chrome://mochikit/content/browser-test.js:1352:14\nnextTest/<@chrome://mochikit/content/browser-test.js:1127:14\nSimpleTest.waitForFocus/<@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:1058:13\n","nativeSavedFrame":{}}
[task 2023-12-07T11:03:19.493Z] 11:03:19     INFO - Stack trace:
[task 2023-12-07T11:03:19.494Z] 11:03:19     INFO - chrome://mochitests/content/browser/browser/extensions/formautofill/test/browser/creditCard/browser_creditCard_telemetry.js:assertGleanTelemetry/<:50
[task 2023-12-07T11:03:19.495Z] 11:03:19     INFO - chrome://mochitests/content/browser/browser/extensions/formautofill/test/browser/creditCard/browser_creditCard_telemetry.js:assertGleanTelemetry:35
[task 2023-12-07T11:03:19.495Z] 11:03:19     INFO - chrome://mochitests/content/browser/browser/extensions/formautofill/test/browser/creditCard/browser_creditCard_telemetry.js:test_popup_opened:243
[task 2023-12-07T11:03:19.496Z] 11:03:19     INFO - chrome://mochikit/content/browser-test.js:handleTask:1138
[task 2023-12-07T11:03:19.497Z] 11:03:19     INFO - chrome://mochikit/content/browser-test.js:_runTaskBasedTest:1210
[task 2023-12-07T11:03:19.498Z] 11:03:19     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest:1352
[task 2023-12-07T11:03:19.499Z] 11:03:19     INFO - chrome://mochikit/content/browser-test.js:nextTest/<:1127
[task 2023-12-07T11:03:19.500Z] 11:03:19     INFO - chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.waitForFocus/<:1058
[task 2023-12-07T11:03:19.502Z] 11:03:19     INFO - TEST-PASS | browser/extensions/formautofill/test/browser/creditCard/browser_creditCard_telemetry.js | Expected to have 1 event/s with the name "formPopupShown" - 1 == 1 - 

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2ffdc83cfcb5
Collect telemetry for multiple cc-number cases r=credential-management-reviewers,dimi

Comment 5

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/2ffdc83cfcb5

Comment 6

[Kagami Rosylight [:saschanaz] (they/them)]

Comment on attachment 9366549 [details]
Bug 1867548 - Collect telemetry for multiple cc-number cases r=#credential-management-reviewers

Oh no, I forgot to send data review request. It's better late than never:

Request for data collection review form

All questions are mandatory. You must receive review from a data steward peer on your responses to these questions before shipping new data collection.

1. What questions will you answer with this data?

How many websites out there have multiple autocomplete=cc-number input boxes in a form.

2. Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements?

This is part of the effort to standardize our behavior; we need to know how common it is. If it's common enough then we'll have to standardize the current Gecko behavior, if not we have some other options.

3. What alternative methods did you consider to answer these questions? Why were they not sufficient?

httparchive could be another option, but forms with credit card information are often not publicly accessible without signing in first, which makes httparchive not ideal as it's for public pages.

4. Can current instrumentation answer these questions?

We have no existing way to gather this data, no.

5. List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories found on the Mozilla wiki.

Note that the data steward reviewing your request will characterize your data collection based on the highest (and most sensitive) category.

<table>
<tr>
<td>Measurement Description</td>
<td>Data Collection Category</td>
<td>Tracking Bug #</td>
</tr>
<tr>
<td>How many cc-number field exists in a form</td>
<td>Interaction data</td>
<td>Bug 990203, bug 1834571 (per the existing metrics.yaml)</td>
</tr>
</table>

6. Please provide a link to the documentation for this data collection which describes the ultimate data set in a public, complete, and accurate way.

This collection is documented in the Glean Dictionary at https://dictionary.telemetry.mozilla.org/: https://dictionary.telemetry.mozilla.org/apps/firefox_desktop/metrics/formautofill_creditcards_form_detected

7. How long will this data be collected? Choose one of the following:

• I want to permanently monitor this data. (Kagami Rosylight)

8. What populations will you measure?

• Which release channels? All

• Which countries? All

• Which locales? All

• Any other filters? Please describe in detail below. (N/A)

9. If this data collection is default on, what is the opt-out mechanism for users?

The normal telemetry opt-out option for Firefox.

10. Please provide a general description of how you will analyze this data.

GLAM does not support showing graphs for events, I'll have to query SQL manually to how common this is.

11. Where do you intend to share the results of your analysis?

https://github.com/whatwg/html/issues/9927

12. Is there a third-party tool (i.e. not Glean or Telemetry) that you are proposing to use for this data collection?

No

Comment 7

[Chris H-C :chutten]

Attachment: data collection review request

In future please attach data review requests to the bug as plaintext attachments.

Comment 8

[Chris H-C :chutten]

Comment on attachment 9368271 [details]
data collection review request

PRELIMINARY NOTES:

Did you know that, since the collection is Glean, you can use ./mach data-review to help generate your data review request?

DATA COLLECTION REVIEW RESPONSE:

Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes.

Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. This collection can be controlled through the product's preferences.

If the request is for permanent data collection, is there someone who will monitor the data over time?

Yes, Kagami Rosylight is responsible.

Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2, Interaction.

Is the data collection request for default-on or default-off?

Default on for all channels.

Does the instrumentation include the addition of any new identifiers?

No.

Is the data collection covered by the existing Firefox privacy notice?

Yes.

Does the data collection use a third-party collection tool?

No.

---

Result: datareview+