1837161 - Dynamic Form Support

Status: RESOLVED FIXED

(Toolkit :: Form Autofill, enhancement, P2)

Comment 1

[Janika Neuberger [:janika]]

Attachment: Bug 1837161 - P1. Detect dynamic form changes r=#credential-management-reviewers,dimi

Sets up two observer types in order to detect dynamic form changes:
1. MutationObserver for an address/cc form (document if form-less),
observing node additions or removals in the form's node subtree
2. IntersectionObserver for each invisible elements in an address/cc form (document if form-less),
observing intersections (threshold 1) between the invisible element and the form/document

A "form-changed" event gets dispatched in the observers' callback functions when the fields that were added/removed (MutationObserver)
or became visible (IntersectionObserver) are of address or cc type. When the FormAutofillChild receives a "form-change" event it
compares the previous formDetails with the formDetails after the form change. If they differ, the FormAutofillChild sends the message
"FormAutofill:onFieldsDetected" to the FormAutofillParent, which triggers the form detection process in all children.
In onFieldsDetectedComplete the form handler's elements are updated accordingly to the new set of elements.

Comment 2

[Janika Neuberger [:janika]]

Attachment: Bug 1837161 - P2. Add tests for dynamic form change detection r=#credential-management-reviewers,dimi

Comment 3

[Janika Neuberger [:janika]]

Hey @owlish, my patch is currently blocked by geckoview-reviewers group. I'd appreciate if you or someone from your team could give a quick review on patch P1. It's only adding a new event to FormAutofillChild in mobile/shared/chrome/geckoview/geckoview.js. Thank you!

Comment 4

[Janika Neuberger [:janika]]

Attachment: Bug 1837161 - P4. Integrate dynamic form change detection into autocompletion process r=#credential-management-reviewers,dimi

If a form change occurs immediately after an initial autocompletion process, the FormAutofillChild notifies the
parent to re-trigger another filling process in all children to fill the fields that were added or became
visible on form change. Only form changes occuring within the threshold FormAutofill.fillOnDynamicFormChangesTimeout
(default is 500ms) also lead to another filling process.

Comment 5

[Janika Neuberger [:janika]]

@owlish, thank you for the review on P1!

Comment 6

[Janika Neuberger [:janika]]

Attachment: Bug 1837161 - P3. Keep track of autocompleted but invisible elements for form clearing r=#credential-management-reviewers,dimi

Comment 7

[Janika Neuberger [:janika]]

Attachment: Bug 1837161 - P5. Add tests for filling on form change r=#credential-management-reviewers,dimi

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/796fef1449fe
P1. Detect dynamic form changes r=dimi,geckoview-reviewers,credential-management-reviewers,owlish
https://hg.mozilla.org/integration/autoland/rev/e51320546694
P2. Add tests for dynamic form change detection r=dimi,credential-management-reviewers
https://hg.mozilla.org/integration/autoland/rev/cb0cbbcfd115
P3. Keep track of autocompleted but invisible elements for form clearing r=dimi,credential-management-reviewers
https://hg.mozilla.org/integration/autoland/rev/8ec455a9ba81
P4. Integrate dynamic form change detection into autocompletion process r=dimi,credential-management-reviewers
https://hg.mozilla.org/integration/autoland/rev/163b0ced2c77
P5. Add tests for filling on form change r=dimi,credential-management-reviewers

Comment 9

[Cristina Horotan [:chorotan]]

Backed out for causing bc failures

Backout link

Push with failures

Failure log

Comment 10

[Janika Neuberger [:janika]]

The problem is that when calling triggerAutofillActionInChildren in onFillOnFormChange we need to pass the fieldDetails of section.getAutofillFields() instead of the section.

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/9863ec556d84
P1. Detect dynamic form changes r=dimi,geckoview-reviewers,credential-management-reviewers,owlish
https://hg.mozilla.org/integration/autoland/rev/ff1bbda0856d
P2. Add tests for dynamic form change detection r=dimi,credential-management-reviewers
https://hg.mozilla.org/integration/autoland/rev/45e37689e00e
P3. Keep track of autocompleted but invisible elements for form clearing r=dimi,credential-management-reviewers
https://hg.mozilla.org/integration/autoland/rev/f87e28b48c2c
P4. Integrate dynamic form change detection into autocompletion process r=dimi,credential-management-reviewers
https://hg.mozilla.org/integration/autoland/rev/f27c8ecd8ed6
P5. Add tests for filling on form change r=dimi,credential-management-reviewers

Comment 12

[Atila Butkovits]

Backed out for causing failures at browser_form_changes.js

Backout link: https://hg.mozilla.org/integration/autoland/rev/b1ee17fc754b8e0393eeeb75a5d363968d80982e

Push where failures started: https://treeherder.mozilla.org/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel&revision=a821d37825967d3279c2a9e11d3d3ffecd757f7e&searchStr=os%2Cx%2C10.15%2Cwebrender%2Cdebug%2Cmochitests%2Ctest-macosx1015-64-qr%2Fdebug-mochitest-browser-chrome%2Cbc1

Failure log: https://treeherder.mozilla.org/logviewer?job_id=496741897&repo=autoland&lineNumber=15543

Comment 13

[Janika Neuberger [:janika]]

Updated the patches and fixed failures for browser_form_changes.js.

Try run without failures: https://treeherder.mozilla.org/jobs?revision=e7c7a8bc0c2861aaf4bfbd5bfc86c8c251a94f73&repo=try

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/8ec644ecbdc4
P1. Detect dynamic form changes r=dimi,geckoview-reviewers,credential-management-reviewers,owlish
https://hg.mozilla.org/integration/autoland/rev/730834404aec
P2. Add tests for dynamic form change detection r=dimi,credential-management-reviewers
https://hg.mozilla.org/integration/autoland/rev/1cad27d93ad4
P3. Keep track of autocompleted but invisible elements for form clearing r=dimi,credential-management-reviewers
https://hg.mozilla.org/integration/autoland/rev/3a970dd4c57e
P4. Integrate dynamic form change detection into autocompletion process r=dimi,credential-management-reviewers
https://hg.mozilla.org/integration/autoland/rev/e88442d8bd32
P5. Add tests for filling on form change r=dimi,credential-management-reviewers

Comment 15

[Cristian Tuns]

https://hg.mozilla.org/mozilla-central/rev/8ec644ecbdc4
https://hg.mozilla.org/mozilla-central/rev/730834404aec
https://hg.mozilla.org/mozilla-central/rev/1cad27d93ad4
https://hg.mozilla.org/mozilla-central/rev/3a970dd4c57e
https://hg.mozilla.org/mozilla-central/rev/e88442d8bd32

Comment 16

[Mayank Bansal]

This lead to a 5.6KB regression in installer size on Windows. This small increase is below any alertable threshold, so consider this comment only as a FYI

Comment 17

[Janika Neuberger [:janika]]

Attachment: Bug 1837161 - P6. Add form interaction event filled-on-fields-update for the categories creditcard and address r=#credential-management-reviewers,dimi

• Adds two events in formautofill/metrical.yaml: filledOnFieldsUpdateAddressForm and filledOnFieldsUpdateCcFormV2
  that record the filling state of a form after the second autofill action and which fields were filled after the update.
• Adds test cases in browser_dynamic_form_telemetry.js
• Renaming all dynamic form related test files to follow the same name pattern

Comment 18

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/08f4a2a84c19
P6. Add form interaction event filled-on-fields-update for the categories creditcard and address r=dimi,credential-management-reviewers

Comment 19

[Cristina Horotan [:chorotan]]

https://hg.mozilla.org/mozilla-central/rev/08f4a2a84c19

Comment 20

[Dianna Smith [:diannaS]]

:janika did you want to include this in the 138 release notes?
Also, should we change this to an enhancement instead of a task?

Comment 21

[Janika Neuberger [:janika]]

Yes, I changed it from task to enhancement.

:janika did you want to include this in the 138 release notes?

I'll get back to you on this tomorrow! Thanks for asking!

Comment 22

[Janika Neuberger [:janika]]

Attachment: Bug 1837161 - P7. Remove Nightly restriction from Dynamic Form Support preferences r=#credential-management-reviewers,dimi

Comment 23

[BugBot [:suhaib / :marco/ :calixte]]

A patch has been attached on this bug, which was already closed. Filing a separate bug will ensure better tracking. If this was not by mistake and further action is needed, please alert the appropriate party. (Or: if the patch doesn't change behavior -- e.g. landing a test case, or fixing a typo -- then feel free to disregard this message)

Comment 24

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a4b81f339d44
P7. Remove Nightly restriction from Dynamic Form Support preferences r=dimi,credential-management-reviewers

Comment 25

[Serban Stanca [:SerbanS]]

https://hg.mozilla.org/mozilla-central/rev/a4b81f339d44

Comment 26

[Janika Neuberger [:janika]]

Release Note Request (optional, but appreciated)
[Why is this notable]: Increases user convenience when autofilling address and credit card forms
[Affects Firefox for Android]: Yes
[Suggested wording]: Improves Address Autofill and Credit Card Autofill experience by ensuring that forms adjusting dynamically as users fill them are properly supported. This includes cases where fields appear or change based on input - such as selecting a country - so that newly revealed fields are now filled as expected.

Comment 27

[Dianna Smith [:diannaS]]

Added to the 138 release notes with slightly altered wording.
https://www.mozilla.org/en-US/firefox/138.0a1/releasenotes/