Open Bug 1692980 Opened 2 months ago Updated 13 hours ago

Update new password heuristics model

Categories

(Toolkit :: Password Manager, enhancement, P2)

enhancement

Tracking

()

ASSIGNED

People

(Reporter: dimi, Assigned: dimi)

References

(Blocks 8 open bugs)

Details

Attachments

(1 file)

Bug 1595244 introduced a machine learning model using Fathom to improve detection of new password fields for password generation (v1). The last time we updated this model was in Bug 1638187 (15 May 2020)

It's time to collect those false-positive and false-negative cases discovered since then to improve the accuracy of the model.

Blocks: 1691441
Blocks: 1691444
Blocks: 1691903
Blocks: 1692172
Blocks: 1692182
Blocks: 1692198
Blocks: 1692163
Blocks: 1691003
Severity: normal → N/A

The improvements include:

  • Add three french keywords

    • Add Créer to newStringRegex
    • Add S'inscrire to registerStringRegex
    • Add Créer un compte to registerStringRegex
  • Add a formHasMultipleVisibleInput signal.
    This rule returns true when there is more than 3 visible input in a form.
    Since the idea in the signal is that a registration page often has multiple inputs.
    This rule only selects inputs whose type is either email, password, text, tel or empty,
    which are more likely a input field for users to fill their information.

  • Support formless password field in formHasMultipleVisibleInput signal.
    For password fields don't have an associated form, signals that require a form always return false.
    This patch adds an additional heuristic in formHasMultipleVisibleInput signal to support formless
    password field. The heuristic works as follow:

    1. Locate the closest preceding input with selector
      "input[type=email],input[type=text],,input[type=tel],input[type=password]".
    2. Find the lowest common ancestor of the password field and the input field found in step1
    3. Use the common ancestor as the "form" element, apply formHasMultipleVisibleInput signal

Update the result, Precision doesn't have significant difference and Recall is up around 5%

Before:
Training precision  : 1.0000   Recall: 0.8964
Validation precision: 0.9905   Recall: 0.8125
Testing precision   : 0.9926   Recall: 0.8933

After:
Training precision  : 1.0000   Recall: 0.9321
Validation precision: 1.0000   Recall: 0.8828
Testing precision   : 0.9862   Recall: 0.9533
Pushed by dlee@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/6d1e9ad2dd1d
Update NewPasswordModel.jsm to 78d4bf8. r=sfoster,tgiles
Depends on: 1674499

okay, there are two problems:

  1. isVisible in fathom throws exception in xpcsehll-test because the element doesn't have a window
  2. isProbablyANewPassword returns false for "confirmed password" in password change form (score is 0.73, slight lower than the current threshold holder 0.75). I'll check if that also happens in other password change forms in our samples.

While doing more test, I found out that there is a drawback on the new signal added. The drawback is that it doesn’t work well for password-change forms that have multiple input fields, for example, a password-change form has "current", "new" and "confirm" password fields.

I'll work on the problem and include the fix in this patch.

Flags: needinfo?(dlee)
You need to log in before you can comment on or make changes to this bug.