1474839 - Formautofill does not work, can't change settings in about:config

Status: RESOLVED WORKSFORME

(Toolkit :: Form Autofill, defect)

Description

[Roger Davis]

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko/20100101 Firefox/61.0
Build ID: 20180704194937

Steps to reproduce:

1) Attempt to fill a form automatically in UBUNTU using Firefox


Actual results:

1) Only the information for that field appears - previously I could select to fill the entire form at once.
2) I attempted to find an on/off setting in Preferences - not there
3) I attempted to work on settings in ...formautofill...  I can change them, but settings do not remain changed after restarting.
4) This began immediately after the update for Ubuntu on (7/5/18)
5) I have two almost identical machines, ALL the above are true for both


Expected results:

1) Formautofill should fill all the form
2) Changed about:config settings should not revert unintentionally
3) See http://forums.mozillazine.org/viewtopic.php?f=38&t=3040807&p=14804531#p14804531

Comment 1

[u554753]

Hi Roger,

Thanks for reporting the issue. I can reproduce this behavior with the following specs:

Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko/20100101 Firefox/61.0
Build: 20180704192850

But I'm not sure if this is intended behavior or not. Marking as NEW, and setting appropriate component.

Comment 2

[Matthew N. [:MattN]]

Hello Roger, 

What is the value of the preference browser.search.region in about:config?

Comment 3

[Roger Davis]

Now testing using http://autofillforms.mozdev.org/test.html

Results are same as for random forms on the web.  This is better since it exists for this purpose, so is standardized.

This preference shows: - browser.search.region;US

I can assure you that before that update it was functioning by filling in entire or almost entire forms by one click from a list of response sets from which I could select.

Please also note that I have two very similar machines at two different locations showing the same results after that update.

Thanks!

Comment 4

[Matthew N. [:MattN]]

Hmm… can you check about:studies and see if any mention form autofill? Maybe attach a screenshot?

Comment 5

[Roger Davis]

Attachment: requested shot of about:studies

I can see no reference to formautofill or similar

Comment 6

[Roger Davis]

I see no mention of autofill.  Also, here's the text from about:studies.
------------------------------------------------------------------------
What's this? Firefox may install and run studies from time to time.
Learn more

    p
    pioneer-study-esper-2
    Complete•ESPER (Evaluating Similarity of Pioneers as Exemplars of Release) assesses the degree and sense in which users in opt-in Firefox Pioneer cohort differ from the Firefox release channel population. This is a one-time data collection of fields already collected in Firefox Telemetry. Bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1450951
    P
    Pioneer: Understanding usage of the web
    Complete•This study explores the relationship between online browsing patterns in response to information cues about websites. https://bugzilla.mozilla.org/show_bug.cgi?id=1450092
    l
    looking-glass-2
    Complete•MY REALITY IS JUST DIFFERENT THAN YOURS. Looking Glass is a collaboration between Mozilla and the makers of Mr. Robot to provide a shared world experience. Are you a fan of Mr. Robot? If so, join the hunt for answers! Participating in this shared world experience requires explicit user opt in. If you are not actively participating in the ARG no modifications will be made to firefox. https://support.mozilla.org/kb/lookingglass
    p
    pug-experience
    Complete•My reality is different than yours
    o
    online-news-log-recovery
    Complete•A patch to recover logs from the Pioneer Online News study.
    o
    online-news-patch
    Complete•A patch to fix the Pioneer Online News study.
    e
    esper-pioneer-shield-study
    Complete•ESPER (Evaluating Similarity of Pioneers as Exemplars of Release) is designed to assess the degree and sense in which the Firefox Pioneer cohort differ-from and/or are similar-to the Firefox release population. This is an exploratory data collection study and will not engage the users in any manner. Bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1414900
    o
    online-news
    Complete•Mozilla wants to know more about knowledge and opinions of news on the Web. To do so, we have partnered with the Annenberg School for Communication Research at the University of Pennsylvania to conduct a field study on this subject among Firefox Pioneers.
    s
    safe-browsing-v4-1387651
    Complete•https://bugzilla.mozilla.org/show_bug.cgi?id=1387651

Comment 7

[Matthew N. [:MattN]]

Hmm… it looks like the study is still live AFAICT: https://normandy.cdn.mozilla.net/api/v2/recipe/311/ Matt, should this study still be working?

Can you attach a screenshot of about:config after typing "extensions.formautofill" in the filter box?

In your profile folder (access it from about:support) can you look inside autofill-profiles.json to see if it has saved addresses?

Comment 8

[Robert (rrayborn, he/him)]

We should have killed this recipe.  We left it open to prevent unenrollment as the feature landed in release, however we should have upper bounded the targeting criteria.  This is now fixed for effected users.  Propagation will take up to 6 hours,

Comment 9

[Matthew N. [:MattN]]

Re-opening as it seems like the recipe wouldn't be the cause of the user losing the feature… maybe it was a SHIELD pref flip bug that lost the state.

(In reply to Rob from comment #8)
> We should have killed this recipe.  We left it open to prevent unenrollment
> as the feature landed in release, however we should have upper bounded the
> targeting criteria.  This is now fixed for effected users.  Propagation will
> take up to 6 hours,

I wasn't trying to ask to kill the recipe, I was confirming that that 20% rule was still in effect to make sure that the user would continue to have the feature if they were in the 20% before.

Comment 10

[Roger Davis]

Attachment: Screenshot from 2018-07-19 23-22-05.png

Screen shot - extensions.formautofill - As requested

Comment 13

[Roger Davis]

Please delete the immediately above answer/comment #12, it's a duplicate.

Comment 14

[Matthew N. [:MattN]]

(In reply to Roger Davis from comment #11)
> (In reply to Matthew N. [:MattN] (PM if requests are blocking you) from
> comment #7)
> > Hmm… it looks like the study is still live AFAICT:
> > https://normandy.cdn.mozilla.net/api/v2/recipe/311/ Matt, should this study
> > still be working?
> > 
> > Can you attach a screenshot of about:config after typing
> > "extensions.formautofill" in the filter box?
> > 
> > In your profile folder (access it from about:support) can you look inside
> > autofill-profiles.json to see if it has saved addresses?
> 
> I can't find autofill-profiles.json in about:support.

Sorry, I was saying to find your profile folder using the button in about:support.

> Here is the contents of autofill-profiles.json, from  Profile Directory

OK, thanks, I just wanted to know if it existed and had data in it and indeed it does. I've had the two comments with that data marked as private so your personal information isn't publicly accessible on the internet.

Could you please set the preference extensions.formautofill.loglevel to the string "Debug" (without the quotes) to enable more debug logging in the Browser Console. Could you then restart Firefox and then open the Browser Console[1] (not the web console for webpages) and type "autofill" (again, no quotes) in the filter box and then copy and paste the logs to an attachment on the bug (not a comment please).

Thanks again! 

[1] https://developer.mozilla.org/en-US/docs/Tools/Browser_Console

Comment 15

[Roger Davis]

Attachment: requested shot of Browser Console 7-20-18

NOTE :  This is the entire file, there was NO result from the requested filter - nothing was shown.

Comment 16

[Roger Davis]

NOTE! :  The attachment DOES NOT show results from "autofill" search, as there are NONE resulting.  I copied the entire file as presented without a search entry, in case it might be helpful.

NOTE! : As I mentioned earlier, I have two very similar machines at two different locations.  After about midnight tonight US Pacific time, I will be at the "other" location, which has the SAME problem with autofill.

Comment 17

[Roger Davis]

Attachment: BrowserConsole as requested.png

This was an empty result of filtering for "autofill", so I instead posted the entire file.  But there's been no activity for several days, so I posted this exactly as requested...

Comment 18

[Roger Davis]

FYI - I spend about 2 or 3 days at each location, both with very similar machines with the same problem.  I can work on this at either location.

Comment 19

[Roger Davis]

It's now been about 11 days since any information was posted on here, and I'm not sure what to do next.  Autofill is still not working.  Do I need to do something additional, or ???

Comment 20

[Matthew N. [:MattN]]

Sorry Roger, I'm busy with a project and I don't really have a good idea of what's going wrong. Maybe attach the raw data from about:support as an attachment and maybe I'll see something that could be the cause?

Comment 21

[Roger Davis]

Attachment: about:support data

Comment 22

[Roger Davis]

Raw data file uploaded as requested.  

THANKS!!!

It just seemed really odd that it's a feature, and was being worked on consistently, and then... nothing...

Comment 23

[Matthew N. [:MattN]]

Thanks

(In reply to Roger Davis from comment #21)
> Created attachment 8997638 [details]
> form:autofill-RawData8-4-18  As Requested

Hmm… the following looks a little fishy…

>       "defaultLocale": "und"

Can you try running the follow code snippets one at a time in the Browser Console[1]?
First set the pref devtools.chrome.enabled to true in about:config then run each line at a time in the textbox of the Browser Console and include the output of each.

> Services.locale.getRequestedLocale()

> Services.prefs.getCharPref("browser.search.region", "");

> Services.prefs.getCharPref("extensions.formautofill.supportedCountries");

> Services.locale.isAppLocaleRTL

Are you comfortable using a JS debugger? You could instead set a breakpoint in isAvailable[2] and see if/why it returns false.

[1] https://developer.mozilla.org/en-US/docs/Tools/Browser_Console
[2] https://dxr.mozilla.org/mozilla-central/rev/4e56a2f51ad739ca52046723448f3129a58f1666/browser/extensions/formautofill/bootstrap.js#60

Comment 24

[Roger Davis]

Attachment: Browser Console per requst 8-7-18

Output from commands in Browser Console as requested

Comment 25

[Roger Davis]

My Browser Console doesn't look like the one at "https://developer.mozilla.org/en-US/docs/Tools/Browser_Console", so I'm guessing and struggling.  Furthermore, how is Browser Console[1] different from simply Browser Console (Tools-Web Developer-Browser Console ?  I'm proceeding as though it's correct for Ubuntu Linux, and different from Windows.  

Browser Console content is very active, so I've copied the commands and outputs to a text file.  There is nothing to expand in the results.

- set the pref devtools.chrome.enabled to true = done
- Commands entered, file uploaded - "Browser Console per request 8-7-18"
- Unfortunately, I'm not familiar with JS debugger.

Comment 26

[Matthew N. [:MattN]]

(In reply to Roger Davis from comment #24)
> Created attachment 8998406 [details]
> Browser Console per requst 8-7-18
> 
> Output from commands in Browser Console as requested

Thanks. Can you re-run the last two but without the leading ">"?

Comment 27

[Roger Davis]

So sorry for my lack of attention !!!  I knew better, just didn't notice...

Services.prefs.getCharPref("extensions.formautofill.supportedCountries");
"US"

Services.locale.isAppLocaleRTL
false
-----------------------------------------------
Just in case it matters, I see lots of these:
[Exception... "Component returned failure code: 0x80070057 (NS_ERROR_ILLEGAL_VALUE) [nsIDOMWindowUtils.addSheet]"  nsresult: "0x80070057 (NS_ERROR_ILLEGAL_VALUE)"  location: "JS frame :: resource://gre/modules/ExtensionUtils.jsm :: runSafeSyncWithoutClone :: line 63"  data: no]
runSafeSyncWithoutClone
resource://gre/modules/ExtensionUtils.jsm:63:14
inject/cssPromise<
resource://gre/modules/ExtensionContent.jsm:443:13
next self-hosted:1221:9

Comment 28

[Matthew N. [:MattN]]

Hmm… I'm stumped… I guess it would be useful to figure out whether this is a problem with your Firefox profile, or the Firefox install… to do that, please try creating a new one from about:profiles then wait some seconds until the geolocation kicks in and sets the about:config preference "browser.search.region" to "US". Then restart Firefox in that profile and see if autofill works. Autofill won't work the first time you use a profile since it needs to wait for the geolocation.

You can test form autofill on https://luke-chang.github.io/autofill-demo/basic.html

Comment 29

[Roger Davis]

I created a new profile, checked about:config preference "browser.search.region", saw it as a modified setting "US".  Went to the test location "https://luke-chang.github.io/autofill-demo/basic.html", filled the form, pushed "Submit", went to the first blank, clicked on it, nothing appeared.  Pushed "Reset", still no result.  Restarted that browser instance under the new profile (Launch profile in new browser).  Repeated steps above.  No result.  Pushed "Reset", again filled the test form, pushed "Submit".  Went to first blank, tried again, no result.

I also tried it under one of the other profiles I had set up in the past.  No results.

It appears to me that the profile is not at fault, unless I did something wrong.

Please remember that I have two very similar machines, at two different locations, set up similarly, both began the same problem at the same time, immediately after the update of 7/5/18.  One has Intel MB, other one is ASUS.

Both are Ubuntu 16.04, Gnome Flashback, but also tested under Unity with no improvement.

Comment 30

[Roger Davis]

A major clue? - I synched Chromium with Firefox after previously trying "https://luke-chang.github.io/autofill-demo/basic.html" in Firefox.  I started "https://luke-chang.github.io/autofill-demo/basic.html" in Chromium, and immediately, WITHOUT first filling it in, the form was perfectly filled in by Chromium !

It seems to me that this pinpoints the problem as one of moving the data from the formautofill data file, which is now seen to be correct and operating, to the form on the page (semi-educated guess).

Comment 31

[Matthew N. [:MattN]]

(In reply to Roger Davis from comment #30)

Could you clarify what you're saying in comment 30? Are you saying that after doing something in Chromium it now works in Firefox or what Chromium has imported the data from Firefox and is now filling the addresses saved in Firefox?

Comment 32

[Roger Davis]

Sorry for the confusion.

Chromium has imported the data from Firefox and addresses can be filled successfully in Chromium using only the data imported from Firefox.  I never manually filled the test form in Chromium, it immediately automatically filled itself.

I still cannot fill forms in Firefox.

Comment 33

[Roger Davis]

Furthermore, since individual entries (boxes, frames, whatever they are called) can be successfully entered in Firefox, thus manually, one by one completing a form, it seems to me to be that the problem is in the portion that assembles the individual entries and posts them to the complete forms ?

Comment 34

[Roger Davis]

A collision happened, I hope nothing was lost...

Comment 35

[Matthew N. [:MattN]]

Can you try reproduce the bug in a build of Firefox from https://www.mozilla.org/firefox/? I suspect the problem is with the package you have installed.

Comment 36

[Roger Davis]

I went to the suggested location, but after some digging, I get firefox-61.0.2.tar.bz2 -> firefox-61.0.2.tar-2

I have options to Extract or open with Archive Manager.  I have two concerns - I'm unfamiliar with installing .tar files, and I don't want to overwrite my existing installation with a test installation, and lose the connection with Ubuntu Software, automatic updates, etc...  for test only.  If that's a final fix, maybe ok.

I searched and found https://libre-software.net/how-to-install-firefox-on-ubuntu-linux-mint/ - This how-to explains how to install Firefox 61 on Linux, with or without replacing an existing Firefox installation.  Will this work for this purpose?  However, this says to "move to opt" and "If you already had a previous Firefox version installed in the /opt directory, remove it with the following command:".  I can't find the current installation of Firefox, so I can't be sure if this will destroy the current installation. 

I also note that this problem has already been reproduced at Mozilla :
"Thanks for reporting the issue. I can reproduce this behavior with the following specs:
Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko/20100101 Firefox/61.0
Build: 20180704192850
But I'm not sure if this is intended behavior or not. Marking as NEW, and setting appropriate component.
Status: UNCONFIRMED → NEW
Component: Untriaged → Form Autofill
Ever confirmed: true
Product: Firefox → Toolkit"

Is this sufficient to narrow the package question?  Mine is apparently 61.0.1 64 bit, the one you refer me to is 61.0.2 .

Please instruct me...

Comment 37

[Matthew N. [:MattN]]

Those instructions are ok but you only need the first 2, don't deal with /opt as we just want this temporarily. Once you untar, you can just run the program like ./firefox/firefox (if it was untarred into a directory named "firefox").

Don't worry about the slightly newer version. Yes, it's true comment 1 claimed to reproduce the issue but due to the complexities of where and how the feature gets enabled, I don't want to have to waste time investigating whether that was a valid test.

Comment 38

[Roger Davis]

I downloaded Firefox, expanded it into a completely new directory, started it from that location in Terminal.  It started with my defaults active ?!?  I checked the version, exact same as I now have, 61.0.1 64bit.  So I used a file manager, went to the new directory, firefox executable, started it from there.  Same result.  OK, try the form test - no good.  

Tried the test form on a Windows computer, worked.  Noted that in the settings section for Windows, there is a specific location to activate forms, NOT present on either Ubuntu application.

Tried multiple ways to unpack the file firefox-61.0.2.tar-1, always get the same results as above, including version number.  Assume the new download has an erroneous splash page.

/home/roger/Downloads/FirefoxTest/firefox  is the location.

Does all this make sense to you?

Comment 39

[Sam Foster [:sfoster] (he/him)]

Thanks for sticking with this Roger. One thing to check: if you start the un-tarred firefox with another version of firefox already running, it will actually just open a new window with the previously-running firefox. It sounds like that is what is going on currently. So, make sure you close any other firefox windows before starting the firefox executable you want to test with.

Comment 40

[Roger Davis]

I did eliminate all other instances of Firefox before, as well as tried it again today.  It just loads all the previously open pages.   One thing different - after doing this 3 times tonight, it asked me if I wanted Firefox to be the default browser.  It didn't do that before as best I remember.  But still no luck in forms - it remembers absolutely nothing...

Comment 41

[Roger Davis]

This one now starts as 61.0.2 - 64 bit

Comment 42

[Roger Davis]

I tried the unpacked version again, and definitely closed all other instances.  Did not work.

I got a Ubuntu distro update today, 62.0 (64-bit), and doesn't work there either.

Comment 43

[Sam Foster [:sfoster] (he/him)]

Maybe Matt has more suggestions.

Comment 44

[Roger Davis]

It now works on this machine with the test page after today's update.  I'll need to check the machine at the other location, since it didn't work right after the update of about 5 days ago - maybe something else changed.  I can inform you probably on Saturday.

Comment 45

[Roger Davis]

It now works on the second machine!!!

Any idea what changed ?

Comment 46

[Matthew N. [:MattN]]

Hmm… no idea… very unusual. If you want to figure out the cause you could try use mozregression[1] but for now I guess we can close this.

[1] https://mozilla.github.io/mozregression/

Comment 47

[Roger Davis]

Agreed...