Closed Bug 1520808 Opened 5 years ago Closed 5 years ago

The history is not imported or not imported correctly from another browser (depends on the computer that is used)

Categories

(Firefox :: Migration, defect)

defect
Not set
normal

Tracking

()

VERIFIED FIXED
Firefox 67
Tracking Status
firefox-esr60 --- unaffected
firefox64 --- wontfix
firefox65 --- wontfix
firefox66 --- verified
firefox67 --- verified

People

(Reporter: obotisan, Assigned: standard8)

References

Details

(Keywords: regression)

Attachments

(4 files)

Attached image history.gif

[Affected versions]:

  • Firefox 64.0
  • Firefox 65.0b11
  • Nightly 66.0a1

[Affected Platforms]:

  • Windows 10 x64
  • Mac OS 10.9 / macOS 10.12
  • Ubuntu 18.04 x64
  • Windows 7 x64

[Prerequisites]:

  • Have Chrome or another browser that has a lot of history, bookmarks etc.

[Steps to reproduce]:

  1. Open Firefox.
  2. Import data from another browser:
    • Click on "Import Now" from new:tab
    • Go to Library -> Import and Backup
    • or open Firefox by writing -migration in Terminal/Command Prompt

[Expected results]:

  • The history section is populated by the imported data.
  • The Highlight section of Activity Stream is populated by bookmarks, history, downloads etc.

[Actual results]:

  • Sometimes: the history is not imported.
  • On other times: the history is imported, but the Highlight section is not populated.

[Regression range]:

  • This is a regression. I can't reproduce the issue on Nightly from 2018-01-11. I will try to find a regression range as soon as possible.

[Notes]:

  • This issue doesn't happen on all the computers, but on the ones that happens, the behaviour is constant.

I was not able to reproduce this with new profiles on nightly/Ubuntu 18.04, importing from Chrome and Chromium. The expected history seems to be there under the Library, however I can confirm the Highlight section does not get populated.

If you can get a regression range that would be great.

Is there any property or pattern to the configurations where this does not reproduce? Platforms, Chromium version, etc.

How much is "a lot of history". Do you think the size of the history is factor here?

Flags: needinfo?(oana.botisan)

Including the Browser Console output would also be helpful

Attached file browser console.txt

Regression range:

  • last good: 2018-03-22
  • first bad: 2018-03-23
  • mozregression concluded that Bug 1282770 - Convert uses of PU.asyncHistory.updatePlaces in browser/components/migration to PU.history.insertMany., might have been the one that caused this issue.

I don't think the platform influences this issue, because I can reproduce the issue on my Windows 10 x64, but the other colleagues that use Windows 10 x64 don't have this issue.
I can reproduce the bug on Mac OS 10.11, but I can't reproduce it on macOS 10.12 only partially (like in comment 1). And both of these platforms are installed on the same computer.

I don't think that the size of the history is a factor. You need some history in order for the Highlights section to be filled with at least one site. Or you can have a few bookmarks.

Attached you can find the errors that were displayed in the Browser Console.

Flags: needinfo?(oana.botisan)

Critical line from the log:

PageInfo: Invalid value for property 'visits': [{"transition":2,"date":"2019-05-19T12:07:45.262Z"}] PlacesUtils.jsm:760
some history did not successfully migrate.
Error(s) encountered during statement execution: no such column: secure
Sqlite.jsm:822
Couldn't get rows from the Chrome cookies database. MigrationUtils.jsm:647
some cookies did not successfully migrate.

I suspect the visits one is what we're really looking at here.

I'm assuming the sql error and the cookies error is because there wasn't anything in the Chromium database.

(In reply to Mark Banner (:standard8) from comment #4)

PageInfo: Invalid value for property 'visits': [{"transition":2,"date":"2019-05-19T12:07:45.262Z"}] PlacesUtils.jsm:760

Oh that's interesting - that date is in the future, and we explicitly deny that:

https://searchfox.org/mozilla-central/rev/330daedbeac2bba296d663668e0e0cf248bc6823/toolkit/components/places/PlacesUtils.jsm#371

So either we're importing the date wrong, or the history on those machines was generated with a future date.

Oana, can you check the dates/times on the machines please? See if maybe they're in the future or if the source history has been generated with a future date?

Flags: needinfo?(oana.botisan)
Has Regression Range: --- → yes

I did change the time and date on my computer, but at the moment is set to be updated automatically and it was correct. I can still reproduce the issue.

But if you tell me how can I check the time that the import is triggered, I will try to find more info.

Flags: needinfo?(oana.botisan)

(In reply to Oana Botisan from comment #6)

I did change the time and date on my computer, but at the moment is set to be updated automatically and it was correct. I can still reproduce the issue.

But if you tell me how can I check the time that the import is triggered, I will try to find more info.

If you go into Show Full History in Chromium, you should be able to see the dates/times of the history there. If they aren't in the future, then we could somehow be getting the import wrong, though I doubt it.

Flags: needinfo?(oana.botisan)

I am not sure that the history has anything to do with the history.
At first, when I check the history on Chrome, it had the time stamp placed in the future, but then I deleted it and tried again. The bug is still reproducing. This time I tested on Windows 10 x64.

Flags: needinfo?(oana.botisan)

Oana, could you try one of these builds and see if they help the issue:

They make it so that if there's a future date in the database, then it'll be written into Firefox as today's date.

If it still fails, then please can you get a new browser console log from them? There may be something else it doesn't like.

Flags: needinfo?(oana.botisan)

I tried to reproduce the issue using the builds from comment 9, but I couldn't. All the history was imported and the Highlights section was populated. I used all the previously affected computers with Windows 10 x64, Ubuntu 18.04 x64 and MacOS 10.11.
I think the the issue is fixed in those builds.

Flags: needinfo?(oana.botisan)

Rather than aborting adding data, we set the history items to have the current date/time. This seems mildly better than throwing away the history data.

Assignee: nobody → standard8
Status: NEW → ASSIGNED
Pushed by mbanner@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/1c0da40673e1
Allow importing of history from Chrome even if it was created in the future. r=mak

Oh, this landed before the m-c -> autoland merge happened.

Flags: needinfo?(standard8)
Pushed by mbanner@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/95b606177646
Allow importing of history from Chrome even if it was created in the future. r=mak
Status: ASSIGNED → RESOLVED
Closed: 5 years ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 67
Flags: qe-verify+

Comment on attachment 9039549 [details]
Bug 1520808 - Allow importing of history from Chrome even if it was created in the future. r?mak

Beta/Release Uplift Approval Request

Feature/Bug causing the regression

Bug 1282770

User impact if declined

If a user has happened to be running a Chrome instance where the clock date was in the future, then reset, and then attempted to migrate/import to Chrome, then the import may fail to import some or all of the history.

Bug 1282770 had originally made our history APIs stricter and we hadn't accounted for it on import.

Is this code covered by automated tests?

Yes

Has the fix been verified in Nightly?

No

Needs manual test from QE?

Yes

If yes, steps to reproduce

Steps as per comment 0, but with date in future.

List of other uplifts needed

None

Risk to taking this patch

Low

Why is the change risky/not risky? (and alternatives if risky)

Small patch to change any future dates to the current date. I suspect this happens rarely and only a few users might hit it, however given the desire to be able to import/migrate easily from Chrome, seems we could bump it up to beta.

String changes made/needed

None

Attachment #9039549 - Flags: approval-mozilla-beta?

I verified the fix on latest Nightly 67.0a1 using Windows 10 x64, Ubuntu 18.04 x64 and Mac OS 10.11 and the bug is not reproducing anymore.

Flags: qe-verify+

Comment on attachment 9039549 [details]
Bug 1520808 - Allow importing of history from Chrome even if it was created in the future. r?mak

Fix for migration issue, verified in nightly.
Let's uplift for beta 5.

Attachment #9039549 - Flags: approval-mozilla-beta? → approval-mozilla-beta+

Backed out for eslint failure on test_Chrome_history.js:7

backout: https://hg.mozilla.org/releases/mozilla-beta/rev/1f11e71f4611c12d0aaeb20285ceb99653971bd0

push: https://treeherder.mozilla.org/#/jobs?repo=mozilla-beta&selectedJob=225830715&revision=0c326e548cb88aec3b2fb09337cba0f36ec8bcae&searchStr=linting%2Copt%2Csource-test-mozlint-eslint%2C%28es%29

failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=225830715&repo=mozilla-beta&lineNumber=263

[task 2019-02-04T11:12:28.549Z] New python executable in /builds/worker/checkouts/gecko/obj-x86_64-pc-linux-gnu/_virtualenvs/init/bin/python2.7
[task 2019-02-04T11:12:28.549Z] Also creating executable in /builds/worker/checkouts/gecko/obj-x86_64-pc-linux-gnu/_virtualenvs/init/bin/python
[task 2019-02-04T11:12:30.202Z] Installing setuptools, pip, wheel...done.
[task 2019-02-04T11:12:31.252Z] running build_ext
[task 2019-02-04T11:12:31.252Z] building 'psutil._psutil_linux' extension
[task 2019-02-04T11:12:31.252Z] creating build
[task 2019-02-04T11:12:31.252Z] creating build/temp.linux-x86_64-2.7
[task 2019-02-04T11:12:31.252Z] creating build/temp.linux-x86_64-2.7/psutil
[task 2019-02-04T11:12:31.252Z] x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=543 -DPSUTIL_LINUX=1 -I/usr/include/python2.7 -c psutil/_psutil_common.c -o build/temp.linux-x86_64-2.7/psutil/_psutil_common.o
[task 2019-02-04T11:12:31.252Z] x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=543 -DPSUTIL_LINUX=1 -I/usr/include/python2.7 -c psutil/_psutil_posix.c -o build/temp.linux-x86_64-2.7/psutil/_psutil_posix.o
[task 2019-02-04T11:12:31.252Z] x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=543 -DPSUTIL_LINUX=1 -I/usr/include/python2.7 -c psutil/_psutil_linux.c -o build/temp.linux-x86_64-2.7/psutil/_psutil_linux.o
[task 2019-02-04T11:12:31.252Z] creating build/lib.linux-x86_64-2.7
[task 2019-02-04T11:12:31.252Z] creating build/lib.linux-x86_64-2.7/psutil
[task 2019-02-04T11:12:31.252Z] x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wl,-Bsymbolic-functions -Wl,-z,relro -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/psutil/_psutil_common.o build/temp.linux-x86_64-2.7/psutil/_psutil_posix.o build/temp.linux-x86_64-2.7/psutil/_psutil_linux.o -o build/lib.linux-x86_64-2.7/psutil/_psutil_linux.so
[task 2019-02-04T11:12:31.252Z] building 'psutil._psutil_posix' extension
[task 2019-02-04T11:12:31.252Z] x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=543 -DPSUTIL_LINUX=1 -I/usr/include/python2.7 -c psutil/_psutil_common.c -o build/temp.linux-x86_64-2.7/psutil/_psutil_common.o
[task 2019-02-04T11:12:31.252Z] x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=543 -DPSUTIL_LINUX=1 -I/usr/include/python2.7 -c psutil/_psutil_posix.c -o build/temp.linux-x86_64-2.7/psutil/_psutil_posix.o
[task 2019-02-04T11:12:31.252Z] x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wl,-Bsymbolic-functions -Wl,-z,relro -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/psutil/_psutil_common.o build/temp.linux-x86_64-2.7/psutil/_psutil_posix.o -o build/lib.linux-x86_64-2.7/psutil/_psutil_posix.so
[task 2019-02-04T11:12:31.252Z] copying build/lib.linux-x86_64-2.7/psutil/_psutil_linux.so -> psutil
[task 2019-02-04T11:12:31.252Z] copying build/lib.linux-x86_64-2.7/psutil/_psutil_posix.so -> psutil
[task 2019-02-04T11:12:31.252Z]
[task 2019-02-04T11:12:31.252Z] Error processing command. Ignoring because optional. (optional:packages.txt:comm/build/virtualenv_packages.txt)
[task 2019-02-04T11:18:09.346Z] TEST-UNEXPECTED-ERROR | /builds/worker/checkouts/gecko/browser/components/migration/tests/unit/test_Chrome_history.js:7:3 | Cu.import imports into variables and into global scope. (mozilla/no-import-into-var-and-global)
[taskcluster 2019-02-04 11:18:09.683Z] === Task Finished ===
[taskcluster 2019-02-04 11:18:09.683Z] Unsuccessful task run with exit code: 1 completed in 609.974 seconds

Flags: needinfo?(standard8)
Attached patch Patch for betaSplinter Review

This is a beta version of the patch - just a minor fix in the test for a change in how imports work.

Flags: needinfo?(standard8)

Please land the "Patch for beta" on mozilla-beta.

Keywords: checkin-needed
Flags: qe-verify+
Whiteboard: [qa-triaged]

I retested everything on beta 66.0b5 on Windows 10 x64, Ubuntu 18.04 x64, macOS 10.12 and Mac OS 10.11. I can't reproduce the issue anymore.

Status: RESOLVED → VERIFIED
Flags: qe-verify+
QA Whiteboard: [qa-triaged]
Whiteboard: [qa-triaged]
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: