Closed Bug 441907 Opened 16 years ago Closed 16 years ago

synchronizing passwords etc on second machine fails with 100% CPU load

Categories

(Cloud Services :: General, defect)

x86
Linux
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: neumann, Assigned: anant)

References

Details

Attachments

(1 file, 3 obsolete files)

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9) Gecko/2008061712 Fedora/3.0-1.fc9 Firefox/3.0
Build Identifier: 0.1.30

Second machine I installed weave on goes to 100% CPU and only syncs bookmarks and history (I have asked it to sync everything).

Reproducible: Always

Steps to Reproduce:
1. I installed Weave on one machine and set it to save bookmarks, history, cookies, passwords, form-data. 
2. I then installed weave on a second machine and logged on and successfully synced bookmarks+history. 
3. Then I checkmarked cookies, passwords and formdata on that machine and tried to sync again. 


Actual Results:  
I got 100% CPU usage for about 40 minutes on second machine. Only bookmarks and history seem to be being synced on that machine. They are the only weave/snapshot files on that machine. Killing firefox and restarting returned after a while to 100% cpu usage and cookies, formdata and passwords are still not being synced or saved in the snapshot files.

Firefox+Weave on the machine I first installed weave seem to be fine.



Firefox+Weave on the machine I first installed weave seem to be fine.



Expected Results:  
Second machine should import new passwords that are there on the first machine and should not use 100%cpu.
I had a maxed out cpu on a 2nd macbook, let it run all night, and my macbook crashed the next day when I tried to quit (not force quit) FF3. The 1st setup went smooth. The 2nd computer, which generally outperforms the 1st, fails to get weave going. I am still prompted by the wizard now, after 3 long attempts. e

Both computers running final FF3. 

Here is a paste of the "brief log". 

2008-06-26 19:44:01	Service.Main	INFO	Weave Sync Service Initializing
2008-06-26 19:44:01	Service.Main	INFO	Weave scheduler enabled
2008-06-26 19:44:10	Chrome.Wizard	INFO	Initializing setup wizard
2008-06-26 19:44:10	Chrome.Wizard	INFO	Showing welcome page
2008-06-26 19:44:12	Chrome.Wizard	INFO	Showing backup page
2008-06-26 19:45:42	Chrome.Wizard	INFO	Showing account page
2008-06-26 19:45:57	Chrome.Wizard	INFO	Verifying login
2008-06-26 19:45:57	Chrome.Wizard	INFO	Adding user login/password to password manager
2008-06-26 19:45:57	Service.Main	INFO	Logging out
2008-06-26 19:45:57	Chrome.Wizard	INFO	Logged out
2008-06-26 19:45:57	Chrome.Window	INFO	Logging in...
2008-06-26 19:45:57	Chrome.Window	INFO	User string: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9) Gecko/2008061004 Firefox/3.0
2008-06-26 19:45:57	Chrome.Window	INFO	Weave version: 0.1.30
2008-06-26 19:45:58	Service.Main	INFO	Using server URL: https://services.mozilla.com/user/9ac168913e23ce5704fea2d39fc9c0eb861dd752/
2008-06-26 19:45:59	Chrome.Wizard	INFO	Login verified
2008-06-26 19:45:59	Chrome.Window	INFO	Login successful
2008-06-26 19:46:00	Chrome.Wizard	INFO	Showing initialization page
2008-06-26 19:46:04	Chrome.Wizard	INFO	Sync started
2008-06-26 19:46:04	Service.BmkEngine	INFO	Beginning sync
2008-06-26 19:46:05	Service.BmkEngine	INFO	Got status file from server
2008-06-26 19:46:06	Service.BmkEngine	INFO	Remote/local sync GUIDs do not match.  Forcing initial sync.
2008-06-26 19:46:06	Service.BmkEngine	INFO	Downloading server snapshot
2008-06-26 19:46:09	Service.BmkEngine	INFO	Downloading server deltas
2008-06-26 19:46:11	Service.BmkEngine	INFO	Local snapshot version: -1
2008-06-26 19:46:11	Service.BmkEngine	INFO	Server status: 0
2008-06-26 19:46:11	Service.BmkEngine	INFO	Server maxVersion: 1
2008-06-26 19:46:11	Service.BmkEngine	INFO	Server snapVersion: 0
2008-06-26 19:46:45	Service.BmkEngine	INFO	Reconciling client/server updates
2008-06-26 19:58:42	Chrome.Wizard	INFO	Sync started
2008-06-26 19:59:25	Service.BmkEngine	INFO	Beginning sync
2008-06-26 20:00:08	Service.BmkEngine	INFO	Got status file from server
2008-06-26 20:00:31	Service.BmkEngine	INFO	Downloading server snapshot
2008-06-26 20:01:18	Service.BmkEngine	INFO	Downloading server deltas
2008-06-26 20:14:01	Service.Main	INFO	Running scheduled sync
2008-06-26 20:14:04	Chrome.Wizard	INFO	Sync started
2008-06-26 20:14:47	Service.BmkEngine	INFO	Beginning sync
2008-06-26 20:15:31	Service.BmkEngine	INFO	Got status file from server
2008-06-26 20:15:56	Service.BmkEngine	INFO	Downloading server snapshot
2008-06-26 20:16:46	Service.BmkEngine	INFO	Downloading server deltas
2008-06-26 20:25:04	Service.Main	INFO	Weave Sync Service Initializing
2008-06-26 20:25:04	Service.Main	INFO	Weave scheduler enabled
2008-06-26 20:25:45	Chrome.Window	INFO	Sync window closed
2008-06-26 20:25:52	Service.Main	INFO	Weave Sync Service Initializing
2008-06-26 20:25:52	Service.Main	INFO	Weave scheduler enabled
2008-06-26 20:26:09	Chrome.Wizard	INFO	Initializing setup wizard
2008-06-26 20:26:10	Chrome.Wizard	INFO	Showing welcome page
2008-06-26 20:26:13	Chrome.Wizard	INFO	Showing backup page
2008-06-26 20:26:14	Chrome.Wizard	INFO	Showing account page
2008-06-26 20:26:35	Chrome.Wizard	INFO	Verifying login
2008-06-26 20:26:35	Chrome.Wizard	INFO	Adding user login/password to password manager
2008-06-26 20:26:35	Service.Main	INFO	Logging out
2008-06-26 20:26:35	Chrome.Wizard	INFO	Logged out
2008-06-26 20:26:35	Chrome.Window	INFO	Logging in...
2008-06-26 20:26:35	Chrome.Window	INFO	User string: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9) Gecko/2008061004 Firefox/3.0
2008-06-26 20:26:35	Chrome.Window	INFO	Weave version: 0.1.30
2008-06-26 20:26:36	Service.Main	INFO	Using server URL: https://services.mozilla.com/user/9ac168913e23ce5704fea2d39fc9c0eb861dd752/
2008-06-26 20:26:38	Chrome.Wizard	INFO	Login verified
2008-06-26 20:26:38	Chrome.Window	INFO	Login successful
2008-06-26 20:26:49	Chrome.Wizard	INFO	Showing initialization page
2008-06-26 20:27:03	Chrome.Wizard	INFO	Sync started
2008-06-26 20:27:04	Service.BmkEngine	INFO	Beginning sync
2008-06-26 20:27:05	Service.BmkEngine	INFO	Got status file from server
2008-06-26 20:27:06	Service.BmkEngine	INFO	Remote/local sync GUIDs do not match.  Forcing initial sync.

Getting the same issue with syncing passwords and/or form data on a second computer.  All other engines appear to be syncing without exhibiting this behavior.

This happened with 0.1.30 as well as 0.1.32.

It happened with 0.1.30 with a Vista machine (FF3 release version) as the first machine and the second machine XP (FF3 release version) was the one getting stuck at 100% CPU 'forever' as weave attempted to sync passwords and/or form data (all other engines worked fine)

It happened with 0.1.32 with an XP machine (FF3 release version) as the first machine and the second machine Vista (FF3 release version) was the one getting stuck at 100% CPU 'forever' as weave attempted to sync passwords and/or form data (all other engines worked fine)

I can dig deeper and provide more details and log output if necessary.
For now, we will disable form history and password sync.  Both seem to cause large reconciliation runs which are impossible to complete in any reasonable amount of time.

Putting this in the queue for 0.2, we may find some solution anyway (which would allow us to re-enable those engines).
Blocks: 433919
Target Milestone: -- → 0.2
This patch maintains an in-memory representation of the wrapped JSON for quicker lookups. The lookup table is passed from the FormStore to the FormSyncCore by the Engine superclass, right after it calls wrap but before calling reconcile (which in turn calls _itemExists).
Assignee: nobody → anarayanan
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true
Attachment #327193 - Flags: review?(thunder)
Comment on attachment 327193 [details] [diff] [review]
Improves perf. of FormEngine by using an in-memory lookup table

>+    // wrappedJSON is set to the result of wrap by the engine
>+    if (this.wrappedJSON[GUID])
>+      return true;
>+    else
>+      return false;

I think 'if (GUID in this.wrappedJSON)' is more correct.  Otherwise you are depending on the object/value it points to being true.
Attachment #327193 - Flags: review?(thunder)
We move _itemExists to Store, and pass a reference to it when a syncCore is constructed to make sure it can be called when reconcile is invoked.

_itemExists can, as a result, directly lookup the table which is now a local property of the Store.
Attachment #327193 - Attachment is obsolete: true
Attachment #327199 - Flags: review?(thunder)
Comment on attachment 327199 [details] [diff] [review]
Make _itemExists a method of the Store, and pass a reference to syncCore so it can be called

This patch is looking really good.  Let's put itemExists in the store superclass as we described, and require wrap to save a copy of the wrapped store for quick lookup.

Make sure to add a comment to wrap in the superclass explaining the required side-effect.

Also:

>+  _itemExists: function Store__itemExists(GUID) {
>+    this._log.error("itemExists needs to be subclassed");
>+    return false;
>+  },
>+  
>+  wrap: function Store_wrap() {
>+    this._log.error("wrap needs to be subclassed");
>+    return false;
>+  },
>+  
>+  wipe: function Store_wipe() {
>+    this._log.error("wipe needs to be subclassed");
>+    return false;
>+  },
>+  
>+  resetGUIDs: function Store_resetGUIDs() {
>+    this._log.error("resetGUIDs needs to be subclassed");
>+    return false;
>+  }

How about throwing, instead of logging a string and returning false?
Attachment #327199 - Flags: review?(thunder)
Store now has a default implementation of _itemExists, which assumes that wrap has previously set the _lookup property with GUIDs as keys. _itemExists now only needs to be overridden if necessary (as in the case of History and Tab engines).
Attachment #327199 - Attachment is obsolete: true
Attachment #327202 - Flags: review?(thunder)
Attachment #327202 - Attachment is obsolete: true
Attachment #327202 - Flags: review?(thunder)
Throwing instead of logging, and commented wrap in stores.js as per previous comment.
Attachment #327203 - Flags: review?(thunder)
Comment on attachment 327203 [details] [diff] [review]
Default implementation of _itemExists, and ensuring wrap sets the _lookup property of the Store

Great patch.
Attachment #327203 - Flags: review?(thunder) → review+
Checked-in changeset 0658e76b6413. Track status of PasswordStore processing removeCommands correctly using the new lookup table in bug 442090.
Status: ASSIGNED → RESOLVED
Closed: 16 years ago
Resolution: --- → FIXED
Component: Weave → General
Product: Mozilla Labs → Weave
QA Contact: weave → general
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: