Closed Bug 228123 Opened 21 years ago Closed 1 month ago

synchronize bookmarks with .Mac/MobileMe support

Categories

(Camino Graveyard :: Bookmarks, enhancement)

Product:
Camino Graveyard
Component:
Bookmarks
Platform:
PowerPC
macOS
Type:
enhancement
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED INCOMPLETE

People

(Reporter: sbwoodside, Unassigned)

References

Details

Attachments

(3 files, 1 obsolete file)

SyncService sync schema for Camino Bookmarks
29.33 KB, application/octet-stream
Details
Attempt at implementing SyncServices into Camino
1.86 MB, application/zip
Details
Alex's changes to existing files as a patch
23.13 KB, patch
Details | Diff | Splinter Review 
This is from the mailing list. This is for people with multiple computers and
they want to keep the bookmarks in sync all the time between them. Apparently
Safari does this through some special API that we can't use.

But the idea is to store the bookmarks.plist file on the .Mac iDisk instead of
the in the usual location. Apparently the iDisk will always be synchronized
between multiple computers that have the same .Mac account. So, if the bookmarks
file is on the iDisk, then it will always be synchronized.

Hopefully I have the general idea correct. I have no use for this feature myself
since I have only one computer, but there were people on the mailing list who
claimed they would like to have it ;-)
Target Milestone: --- → Camino1.0
Now that Tiger (10.4) appears to be opening up the iSynch API, we could probably
do this as a 10.4+ feature. Definitely worth looking into, as it could be
extremely useful.
Apple has released its free SDK for .Mac support, get Details here:
http://devworld.apple.com/internet/dotmackit.html

*** Bug 280979 has been marked as a duplicate of this bug. ***
Bookmark sync for FTP would be nice too. I think Omniweb does this too? Its
pretty neat.

But iSync could probably be used for this, especially after the API is being
opened. A feature for 2.0 maybe. (Or something..)
.Mac support doesn't need to be limited to Bookmarks, basically every user
relevant information could be handled with iSync. Of course one has to balance
what really pays off, but for stuff like prefs, cookie blacklists, passwords
etc. this could be tremendously useful.
I opened a new bug for a similar feature request here:

https://bugzilla.mozilla.org/show_bug.cgi?id=285340

The difference with my request is that there is no explicit endorsement of a
third-party (Apple) for-fee service (iDisk).  Instead I request that SFTP, FTP,
HTTP, or HTTPS be used to implement storage.  There is a Firefox extension (
https://addons.update.mozilla.org/extensions/moreinfo.php?application=firefox&version=1.0&os=MacOSX&id=14
) that does this (except for SFTP support).  My request also includes
interoperability with Firefox, perhaps making it a general Mozilla request.
This won't happen for 1.0.

Targeting for 1.1.
Target Milestone: Camino1.0 → Camino1.1
So is there a problem if I go ahead and take on the task of synchrnonizing bookmarks with .Mac? Via the .Mac SDK, the .Mac login info could be automatically read from user preferences. the only thing the user would need to do would be to specify some sort of scheme for when to synchronize. not too hard to do. i've done some work with the .Mac SDK before. As for the FTP and other WebDAV stuff, I dunno. I doubt I'll get into that. Running low on time these days like everyone else.
No one's actively working on this bug (that's why its status is NEW), so if you'd like to work on it, that would good.

However, in reading the link (from comment 2) about the .Mac SDK, it sounds like one has to include a 500 KB framework inside one's app, and that size is likely to meet with some opposition in Camino :-(
Just wanted to say that .Mac synching is tremendously useful and of the main reasons I use Safari as my main browser. IMO this feature is easily worth 500KB (who cares about the size of an app anyway).
I am looking into this, too, but got distracted by the start of the new semester.

What you might want to do, is have a look at the new SyncServices framework in 10.4. Althouhg going this route would mean for a 10.4 only feature, but you save yourself alot of work, as you basically only have to provide SyncServices with the recent changes, and pulling the supplied changes from SyncServices into the local bookmarks. All the rest, like conflict resolution and defining sync intervalls, is done for you.

No sync support leads me constantly back to Safari, too.
My inclination at the moment is to tackle this using Sync Services. True, it would only be a 10.4 feature -- but it's 1) easier to do this way than doing all of the syncing code manually, and 2) I won't have to worry about the patch getting rejected (after all of that work) because of a 500k framework (which would be really frustrating. 

How's that sound to everyone?
we support a lot more data than safari does, can we adequately map our bookmark schema to theirs? i'd be worried about losing data round-trip with the SS approach.
You don't have to use Apple's predefined schemas, you can define your own, or extend Apple's. If you want to look more into this, it's described here: http://developer.apple.com/documentation/Cocoa/Conceptual/SyncServices/Articles/SchemaDesign.html
Henning,

I'm more than happy to crank on the code -- but am just now getting spun up on the specifics of the differences between Camino and Safari as far as "supported data" goes. Do you want to jump on defining a schema, and then I'll follow up with the implementation once your done? Or does anyone else that's familiar enough with Camino's data want to get started on that? Seeing as how I can't crank on this for a week or so, it would make for good timing if someone knocked it out. Then I could follow up with implementation.
Chris knows pretty well what we support (and what we plan on supporting in the future) in a bookmark, but also see the discussion in bug 307620 and bug 307743.
Matthew,

unfortunately I read your post shortly before leaving for a week of vacations. I'll see what I'll get done here. I have a schema nearly put together already from my earlier explorations, and so I think I can send you something by next week.

Henning
A first attempt at a SyncSchema for Camino Bookmarks.

It is based loosly on Safaris Schema, found under /System/Library/SyncServices/Schemas/Bookmarks.syncschema/, but it defines a new schema, and does not extend Safari's. I tried to keep the Entity and Relationship names in sync with the class attributes in the source code.

Every record supplied to SyncServices must have a unique identifier. As we already have a UUID for every BookmarkItem, I think we can just use that as an identifier. If a user creates the same bookmark independently on 2 computers, and then syncs, this would result in a change of the UUID for one of this bookmarks, but as we'll treat them as the same bookmark from then on, I think this would be correct. But maybe you should ask Chris or someone who knows the Bookmark Code if somewhere else something relies on not-changing UUIDs.

Furthermore, one has to define the properties which uniquely identify a record, for just the case described above (A user creating the same bookmarks on more then one computer and then syncing, in this case, SyncServices has to know how to identify those bookmarks as the same). For Folders, I defined Title and Parent as the defining properties. I am not sure, if maybe SpecialFlag has to be used, too. For Bookmarks, it's Title, URL, and Parent, which should be sufficient.
(In reply to comment #18)
> Every record supplied to SyncServices must have a unique identifier. As we
> already have a UUID for every BookmarkItem, I think we can just use that as an
> identifier. If a user creates the same bookmark independently on 2 computers,
> and then syncs, this would result in a change of the UUID for one of this
> bookmarks, but as we'll treat them as the same bookmark from then on, I think
> this would be correct. But maybe you should ask Chris or someone who knows the
> Bookmark Code if somewhere else something relies on not-changing UUIDs.

On second thought, as the Record-IDs from SyncServices are specific to every client, the UUID wouldn't even change in this case, making the UUID, IMHO, a good candidate for identifying sync records.
*** Bug 323626 has been marked as a duplicate of this bug. ***
I thought I would comment, as this is the only thing that is stopping me moving to Camino as my main browser from Safari.

The .Mac synchronisation of Safari bookmarks works flawlessly and allows me to synchronise my bookarks on both of my Macs and across to my brothers.

I have a G5, and a Powerbook which I take on the road with me every day.  I tried using Camino and synchronising my two computers with Chronosync, but it only works  if only one of the bookamrk files has changed.

With .Mac and Safari individual bookmarks are synchronised which works really well.  as far As I am concerned this is worth the .Mac subscription in itself.

The sad thing is that Camino actually browses a lot better than Safari does for the web pages I mainly use.

Ed

Henning, Matthew, any news on this?  

We'll need something more-or-less ready in about a month in order for this to make 1.1 (and get reviewed and some real-world testing before the release).
As normal, distractions take precedence :) But seriously, I'm just wrapping up a ton of things this week, and I'll have a lot of time this weekend to get to the bottom of this feature. I've been chipping away at it off and on for a while now. I'll shoot for having something ready to test the latter part of March and see how it goes from there. Incidentally, up until recently, I wasn't even able to get the SyncServices code to compile/run correctly (http://www.oreillynet.com/mac/blog/2006/02/magic_startup_key_combinations.html and http://www.oreillynet.com/mac/blog/2005/12/funniest_thing_ive_ever_seen_i.html) but once I did, the learning curve hasn't been to bad with SyncServices.

I'll try to remember and sent out an update in ~2 weeks with a better idea of how far along things are coming. I really do want this to make it into 1.1.

M.
Matthew, can you update us on how this is coming?  1.1 obviously isn't coming out in April, but we'll need a reviewable patch by mid-April (sooner is always better) in our new timeframe.
QA Contact: bookmarks
Matthew, any updates on this?
Assignee: mikepinkerton → nobody
Not gonna make 1.1.
Target Milestone: Camino1.1 → Camino1.2
I certainly hope we can get .Mac syncing abilities into Camino (the whole `~/Application Support/Camino` folder, plus the Camino pref/plist file).

Re; <a href="#c6">Comment 6</a> (FTP/SFTP syncing abilities);  There is already a Camino PrefPane for this...
<a href="http://www.nadamac.de/camitools/index.php">CamiTools</a>.

That said, a <a href="#c9">500k payload for the .Mac syncing framework</a> facility doesn't seem like a huge deal to me.

I can take a shot at this if you'd like.  There are other F/OSS apps that leverage the .Mac SyncServices/Framework (ala Smultron) I can look at as examples).
Target Milestone: Camino1.6 → ---
Does anyone know the status of this? It really seems like a minor thing to add considering the benefits and the apparent demand. Can somebody provide an update? I really hope this makes it to the next major release! 

If this feature is there then it could be used over Safari with complete set of OS X features, this is the only thing I can see which is lacking!

I love Camino, but this is such a silly little feature which is missing. 
The status of this bug is that it is still open, as indicated by the fact that is has not been closed as FIXED. It will be in the next major release if and only if someone provides working code for it in the very near future.

Anyone planning on commenting in this bug should first read:
https://bugzilla.mozilla.org/page.cgi?id=etiquette.html
Unless you are planning on working on this bug, commenting is probably not appropriate.
(In reply to comment #29)

> I love Camino, but this is such a silly little feature which is missing. 

If it's such a silly little feature, then you won't have any problem writing a patch for it in the next two weeks.

Get cracking.
Summary: synchronize bookmarks with .Mac support → synchronize bookmarks with .Mac/MobileMe support
I've made a somewhat working integration of SyncServices.  I have created a helper tool that is used by SyncServices when .Mac/MobileMe syncs.  It syncs bookmarks and bookmark folders perfectly.  There are some bugs when trickle syncing as I have had to implement a trivial way of detecting changes made to bookmarks.  Also, one problem is the fact that Camino does not update the bookmarks in its GUI if the bookmark file changes while Camino is running.  If you quit Camino and open it again, the changes will be seen.

The full implementation is far from perfect but if someone would like to tidy it up and fine tune it, Id be happy to send you the source code.  I'm not sure exactly how I submit my fixes so I'm posting this comment instead.
That's great news! The best thing to do would be to use the "Add an attachment" link above to post your source here; that way anyone interested can pick it up and go from there (and it will still be available if you vanish mysteriously, as has often happened to people working on this bug ;) )

Please do make sure your source has a licensing block that will allow us to use it in Camino when you post it. See for example the top of:
http://mxr.mozilla.org/mozilla/source/camino/src/application/main.m
Archive contains:

CaminoSync  - an Xcode project for the SyncServices schema
Modified Application Source Files  - Modified versions of the MainController
Modified Bookmark Backend  - Modified versions of the Bookmark classes

New Source Files  - Source files for 'CaminoSyncTool' and modified versions of the Bookmark classes to be included with 'CaminoSyncTool' to aid bookmark management

Current known problems:

- Does not pull bookmark folders properly
- Does not keep bookmarks in order
- Camino does not detect changes when running
- Camino overwrites any changes made by 'CaminoSyncTool' when quitting

(Note:  Use a file comparator to find what modifications I have made to the source tree files)
Alex, this is great!

Going forward, it's definitely helpful to have patches rather than code drops to help keep the changes clear as the existing files keep changing (but given the history of this bug, anything is better than nothing!).  

It's a bit difficult to create a patch for everything when you're adding lots of new files, especially new files in new directories, if you don't have cvs write permissions; we do have instructions on that here: http://wiki.caminobrowser.org/Development:Coding#Proper_patch_format

I haven't done any investigation of the file layout of the new code, so what I'm attaching is just a diff of the existing files to facilitate my earlier comment.

I do hope you'll be interested in continuing to work on the "Known Problems" you mentioned; if you have specific questions about parts of the bookmarks code or need pointers, Stuart Morgan and Chris Lawson probably know it best, but most anyone on the team should be able to help some.
Could someone tell me why this extract of code won't work.  It is supposed to add all the bookmarks and bookmark folders.  The "record" parameter of the second method is an NSDictionary containing the data for any BookmarkFolder that has the root BookmarkFolder as its parent.  The "identifier" is the UUID of the BookmarkItem.  The "bookmarks" and "bookmarkFolders" parameters contain an NSDictionary of the data for all the Bookmarks and BookmarkFolders. If anyone needs any more info to help them identify the problem, just say.
Attachment #378684 - Attachment mime type: application/octet-stream → text/plain
Attachment #378684 - Attachment is obsolete: true
Please disregard my last comment.  I have sorted it now.  Apple's SyncServices makes things a lot more confusing than I would have thought they need to be.  For some reason, SyncServices was converting all of the UUIDs into RecordIDs which messed up the if statement to determine if the BookmarkItem was a child of the current BookmarkFolder.
Alex, any progress here (or any further questions for us)?
No, sorry.  I haven't been working on it at all.  When Snow Leopard came out I had problems building Camino.  I have managed to fix these problems, however, I haven't had the time recently to even look at it.  It will take me a while just to remember what I've actually written.  I sort of hoped someone else would pick up where I left off.  Unfortunately, it seems no one has done that.  So...I'll try and find some time to work on it and I'll let you know if I get any further or need any questions answering.
If you do have a chance to get back to working on this and it's ready early enough to get plenty of testing, it is something we'd like to include in 2.1.
Can someone tell me how I can reload the bookmarks from file while Camino is running?  Just calling the method, "[[BookmarkManager sharedBookmarkManager] loadBookmarksLoadingSynchronously: YES];", doesn't work.  Been digging around to find out why, but can't for the life of me figure it out!  I'm sure its something really trivial that I'm just missing.  So would really appreciate it if someone could help me out here.
Camino's bookmark system is not designed to repeatedly load bookmarks from disk, just once at startup.

Is there no way for Sync Services to communicate with Camino, rather than a helper tools, so that the changes can be made in memory instead of causing a (potentially very expensive) fully re-read of bookmarks from disk? If a separate helper is really the only option, then the bookmark code will need restructuring to allow for multiple loads.
That's what I thought might be the problem.

No, unfortunately.  That would be the ideal way to sync the bookmarks, however, this would require Camino to be opened every time a sync occurs.  I wonder if there is any way SyncServices can get Camino to handle the syncing when it is running, and the helper tool to handle it when Camino isn't running.  Or maybe the helper tool can do this by checking if Camino is running when it is executed by SyncServices and then hand over control of the sync operation, or communicate with Camino during the sync process.  Any ideas about what the best route to take is?
I've been re-reading the sync services programming guide looking for anything that might suggest that Camino needs to be running when a sync occurs and have drawn a blank.

My understanding is SyncServices syncs to the 'Truth Database' when MobileMe (or some other server) syncing occurs, and then Camino syncs to the database when it starts up. And then syncs while it is running, whenever needed.

Have I missed something?
Andrew, (not having read the SyncServices guide, but having paid some attention to the bug) I think the issue was that the sync helper tool that Alex was using (I don't know if that's "The Way" for sync or just a way, but Safari has one, too) could make changes to our bookmarks file with impunity, but Camino would only ever notice changes on open (if a sync had happened since Camino had last quit), i.e. no changes that happen while Camino is running would update the bookmarks in Camino's UI and internal data structures, and any changes made to the bookmarks file while Camino was running would actually be stomped when we write bookmarks from memory to disk on quit.  The when-Camino-is-not-running syncs are fine.

The discussion in comment 44, then, is how do we handle syncs while Camino *is* running, when the in-memory copy of the bookmarks is the only true copy of bookmarks.  (And then, after that, making sure the sync triggers Spotlight metadata changes for added, deleted, and changed bookmarks, updates the autocomplete tries, updates the UI and so forth.)

Offhand, 

(In reply to comment #44)
> Or maybe
> the helper tool can do this by checking if Camino is running when it is
> executed by SyncServices and then hand over control of the sync operation, or
> communicate with Camino during the sync process.

sounds to me like the best solution (particularly the latter flavor, to reduce code duplication?), but I'm not very familiar with the bookmarks system or the feasibility of any sort of code change ;)
It remains my belief that nothing in the SyncServices programing guide requires use of an external helper tool. The fact Safari has one suggests there might be some very good reason to do so, but what that might be I have no idea.

As you point out above, having a helper makes certain aspects more complex.

I'd like to have a go at this, but have no time.
Status: NEW → RESOLVED
Closed: 1 month ago
Resolution: --- → INCOMPLETE
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: