1329295 - When dragging an existing entry from Address Book to a List the complete entry is not moved

Status: RESOLVED INCOMPLETE

(Thunderbird :: Address Book, defect)

Description

[hgruenberg]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:50.0) Gecko/20100101 Firefox/50.0
Build ID: 20161208153507

Steps to reproduce:

I clicked on Address Book and then entered a name where it called for Name or e-mail.  The desired name was displayed showing the supplementary data included in its Properties (and it was the only entry.  I also clicked on All Address Books and there was still only the one entry).  I selected it and dragged it to a Mailing List within the Address Book.  I clicked on the Mailing List but it was not there.  So, I clicked on the Mailing List Properties and the name and e-mail address appeared as the last entry.  So I clicked Add.  The entry then appeared in the Mailing List, but only the Name and the E-mail address.  I clicked on the Address Book again and found that a new entry had been added with only the name and the e-mail address.  The original entry was still there.


Actual results:

I was completely unable to add the complete name and all its properties to the Mailing List


Expected results:

The complete entry from the Address Book should have been added to the Mailing List.

Comment 1

[Jorg K (CEST = GMT+2)]

Summary: When dragging an existing entry from Address Book to a List the complete entry is not moved.

(In reply to hgruenberg from comment #0)
> The complete entry from the Address Book should have been added to the
> Mailing List.

I think this works as designed. Mailing lists are just set of addresses so you can send a message to a group of people. When you move an address book entry to the mailing list, this entry is copied, not moved.

Typically you enter names directly into the mailing list, which creates new entries in the list, but you can also drag onto the list, which creates a copy.

Thomas, do you have further comments here?

Comment 2

[hgruenberg]

I agree that it should have been copied, but it wasn't.  I don't agree that one normally enters names directly into the mailing list.  That is one way.  However, if one has a number of entries in an Address Book and one wishes to create a mailing list, copying is the way to go.
I have since copied other Address Book entries into the Mailing List successfully but the one which caused me to report the bug won't copy correctly but instead creates an entry in the Mailing list at the bottom where one would normally make the addition but this entry is additional to the original one since that one is not in the mailing list and the additional one is added to the Address Book so that there are now two entries there.  One (the original) is complete but the added one only contains the display name and the e-mail address.
Since it has worked for other Address Book entries but not this one, there is a bug somewhere.

Comment 3

[Jorg K (CEST = GMT+2)]

No doubt that the address book isn't TB's best feature and it's surely possible that there is a bug somewhere. But to find and fix it, we need clear steps to reproduce it.

Since we can't see what your doing, it's very important to be clear and use precise terminology.

So we agree that one method is called "direct entry":
Double-click onto the list or chose "Properties" (or "Edit List" in later versions of TB) from the context (right-click) menu and enter the address like you enter it for an e-mail.

The second method is adding/copying an address to the list by drag and drop. There is no "Copy" option in the UI anywhere. So that's the option we're talking about. You're dragging (and then dropping) an existing address card from a regular address book onto a list and something doesn't work.

So please write down the steps clearly and state what each step should do and does do or doesn't do. I'm really confused by long sentences like:

===
1) I have since copied other Address Book entries into the Mailing List successfully 2) but the one which caused me to report the bug won't copy correctly but 3) instead creates an entry in the Mailing list at the bottom where one would normally make the addition 4) but this entry is additional to the original one since that one is not in the mailing list and the additional one is added to the Address Book so that there are now two entries there.  One (the original) is complete but the added one only contains the display name and the e-mail address.
===

Let me see whether I read this correctly:
1) In general dragging entries (cards) on mailing lists works and a full and complete
   copy is created in the list that shows up in the tree view.
2) However, there is one particular card that when dragged, doesn't appear
   in the mailing list in the tree view (or does it?)
3) In this case, an additional entry (line) can be seen when opening the mailing list
   properties where one would usually add more addressed manually.
4) And now you lost me. Sorry. I think you're describing that the card was
   added to the mailing list twice? Once complete, and once as a new last entry only with
   name and e-mail address? But what happens in the tree view?

Sometimes it's better to attach a screenshot. We'd also need the data to reproduce the problem. So either write down the exact steps to create this data, or attach the .mab files that contain the data we need to reproduce it.

Comment 4

[hgruenberg]

Your 4 step analysis is correct with the following elaboration:

2) However, there is one particular card that when dragged, doesn't appear
   in the mailing list in the tree view (or does it?) - No, it doesn't.
4)The new entry in the Properties of the Mailing List (as in 3) does not appear in the "tree" until one clicks "Add" in the Properties window, whereupon an entry appears in the tree but nothing changes in the Properties window.  That entry only contains the name and the e-mail address but none of the other entries for that addressee.
Then, if one clicks on the Address Book itself and enters the addressee's name, there are now two entries - the one which was dragged to the mailing list and a new one which contains only the name and e-mail address (as it appeared in the Mailing List tree)

Comment:  One of the problems is that when one clicks or double clicks on an entry in the Properties window it doesn't permit you see the full entry for that addressee.  So, the newly entered line looks the same as all the other entries although the others have additional data in the underlying address cards.  It becomes apparent only after "Add" is clicked and the new entry appears in the tree.

I have tried to determine what might be different about the problem entry relative to all the other ones.  They all include entries for 1st Name, Last Name, Display Name, Nickname, e-mail address and Organization under the "Work" tab. The Nickname is always a 4 digit number.  In a few cases there may be a telephone Number, also.  There are no other entries.

Since then, for other entries I've confirmed what you state in 1)

Comment 5

[Jorg K (CEST = GMT+2)]

I'm a bit confused by two things here:
1) The properties panel doesn't have and "Add" button, there is only an "OK" button.
2) To drag a new address onto a list, you must first click on the book/list containing
   the address to be dragged. Then you select that address and drag it to the list.
   Now you can click onto the list. I would suggest that at that stage it should
   really show in the tree view. You say it doesn't and it will show up when you
   click a "non-existent" "Add" button on the properties panel of the list.

Hmm, are you using a malfunctioning add-on?

Comment 6

[hgruenberg]

You are obviously correct.  It's "OK", not "Add">
I've tried to reproduce the behavior.  In the meantime, I had added the additional data to the new tree entry which contained only the name and e-mail address. This resulted in what appeared to be two similar entries in the Address Book tree.

Now, when I dragged one entry (I think it's the one I augmented, but I can't be sure because they both contain the same data) everything worked as expected.  When I erased this from the Mailing List tree and repeated the process with the other entry from the Address Book tree, the behavior defaulted to the problem, but with a slight difference.

1) The entry did not appear in the Mailing List tree but did in the Properties window.
2) When I clicked on OK in the Properties window the complete entry from the Address Book appeared in the Mailing List Tree
3) When I reviewed the Address Book tree there was no new entry that contained only the name and e-mail address.

So, what I conclude from that is there was something (maybe invisible)unusual in the original Address Book entry. Unfortunately, once I added the missing data to the entry with name only which had been created as a problem originally, I could no longer tell which of the now similar two entries in the Address Book tree, was which. So, I can no longer identify the original problem entry.

However, it is reasonable to conclude that the problem was not with the Address Book but with my entry and what that was I can't tell.  Nevertheless, the Address Book exhibited strange behavior as a result without giving any indication that there was a problem with the entry.

So, although there is still a bug, it isn't causing a problem with proper entries.  Consequently, I think it isn't worth pursuing the issue further.  I thank you for your efforts.

Comment 7

[Jorg K (CEST = GMT+2)]

Well, I'm happy to pursue it further if you can supply the address book with the failing entry.
I'd like to see an entry that when dragged onto a mailing list doesn't show up.

There is something a strange I can observe.

STR:
- Create a new address book
- Create an entry:
  First: First
  Last: Last
  Display: Firstname Surname
  Email: [e-mail], needs to be a new e-mail never seen before
- Create a new mailing list in a different address book
- Drag the entry to the ML and receive a perfect copy
- Delete the entry from the ML.
- Edit the entry, remove First and Last, and add Work: 8888 and Home: 7777
- Now drag the entry again.

What you get is the original entry that was first dragged, not the modified one.

I can reproduce these steps on TB 45.6 and current trunk TB 53.

In any case, the problem I've reproduced is
"When dragging an existing card from Address Book to a List, the complete card is not copied in case a card with the same e-mail address existed before in that List".

Is you list short enough that you would have noticed by count that the dragged entry didn't show up? Or were you simply looking for the dragged entry, but instead of seeing it, a previous version with different details sprang back to life, so the entry you dragged didn't appear to have shown up, however, an old entry showed up in its place elsewhere?

It appears that the list has a reference to the previously deleted entry when you add an entry with the same e-mail address again.

In any case, the cards are indexed by the e-mail address, so I think the "special" card that caused you a problem must have been in the list before? Is that possible?

Aceman, please take a look at the STR, something is very fishy here.

Comment 8

[hgruenberg]

I'd rather not release my address book. But thanks, anyway.

Comment 9

[Jorg K (CEST = GMT+2)]

Fine, but can you at least answer my questions:

... so I think the "special" card that caused you a problem must have been in the list before? Is that possible?

Comment 10

[hgruenberg]

Yes. That's definitely true.  But when I filled out the added abbreviated entry in the Address Book tree which had been caused by the problem there were then two entries in the Address Book List and I could no longer tell which was the original "problem" one and which was the one that I later filled out.

Comment 11

[hgruenberg]

Additional Comment When I wrote "definitely true" I meant that it had been in the Address Book tree but not in the Mailing List.  That was the original cause of the problem because I tried to drag that Address Book entry to the Mailing List.

Comment 12

[Jorg K (CEST = GMT+2)]

Hmm, the whole thing is very confusing. I had the impression a while ago, that at times, the copy of the dragged card in the ML didn't coincide with the original card. In fact, an old version seemed to come back to life. I could only reproduce the problem with the steps given in comment #7.

If you can reproduce the problem "copy in ML != original" in some other form, please write down the exact steps.

I'm sure that when we get around to fixing the bug, other scenarios with the same root cause will be fixed, too.

Comment 13

[hgruenberg]

At the moment the Address Book tree contains the two alike entries, one of which, as I have said, probably contains an invisible quirk that caused the original problem.  Now, to reproduce the problem I would have to erase the one that I augmented subsequently and also erase the copy in the ML (although that may be erased automatically - I don't know.).  The problem is that since I don't know which one it is I may mistakenly erase the one that originally caused the problem, and if so, and it does contain an invisible character or byte,  I can't resurrect it.

Comment 14

[hgruenberg]

Further strange behavior:  I had occasion to make more changes to the ML.  First I deleted an entry from the ML tree and then added a new name to the ML Properties but without an e-mail address.  When I pressed OK the ML tree "blinked" but nothing changed.  I thought that maybe without an e-mail address the name would not be added to the ML tree.  So, I went back into the ML Properties and added an e-mail address to the previous entry.  I pressed OK again.  Still no effect in the ML tree.  So I checked the Address 
Book tree.  There I found two new entries - the first was just the added name and the second was the added name with the e-mail address.

Comment 15

[hgruenberg]

There's no end to this.  I removed the 2 entries from the Address Book tree.  Then I created a new card for this entry with all the information except an e-mail address (because I don't have it). I then dragged the Address Book entry from the tree to the ML.  (Just to be sure, what happened is that the ML became selected and a small document icon with a dotted outline and an arrow appeared pointing to the ML. I released the button and the ML became unselected and the icon disappeared.  That, I assume is supposed to indicate the copy has been completed)  I looked at the ML tree.  No entry.  I looked at the ML Properties.  No entry.  This ML is different from the one with the problem we have been discussing previously.

Comment 16

[Jorg K (CEST = GMT+2)]

Thanks for reporting some more issues. The address book is not the best piece of software in TB.

We'll take a very thorough look at what happens when your drap/drop cards from elsewhere onto the mailing list. I'm sure there is a blatant bug somewhere which will explain all the strangeness.

Right now, we're preparing for the new release TB 52 ESR due out in March 2017. Once we're less busy, we'll dedicate some time to this. So don't expect any immediate action here.

Comment 17

[Thomas D. (:thomas8)]

(In reply to Jorg K (GMT+1) from comment #16)

> We'll take a very thorough look at what happens when your drap/drop cards
> from elsewhere onto the mailing list. I'm sure there is a blatant bug
> somewhere which will explain all the strangeness.

Speaking of blatant bugs, among them is the inherently flawed design of mailing lists...

[Bug 757736] [Meta] Relationship of mailing lists with address book cards is conceptually flawed - explore improvement options (e.g. tagging system)

Comment 18

[Thomas D. (:thomas8)]

Another bug to look at is this:

Bug 758969 - [Meta] Mailing Lists: Bugs caused by problem: Contacts list pane of contained contacts not updated/refreshed (artefacts, duplicate entries, apparent and real dataloss of cards, display names etc.)

Although that *might* have been fixed in the meantime.
Find lots of related bugs all linked up from these two meta bugs. Courtesy of a responsible volunteer bug manager.

Comment 19

[Jorg K (CEST = GMT+2)]

Thanks for the hint re. the meta-bug. I saw some funny behaviour (which I cannot recall exactly now). I think, when clicking on an address book, I saw a contained card and a contained ML in right-hand pane. And when I edited the card (or the ML?) it looked like a duplicate entry and the other entry had disappeared, or "visually" been overwritten be the edited card(ML?).

Comment 20

[Wayne Mery (:wsmwk)]

(untriaged component is bad place for a confirmed bug - users who watch the proper component don't get to see the bug, plus we get fewer good eyes to help resolve it)

Comment 21

[Jorg K (CEST = GMT+2)]

OK, finally after many months I'm getting back to this and executing the STR from comment #7:

STR:
- Create a new address book
- Create an entry: First: First Last: Last
  Email: [e-mail], needs to be a new e-mail never seen before
- Create a new mailing list in a different address book
- Drag the entry to the ML and receive a perfect copy
- Delete the entry from the ML.
- Edit the entry, add Work: 8888 and Home: 7777
- Now drag the entry again.
Result: The entry that was dragged to the ML doesn't match its original.

Here is why:
We're at nsAbMDBDirectory.cpp#731:
https://dxr.mozilla.org/comm-central/rev/db4917476acb50edbc625638112f8e22c65e025d/mailnews/addrbook/src/nsAbMDBDirectory.cpp#731
m_IsMailList==true since we're dropping onto a mailing list, needToCopyCard==true.
Then we call mDatabase->FindRowByCard() and that, believe it or not, returns the deleted card which at line 740 overwrites the details of the card being dragged.

So if you drag a card onto a mailing list, the copy will get the details from the dead card, not the card being dragged.

The key to this is in nsAddrDatabase::FindRowByCard() which calls nsAddrDatabase::GetRowForCharColumn().

And there we read a very confusing comment (also missing punctuation):
https://dxr.mozilla.org/comm-central/rev/db4917476acb50edbc625638112f8e22c65e025d/mailnews/addrbook/src/nsAddrDatabase.cpp#3247

  // see bug #198303
  // the addition of the m_mdbDeletedCardsTable table has complicated life in the addressbook
  // (it was added for palm sync).  until we fix the underlying problem, we have to jump through hoops
  // in order to know if we have a row (think card) for a given column value (think email=[e-mail])
  // there are 4 scenarios:
  //   1) no cards with a match
  //   2) at least one deleted card with a match, but no non-deleted cards
  //   3) at least one non-deleted card with a match, but no deleted cards
  //   4) at least one deleted card, and one non-deleted card with a match.
  //
  // if we have no cards that match (FindRow() returns nothing), we can bail early
  // but if FindRow() returns something, we have to check if it is in the deleted table
  // if not in the deleted table we can return the row (we found a non-deleted card)
  // but if so, we have to search through the table of non-deleted cards
  // for a match.  If we find one, we return it.  but if not, we report that there are no
  // non-deleted cards.  This is the expensive part.  The worse case scenario is to have
  // deleted lots of cards, and then have a lot of non-deleted cards.
  // we'd have to call FindRow(), HasRow(), and then search the list of non-deleted cards
  // each time we call GetRowForCharColumn().
  if (!aRowPos && !HasRowButDeletedForCharColumn(unicodeStr, findColumn, aIsCard, aFindRow))

Debugging in nsAddrDatabase::HasRowButDeletedForCharColumn() I can see that a a "deleted table" doesn't exist and InitDeletedCardsTable(false); also doesn't initialise/create it.

So since this table doesn't exist, the deleted row is returned as live.

I assume that to solve the riddle here, one will need to follow the life of the "deleted table" to see where it went missing.

Sadly all this code predates the import into Mercurial in 2008 so we have no documentation. Bug 198303, attachment 121612 [details] [diff] [review], indeed landed this code in 2003. I guess the code isn't even wrong, it's just that it's build on shifting sand, that "deleted table" that has disappeared.

If we can't tell reliably whether a row/card is dead or live, we indeed have a problem that will cause many bugs.

Comment 22

[Jorg K (CEST = GMT+2)]

I've heard on IRC that the palm sync stuff "was killed several years ago", so perhaps time to remove the "deleted table".

Comment 23

[Jorg K (CEST = GMT+2)]

Attachment: 1329295-deleted-cards-table.patch - WIP (v1)

This removes the "deleted cards table". That doesn't fix the bug yet, but it removes unnecessary complication.

Comment 24

[Jorg K (CEST = GMT+2)]

Attachment: 1329295-deleted-cards-table.patch (v2).

Comment 25

[Jorg K (CEST = GMT+2)]

While playing around with this I noticed the following:
When we delete an entry from a ML in AB 1, it doesn't get deleted from AB 1.
When an entry is dragged from AB 2 onto the ML again, the matching entry from AB 1 is used. So the STR actually describe expected behaviour.

Using the existing entry is done here:
https://dxr.mozilla.org/comm-central/rev/db4917476acb50edbc625638112f8e22c65e025d/mailnews/addrbook/src/nsAbMDBDirectory.cpp#740

I think we should still remove the "deleted table", but that's about all we can do here.

Comment 26

[Jorg K (CEST = GMT+2)]

Ultimately I couldn't reproduce the original problem described here. I found something else a little unexpected (comment #7), but in the end that turned out to be expected behaviour (comment #25).

So I'm restoring the original summary and changing the bug status back to "unconfirmed". I've moved the removal of the "deleted cards" table to bug 1383617.

Comment 27

[hgruenberg]

I haven't had occasion to add an entry to the address book which I would then need to add to an existing Mailing List by dragging it (which occasioned the original bug report)so I don't know the current status of the bug.

Comment 28

[Thomas D. (:thomas8)]

Per Jorg's painstaking analysis (thank you!), and in view of the pending mailing list revamp, there's really nothing left to do here apart from acknowledging that the current (old) mailing list implementation isn't great at all.
Otherwise this bug hasn't been in a good condition from its inception.

(In reply to Jorg K (CEST = GMT+2) from comment #26)

Ultimately I couldn't reproduce the original problem described here. I found
something else a little unexpected (comment #7), but in the end that turned
out to be expected behaviour (comment #25).

So I'm restoring the original summary and changing the bug status back to
"unconfirmed". I've moved the removal of the "deleted cards" table to bug
1383617.