1789990 - Address Book "Screen Name" data lost in TB v102.2.1 update

Status: VERIFIED FIXED

(Thunderbird :: Address Book, defect)

Description

[marty]

Steps to reproduce:

TB automatically updated to version 102.2.1.
(TB is on the automatic release update channel.)

Actual results:

The update involved an address-book revamp.
The new address book is missing the "Screen Name" field in its new data model.
That field was in use in the old address book. It held important data to me.
Now the data is missing. It has not been migrated.

Expected results:

The "Screen Name" data in the old data model should have been migrated to the "IMPP" field in the new data model. Now it is lost.
Before the lossy migration of the data from the the old address book data model to the new post v102.2.1 data model. All the old address books data should have been backed-up in a .csv files.

Comment 1

[marty]

For more information see: http://forums.mozillazine.org/viewtopic.php?f=39&t=3100335

Comment 2

[marty]

Attachment: update path

The update path that was taken to arrive at this point.

Comment 3

[Wayne Mery (:wsmwk)]

The old AB files still exist.

Comment 4

[Magnus Melin [:mkmelin]]

The data is also there, it's just that we don't display it.

Comment 5

[Magnus Melin [:mkmelin]]

Attachment: Bug 1789990 - Show/migrate old chat fields over to IMPP uris. r=freaktechnik

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/ced6e0d8aed7
Show/migrate old chat fields over to IMPP uris. r=freaktechnik

Comment 7

[marty]

Seems you fixed it for all future users. That's great.
Now please help me too.
A week ago I mailed Wayne about his comment, but unfortunately I didn't get a reply. I'll ask you the same questions.

Wayne wrote: "The old AB files still exist.".

Could you maybe explain to me or point me in the right direction?

How did you come to the conclusion that the old Address Book files still exist?
Where are the old Address Book files located?
What are the old Address Book files? (file names and file types?)
How can I export them to a .csv to recover my data?

When I look in TB's profile folder, these are the files, that I find:

$ ls abook*
abook-1.sqlite abook-2.sqlite abook-3.sqlite abook.sqlite
abook-1.sqlite-shm abook-2.sqlite-shm abook-3.sqlite-shm abook.sqlite-shm
abook-1.sqlite-wal abook-2.sqlite-wal abook-3.sqlite-wal abook.sqlite-wal
abook-1.v2.sqlite abook-2.v2.sqlite abook-3.v2.sqlite abook.v2.sqlite
abook-1.v3.sqlite abook-2.v3.sqlite abook-3.v3.sqlite abook.v3.sqlite

Could you please help me in recovering the old Address Book data so I can save my data which I painstakingly collected over many years?

Comment 8

[Magnus Melin [:mkmelin]]

So the data is still there, it didn't change. There is nothing to recover. All that is needed is (was) for the UI to show it.
I plan on getting the patch uplifted to 102 in some weeks. Likely for 102.4.

Comment 9

[marty]

(In reply to Magnus Melin [:mkmelin] from comment #8)

So the data is still there, it didn't change. There is nothing to recover. All that is needed is (was) for the UI to show it.
I plan on getting the patch uplifted to 102 in some weeks. Likely for 102.4.

Thanks for your reply. How do I get my data out in .CVS format?

I tried to export the address book data to a CSV format, but nothing happened.

You can click it, but nothing is exported...

I tested all the export formats (csv, tab/txt, ldif) and none worked except for vcard format (.vcf).

I did an analysis of the vcard data that was exported and compared it to an old pre v102.2.1 .CSV export in order to derive what information appears to be missing:

Old pre v102.2.1 data fields in an old .csv export (and occurence in post v102.2.1 vcard data export):
1 First Name
2 Last Name
3 Display Name
4 Nickname
5 Primary Email
6 Secondary Email
7 Screen Name (MISSING post v102.2.1)
8 Work Phone
9 Home Phone
10 Fax Number
11 Pager Number
12 Mobile Number
13 Home Address
14 Home Address 2
15 Home City
16 Home State
17 Home ZipCode
18 Home Country
19 Work Address
20 Work Address 2
21 Work City
22 Work State
23 Work ZipCode
24 Work Country
25 Job Title
26 Department
27 Organization
28 Web Page 1
29 Web Page 2
30 Birth Year MISSING (in another format now)
31 Birth Month MISSING (in another format now)
32 Birth Day MISSING (in another format now)
33 Custom 1
34 Custom 2
35 Custom 3
36 Custom 4
37 Notes

I reverse engineered the new data fields that seem to have appeared in the new data model in the VCARD export:
New fields as it seems post v102.2.1:

Chat Accounts (+): IMPP:TEST_chat_Accounts, IMPP:TEST_chat_Accounts2
(Org) Role: ROLE:TEST_Role
Phone Numbers (+): TEL;VALUE=TEXT:TEST_phone_no_none
Email (+): EMAIL:TEST-Add_email_address
Website (+): URL:https://TEST-WEBSITE

(the + denotes 1 or more entries possible)

So there you have it. IMPP fields are unpopulated and the "Screen Name" field is missing in v102.2.1.

If the "Screen Name" data is still there, then I can't get to it, but the code change in #6 suggests otherwise.
How can I export all the (pre v102.2.1) data to a .csv format?

Comment 10

[Wayne Mery (:wsmwk)]

Marty, thanks for the excellent analysis.

How can I export all the (pre v102.2.1) data to a .csv format?

Right+click the address book > Export

The fix for this bug will appear in 106 beta in the next several days.
For version 102, I would hope in 2-3 weeks - 102.3.1 or 102.3.2

Comment 11

[marty]

(In reply to Wayne Mery (:wsmwk) from comment #10)

Right+click the address book > Export

I tried exactly that many times, but nothing happens in v102.2.1 on Win 10.
Here's a screenshot: https://i.postimg.cc/13LZs2xP/image.png
None of the export formats (csv, tab/txt, ldif) work except for vcard format (.vcf) which doesn't contain Screen Name / IMPP data.

Thanks for pointing to the intended release candidate. (I'm holding back updates for now until I can same my full address books data. Which I don't want to loose.)

Comment 12

[Wayne Mery (:wsmwk)]

1. Check the error console after attempting export for failure information
2. If you are on version 102, I encourage you to try 102.3.0 because there are address book fixes that may help the export.
3. I think my comment 10 is imprecise, I must have meant that the fix for screen name should be in 2-3 weeks - 102.3.1 or 102.3.2

Comment 13

[marty]

(In reply to Wayne Mery (:wsmwk) from comment #12)

1. Check the error console after attempting export for failure information

I checked the error console after trying to export an addressbook in .csv format (the default export format).
There I find the following error:

Error: Could not extract integer from "NaN-" Ical.jsm:154:13

If you are on version 102, I encourage you to try 102.3.0 because there are address book fixes that may help the export.

I'm on version v102.2.1 and am holding back further updates. I'm very worried about loosing precious addressbook data. They are actively maintained and were in pristine condition. I took a lot of work to maintain. It's my master database for addresses that contain updates, that when lost, I can't recover any more. There's hundreds of entries. I don't dare doing any more updates until I get my data saved to .csv first.

Comment 14

[Wayne Mery (:wsmwk)]

(In reply to marty from comment #13)

(In reply to Wayne Mery (:wsmwk) from comment #12)

1. Check the error console after attempting export for failure information

I checked the error console after trying to export an addressbook in .csv format (the default export format).
There I find the following error:

Error: Could not extract integer from "NaN-" Ical.jsm:154:13

If you are on version 102, I encourage you to try 102.3.0 because there are address book fixes that may help the export.

What you describe is bug 1789793, fixed in 102.3.0 as I mention above.

Comment 16

[Magnus Melin [:mkmelin]]

Comment on attachment 9294179 [details]
Bug 1789990 - Show/migrate old chat fields over to IMPP uris. r=freaktechnik

[Approval Request Comment]
Regression caused by (bug #): new ab
User impact if declined: can't see chat fields
Testing completed (on c-c, etc.): c-c, beta
Risk to taking this patch (and alternatives if risky): not very risky. needs bug 1791347 as well

Comment 17

[Wayne Mery (:wsmwk)]

Comment on attachment 9294179 [details]
Bug 1789990 - Show/migrate old chat fields over to IMPP uris. r=freaktechnik

[Triage Comment]
approved for esr102

Comment 18

[Rob Lemley [:rjl]]

This bug is changing code in aboutAddressBook.js that was added in bug 1776706. That bug has not been uplifted to c-esr102 because it has string changes.

Comment 19

[Rob Lemley [:rjl]]

Thunderbird 102.3.2:
https://hg.mozilla.org/releases/comm-esr102/rev/0154c7cfd8ee