1920339 - Feed account: localised .hostname and .name get changed to serverNN.invalid leading to RSS feed folder corruption (after 128.2.3 upgrade)

Status: RESOLVED FIXED

(Thunderbird :: Account Manager, defect)

Description

[Johannes]

Attachment: rss.png

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:130.0) Gecko/20100101 Firefox/130.0

Steps to reproduce:

I noticed that today my RSS feed list got corrupted, and I guess this was right after the upgrade to 128.2.3 was installed.

Actual results:

The item that was previously called "RSS-News & Weblogs" now shows up as "server3.invalid". I still see all the sub folders in there, but they don't seem to contain their original feeds anymore. For example, I have a sub folder "Youtube" and this should contain several feeds for various YouTube channels, but if I right-click the folder and go to show the feed items, the list is empty. The items that were already retrieved from the RSS feeds are still present. See attached screenshot

Expected results:

RSS functionality should continue to function as before.

Comment 1

[Francesco]

Bug 1911951 introduced those server names.

Comment 2

[Johannes]

By the way, Mail\RSS-News & Weblogs\feeditems.json still contains all the feed URLs as it seems, but quite obviously Thunderbird is not polling them anymore.

Comment 3

[Francesco]

My "Blogs & News Feeds" still work in both TB 131 beta and 128.2.3, even if I rename to "RSS-News & Weblogs".

In the Config Editor, can you identity which server corresponds to your feeds and then check the values of
mail.server.serverNN.hostname and mail.server.serverNN.name.

Comment 4

[rouf]

Same here : upgrade from 128.2.2 to 128.2.3
Item "Nouvelles et Blogs" became "server6.invalid"
In my profile the files "feeds.json" and "feeditems.json" seem ok. But feeds are not working anymore as if thunderbird forgot the feeds urls.

Comment 5

[Johannes]

mail.server.server3.hostname is server3.invalid

mail.server.serverNN.name is RSS-News & Weblogs but probably only after I manually renamed it back from server3.invalid in the account settings (assuming that this is indeed the display name).

Comment 6

[Francesco]

Can you change hostname to Feeds, that's what I have. Do you have a backup to see what the name/hostname was originally.

Comment 7

[Johannes]

"Feeds" doesn't fix it. From your description I assume that the hostname used to be identical to the display name of the folder? Because if I set the hostname to "RSS-Feeds & Weblogs" (which is, or was at the time I installed Thunderbird at least, the default name of this folder for German installation), it automatically is corrected to "server3.invalid" again.

Comment 8

[Johannes]

Addition to previous comment: BOTH hostname and name are automatically corrected to "server3.invalid", not just the hostname.

Comment 9

[Johannes]

Backup from before the upgrade:

user_pref("mail.server.server3.hostname", "RSS-News & Weblogs");
user_pref("mail.server.server3.name", "RSS-News & Weblogs");

To me this sounds like Thunderbird indeed no longer likes the server name that it itself chose for this folder.

Comment 10

[Francesco]

That hostname is invalid and 128.2.3 will turn it into serverNN.invalid, as you saw. The account name should not be affected. And a feed account doesn't have a server, the subscriptions are handled elsewhere.

We'll have to wait until a developer analyses it. Looks like we have all the needed facts.

Comment 11

[Francesco]

I can reproduce. If I edit prefs.js and set these values:
user_pref("mail.server.server1.hostname", "RSS-News & Weblogs");
user_pref("mail.server.server1.name", "RSS-News & Weblogs");
after the next start, I get both mail.server.server1.name and mail.server.server1.hostname set to server1.invalid.
And the feeds functionality is broken, the subscribe dialog shows: Store articles in: server1.invalid.

However, correcting the .hostname pref to value Feeds and the .name pref to the original string fixes the issue for an English version, since Feeds was the original value of .hostname. Of course trying to change the value back to RSS-News & Weblogs will just trigger the error again. So the workaround would be to change .hostname to Feeds and rename the folder that stores the feeds to match.

Comment 12

[Johannes]

Do you mean the folder inside the Mail folder? Renaming that doesn't seem to fix it for me.

Comment 13

[Francesco]

I can't tell since I don't have a German installation. Unfortunately this bug constitutes dataloss. In my installation it's stored in Mail\Feeds, so I would set the .hostname to Feeds and rename the folder, likely called Mail\RSS-News & Weblogs to Mail\Feeds. Or go back to 128.2.1, there you can just restore the original value of the prefs.

Comment 14

[gaugg]

Unfortunately, this error also occurs on my Thunderbird installation since the upgrade from Mozilla Thunderbird 128.2.1 to Mozilla Thunderbird 128.2.3. Thunderbird can no longer resolve and display the feed addresses, although the addresses are still in the “Mail\RSS-News & Weblogs\feeds.json” file.

Comment 15

[Francesco]

While TB is closed, edit the prefs file and restore
user_pref("mail.server.serverNN.hostname", "RSS-News & Weblogs");
user_pref("mail.server.serverNN.name", "RSS-News & Weblogs");

You need to search for .invalid in a text editor so you find the right NN, in comment #3 it's
user_pref("mail.server.server3.hostname", "RSS-News & Weblogs");
user_pref("mail.server.server3.name", "RSS-News & Weblogs");
but the number varies. You also need to return to TB 128.2.1 otherwise the change will be immediately undone by 128.2.3.

Comment 16

[gaugg]

Thanks for your tip and suggested solution :-)

Comment 17

[Johannes]

Thanks, I have used that workaround for now and returned to 128.2.1.

Comment 19

[kompre]

After the I experienced the bug described in https://bugzilla.mozilla.org/show_bug.cgi?id=1911951 (blank tree), I changed all the hostname of my local folder accounts so that they would not contain @ symbols, which is was caused the issue for me. That fix worked fine for me.

Now with TB has upgraded to 128.2.3esr the previous renamed local folders, have been renamed serverNN.invalid where NN is a number. This has broke some settings that relied on the name of the folder, for example the archive function could not find the proper folder and reset to default settings (I've set up that an IMAP folder would archive the messages on the twin local folder; since the local folder got renamed in the update, that setting was invalid and reverted back to default values)

So now, after having already all my local folder account to fix the blank tree issue, I have to rename again all those folders because now they are named serverNN.invalid and fix all those settings that got reset because of this.

Doesn't help that the upgrade renaming was done without notification to the user. I noticed only when I try to archive email in the local folder and noticed that it was not going where it was supposed to.

Comment 20

[Magnus Melin [:mkmelin]]

This only affects profiles that were created before Thunderbird 15, bug 750292 (over 12 years ago).
I'm not sure how we can reasonably fix it. We can fix it for profiles that had not yet upgraded past 128.2.1 by backout, BUT they would still not work in current betas because there were other significant m-c changes that made invalid hostnames completely non-workable. For "local folders" we added a special case for now, but that is not localizable, while the feed hostname was - so that solution can't really be used. Ugh!

Comment 21

[Magnus Melin [:mkmelin]]

(In reply to kompre from comment #19)
That's an add-on issue - the add-on must, like core, stop using invalid hostnames.

Comment 22

[Magnus Melin [:mkmelin]]

This should be a workaround:

• In the config editor change the hostname pref of the feeds account to Feeds
• Check where the feeds are located in your profile open the feeds.json file in a text editor (with Thunderbird closed)
• Do a global replace; replace mailbox://nobody@<whatever you had here>/ with mailbox://nobody@Feeds/

Comment 23

[Francesco]

(In reply to Magnus Melin [:mkmelin] from comment #20)

This only affects profiles that were created before Thunderbird 15, bug 750292 (over 12 years ago).

Right, in an Italian version, .hostname is Feeds and .name is Blog e feed delle news. So clearly this feed account will not have an issue. So it's only older profiles which use some string with spaces and & in the hostname. The hostname was used for the storage location (comment 14), so people would have to check whether changing the .hostname pref to Feeds and the storage location form (for example) Mail\RSS-News & Weblogs to Mail\Feeds works.

(In reply to Magnus Melin [:mkmelin] from comment #22)

This should be a workaround:

• In the config editor change the hostname pref of the feeds account to Feeds
• Check where the feeds are located in your profile open the feeds.json file in a text editor (with Thunderbird closed)
• Do a global replace; replace mailbox://nobody@<whatever you had here>/ with mailbox://nobody@Feeds/

As per my comment above, you also need to change the storage location on disk.

Comment 24

[Johannes]

Something must still be missing.

I did:

• Change server hostname to Feeds
• Closed Thunderbird
• Changed the mailbox:// links in Mail\RSS-News & Weblogs\feeds.json as described avove
• Renamed Mail\RSS-News & Weblogs to Mail\Feeds and Mail\RSS-News & Weblogs.msf to Mail\Feeds.msf
• Restarted Thunderbird 128.2.1
• Thunderbird recreates the Mail\RSS-News & Weblogs folder, feeds are empty

I think the following actions are also necessary:

• Change mail.serverNN.directory from %appdata%\Thunderbird\Profiles\profileID.default\Mail\RSS-News & Weblogs to %appdata%\Thunderbird\Profiles\profileID.default\Mail\Feeds
• Change mail.serverNN.directory-rel from [ProfD]Mail/RSS-News & Weblogs to [ProfD]Mail/Feeds

Comment 25

[Johannes]

Actually I think Thunderbird longer uses that RSS-News & Weblogs.msf file I referenced in step 4; it was last touched in 2012.

Comment 26

[Magnus Melin [:mkmelin]]

Right, storage would be the server<X>.directory and server<X>.directory-rel prefs.

Comment 27

[Francesco]

(In reply to Magnus Melin [:mkmelin] from comment #20)

For "local folders" we added a special case for now, but that is not localizable, while the feed hostname was - so that solution can't really be used. Ugh!

The server type for feeds is "rss" so you could skip the hostname check for those servers.

Comment 28

[Magnus Melin [:mkmelin]]

Not quite, the hostname is in the internal url we use to access items (and store them, like the feeds.json).

Comment 29

[Himanshu J Gohel[:hgohel]]

So I'm a long time user of Thunderbird so I had a very old preferences/configuration file and the name of the account that got messed up with Thunderbird 128.2.3esr was "News & Blogs". I followed the steps in previous messages and I was able to restore my feeds.

This hasn't been mentioned, but while the profile was in the buggy state, a "Podcasts" account was list of folders, but after the fix, it has disappeared. I'm suspecting that in Comment 15 by @Francesco where two lines in the prefs.js file were modified, one of them was the News & Feeds, and the other was a reference to the Podcasts folder. But I didn't have any podcasts so I haven't debugged further

Comment 30

[Rich Gray]

(In reply to Johannes from comment #24)

I think the following actions are also necessary:

• Change mail.serverNN.directory from %appdata%\Thunderbird\Profiles\profileID.default\Mail\RSS-News & Weblogs to %appdata%\Thunderbird\Profiles\profileID.default\Mail\Feeds
• Change mail.serverNN.directory-rel from [ProfD]Mail/RSS-News & Weblogs to [ProfD]Mail/Feeds

I am trying to fix my broken Feeds on TB and have done the first set of changes but can you clarify where these 2 items should be changed?
Thanks!

Comment 31

[Francesco]

In the prefs.js file, of course while TB is closed. Go through the prefs file with an editor looking for "RSS-News & Weblogs" and replace with "Feeds".

Comment 32

[Rich Gray]

(In reply to Francesco from comment #31)

In the prefs.js file, of course while TB is closed. Go through the prefs file with an editor looking for "RSS-News & Weblogs" and replace with "Feeds".

Thanks! That was the missing piece of the jigsaw

Comment 33

[Magnus Melin [:mkmelin]]

Attachment: Bug 1920339 - migrate invalid feed hostnames. r=#thunderbird-reviewers

This migrates invalid feeds names to use our standard Feeds.
I found even Thunderbid 5.0 uses "Feeds" at least normally... so might not be super common.

For testing, with Thunderbird closed set the hostname to "RSS-News & Weblogs" and edit feeds.json for that account to have
destFolder like "mailbox://nobody@RSS-News & Weblogs/comm-central%20Changelog".
After starting, the hostname should be corrected, and the feeds.json contain good url.

Comment 34

[Francesco]

Does that patch also rename the storage folder under Mail/ ?

Comment 35

[Magnus Melin [:mkmelin]]

No, that shouldn't be necessary, and we'd then have to also change the server<X>.directory and server<X>.directory-rel prefs.

Comment 36

[Francesco]

Thanks for the clarification, so that was only necessary when changing the prefs file wholesale replacing all occurrences.

Comment 38

[Deneice]

What I did to get back all my Feed folders :

1. Close Thunderbird

2. Open your prefs.js file. Find ALL old references that looks like either with the erroneous nameid (ex: "server6.invalid") that appears after the last update or the name of the folder that hosts your feeds (ex: "Blog et nouvelles" in my french case). Replace them by "Feeds".

Once you have done it, you should have something similar to those lines with your own server number, mine is 6 but you should have a different one :
user_pref("mail.server.server6.directory", "C:\yourInstallationDirectory\Profiles\Thunderbird\yourProfileName\Mail\Feeds");
user_pref("mail.server.server6.directory-rel", "[ProfD]Mail/Feeds");
user_pref("mail.server.server6.hostname", "Feeds");
user_pref("mail.server.server6.spamActionTargetAccount", "mailbox://nobody@Feeds");

You could keep this Line "as is". It's the name that appears in your folder list in Thunderbird. There is no incidence to change it with other thing...
user_pref("mail.server.server6.name", "Blog et nouvelles");

Save and close the file.

3. Rename your feed folder that could be called "RSS-News & Weblogs" or "Blog et nouvelles" in french and give it the name "Feeds"

4. Open the file "feeds.json" located inside this folder and replace all OLD references founded after the nametag "destFolder" with "Feeds" .
   This should look like that for a Feed called for example MyFeed :

before :  "destFolder":"mailbox://nobody@Blog%20et%20nouvelles/MyFeed"
after : "destFolder":"mailbox://nobody@Feeds/MyFeed"

Save and close the file

5. Reopen Thunderbird and Update your Feeds Folder. You should get again your Thunderbird as always ;-)

Comment 39

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/1a310ba3ec3e
migrate invalid feed hostnames. r=tobyp

Comment 41

[Magnus Melin [:mkmelin]]

Comment on attachment 9426791 [details]
Bug 1920339 - migrate invalid feed hostnames. r=#thunderbird-reviewers

[Approval Request Comment]
Regression caused by (bug #): bug 1911951 kind of (would anyway have broken a bit later due to m-c changes)
User impact if declined: feeds stop working in ancient profiles.
Testing completed (on c-c, etc.): c-c, beta
Risk to taking this patch (and alternatives if risky): some small risk, but should only affect users with the particular very old settings