TB 86.0b1: server.spamSettings.spamFolderURI returns a garbled string
Categories
(Thunderbird :: Add-Ons: Extensions API, defect)
Tracking
(Not tracked)
People
(Reporter: knu, Unassigned)
Details
User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 11_2_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.96 Safari/537.36
Steps to reproduce:
I've been using the tbkeys (https://github.com/wshanks/tbkeys) extension for scripting, and an expression folder.server.spamSettings.spamFolderURI
(where folder
is a folder object obtained via gFolderDisplay.displayedFolder
) would return a valid UTF-8 encoded string in earlier versions of Thunderbird (I don't recall, but at least in 78.0b1), but after upgrading to Thunderbird 86.0b1 today it returns a garbled string.
Actual results:
"imap://xxxxx@imap.googlemail.com/[Gmail]/è¿·æ\u0083\u0091ã\u0083¡ã\u0083¼ã\u0083«"
Expected results:
"imap://xxxxx@imap.googlemail.com/[Gmail]/迷惑メール"
"迷惑メール" is Japanese for "Spam", whose UTF-8 representation is [e8 bf b7] / [e6 83 91] / [e3 83 a1] / [e3 83 bc] / [e3 83 ab], so the actual results above very much look like the UTF-8 value interpreted as ISO-8859-1 and then transcoded to UTF-8.
I'm not sure if a URI should include non-ASCII characters or should be percent-encoded, but that's another issue.
Comment 1•3 years ago
|
||
Thanks for reporting this. Could you check if this issues still exists in the current Thunderbird Daily and if the returned spamFolderURI
differs from gFolderDisplay.displayedFolder.folderURL
if you select the spam folder?
Are you having any issues besides the URL looking different? Is the URL not working? What are you using it for?
Could you run mozgeression to identify the changeset, which may have introduced this, so we can check if it was intended?
https://mozilla.github.io/mozregression/install.html
Reporter | ||
Comment 2•3 years ago
|
||
Thanks for taking a look. I can't test this in Thunderbird Daily because the tbkeys extension does not seem to install on 90.0a1. (I'm not an extension developer myself)
I forgot to mention in the original report that URI
and folderURL
properties return different representations of the same folder. I can confirm this with 89.0b2.
gFolderDisplay.displayedFolder.URI == "imap://xxxxx@imap.googlemail.com/[Gmail]/迷惑メール"
This is the valid UTF-8 representation of the folder name, but it's questionable if it's a valid URI.
gFolderDisplay.displayedFolder.folderURL == "imap://xxxxx@imap.googlemail.com/%5BGmail%5D/%E8%BF%B7%E6%83%91%E3%83%A1%E3%83%BC%E3%83%AB"
This is a valid URL with the correct UTF-8 representation of the folder name properly percent-encoded to put into the path part of a URI.
```javascript
gFolderDisplay.displayedFolder.server.spamSettings.spamFolderURI == "imap://akinori%40g.musha.org@imap.googlemail.com/[Gmail]/è¿·æ\u0083\u0091ã\u0083¡ã\u0083¼ã\u0083«"
This is, as reported, a garbled version of the folder name in UTF-8.
What I want to do is to check if I'm in the spam folder or not by comparing gFolderDisplay.displayedFolder.URI
with gFolderDisplay.displayedFolder.server.spamSettings.spamFolderURI
, so whether the return values are valid as URI is not important for the purpose and they just need to be consistent, but I think these above should all return the percent-encoded version because a URI/URL should be in ASCII encoding considering interoperability.
Comment hidden (typo) |
Reporter | ||
Comment 4•3 years ago
|
||
Sorry for the duplicated comments. Also I failed to redact the userinfo part of the third URI. 😣
Comment 5•3 years ago
|
||
For Thunderbird both these URIs are identical:
imap://xxxxx@imap.googlemail.com/%5BGmail%5D/%E8%BF%B7%E6%83%91%E3%83%A1%E3%83%BC%E3%83%AB
imap://xxxxx@imap.googlemail.com/[Gmail]/迷惑メール
You can get the second from the first by piping it thru decodeURI(). I cannot tell you why the spamFolderURI returns the url after it has been piped through encodeURI(). But it does not look like a bug to me.
Can you confirm, that this fixes your issue?
Comment 6•3 years ago
|
||
Do you want me to contact bugzilla admins to remove the comment with your private data?
Reporter | ||
Comment 7•3 years ago
|
||
No, the problem is that the value of spamFolderURI is broken. I need a way to get a valid URI that represents the spam folder of an account.
Comment 8•3 years ago
|
||
It is not broken, it is just in a different representation. You can convert as described in Comment 5.
Reporter | ||
Comment 9•3 years ago
|
||
I've reported that spamFolderURI returns imap://xxxxx@imap.googlemail.com/[Gmail]/è¿·æ\u0083\u0091ã\u0083¡ã\u0083¼ã\u0083«
in Thunderbird 86.0b1 and 89.0b2. It is obviously broken. How would you transcode it back to imap://xxxxx@imap.googlemail.com/%5BGmail%5D/%E8%BF%B7%E6%83%91%E3%83%A1%E3%83%BC%E3%83%AB
or imap://xxxxx@imap.googlemail.com/[Gmail]/迷惑メール
?
Do you mean it's fixed in a recent Thunderbird Daily?
Comment 10•3 years ago
|
||
Sorry, I misread your Comment 2. Back to square one.
I currently fail to reproduce this. When I rename my Spam folder to 迷惑メール I get back a totally different imap uri from google. So yes, it would be great if you could verify that in Daily. You do not need the add-on for this test. Open the error console (Ctrl + shift + J) and just enter the two vaiables and you can see its value. A fresh profile would be good though.
Thanks,
John
Reporter | ||
Comment 11•3 years ago
|
||
Thanks for the information. I created a new profile on Thunderbird 90.0a1 and tried an error console session. Unfortunately, the issue seems to persist.
>> gFolderDisplay.displayedFolder.server.spamSettings.spamFolderURI
"imap://*****@imap.gmail.com/[Gmail]/è¿·æ\u0083\u0091ã\u0083¡ã\u0083¼ã\u0083«"
>> gFolderDisplay.displayedFolder.URI
"imap://*****@imap.gmail.com/[Gmail]/迷惑メール"
>> gFolderDisplay.displayedFolder.folderURL
"imap://*****@imap.gmail.com/%5BGmail%5D/%E8%BF%B7%E6%83%91%E3%83%A1%E3%83%BC%E3%83%AB"
However, I managed to find a way to decode the garbled value.
>> ungarbled = decodeURI(gFolderDisplay.displayedFolder.server.spamSettings.spamFolderURI.replace(/[^\u0000-\u007f]/g, c => '%' + c.charCodeAt(0).toString(16).toUpperCase().padStart(2, '0')))
"imap://akinori%40g.musha.org@imap.gmail.com/[Gmail]/迷惑メール"
>> ungarbled === gFolderDisplay.displayedFolder.URI
true
This is far from ideal but can be used as a tentative workaround for me.
Comment 12•3 years ago
|
||
While I am looking at the source, could you use mozregression to find when this change was introduced? The tool is very simple. It allows to specify a date range which you want to check and it downloads daily snapshots of Thunderbird and starts them after the download has finished. You repeat the test you just did for TB90 with each snapshot. After closing the snapshot TB you can declare the test to be "good" or "bad" and the tool will find the exact moment/changeset this regression was introduced.
https://mozilla.github.io/mozregression/install.html
The only settings you need to watch out for is to select Thunderbird, the comm-central repository and on the profile page select reuse, so you do not have to setup gmail for each test.
For some strange reason FF and Chrome report a virus when downloading, but that is a false positive. I use that tool to find regression but this time I cannot, as I cannot reproduce it here.
Thanks for your help.
Comment 13•3 years ago
|
||
I can organize a zoom call so I could guide you thru the process, but I think this bug will not move forward without the information from mozregression.
Comment 14•3 years ago
|
||
Since nothing in Thunderbird itself is broken, but just the returned value does not match what you expected, I close this as invalid.
Reporter | ||
Comment 15•3 years ago
|
||
I believe that the values of spamFolderURI and URI are invalid as URI, and Thunderbird is violating RFC 3986 there.
I have an ugly but working workaround as above and I don't have the time to track this down for now. Thank you for your time.
Description
•