ThunderBird 68 cannot open/save attachments from IMAP accounts (unless folder is kept in sync locally)
Categories
(MailNews Core :: Attachments, defect)
Tracking
(Not tracked)
People
(Reporter: ml, Unassigned)
Details
(Keywords: regression)
Attachments
(2 files)
User Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101 Firefox/60.0
Steps to reproduce:
Create a new profile, set up an IMAP account and disable "Synchronization & Storage" -> "Keep messages ... on this computer".
Actual results:
When a message with attachment is opened some attachments are marked as "size unknown" and there's no way they can be opened or saved.
Images work (possibly because they must be displayed inline).
XMLs work (no idea why).
PDFs, zips, etc... don't work.
Expected results:
I would expect TB 68 to work as in previous versions (i.e. allow me to open or save any kind of attachments from any message).
|
||
Comment 1•5 years ago
|
||
Gene, I don't recall seeing other reports thus far, do you?
|
||
Comment 2•5 years ago
|
||
Not Gene, but this sounds similar to a few bugs in much older versions.
Andrea, what version of 68 are you using? The latest beta installed on my system via the built in updater is 69.0b3. This problem may have already been resolved.
|
|
||
Updated•5 years ago
|
|
Reporter | |
Comment 3•5 years ago
|
||
FreeBSD's package 68.0_1 (on amd64).
I don't think we'll see any beta here soon.
Is any 68.x version planned with this fix?
If it's solved, is there some patch I can apply to 68.0 source code? I saw no other bug regarding this, so I cannot pinpoint the commit and diff set.
|
||
Comment 4•5 years ago
|
||
I don't know of any changes that would have caused the reporter's problem. Reporter, are you saying this works on the 60.* versions but not on 68? Also, can you possibly attach a sample .eml that you save that has the problem? (I run with no offline storage too.)
There is sometimes a problem with determining the exact size of attachment. I think it is triggered by whether you have the "View" menu setting "Display attachments inline" set or not but don't remember which way causes the sometime problem. But I don't think this causes problems with downloading or saving attachments. You might try changing this setting and see if it affects the result.
|
|
Reporter | |
Comment 5•5 years ago
|
||
Absolutely: this worked until 60.* and stopped working when I upgraded to 68.0.
I'm attaching a simple mail which triggers the problem here (I run yes>a.txt - Ctrl-C - zip test.zip a.txt - attached test.zip).
Notice I previously tried with a much smaller txt (zipped as before) and that was downloaded correctly.
I have no time now to find which is the smallest size to show the problem, but I will if needed.
"Display attachments inline" doesn't seem to make any difference here.
|
|
Reporter | |
Comment 6•5 years ago
|
||
Message with zip file attached which I cannot open/download.
|
|
||
Comment 7•5 years ago
|
||
(In reply to Andrea Venturoli from comment #0)
When a message with attachment is opened some attachments are marked as "size unknown" and there's no way they can be opened or saved.
Images work (possibly because they must be displayed inline).
XMLs work (no idea why).
PDFs, zips, etc... don't work.Expected results:
I would expect TB 68 to work as in previous versions (i.e. allow me to open or save any kind of attachments from any message).
Thanks for the sample file. I have only had a chance to try it on 60.8.0 and, as you say, it worked OK and I could open the zip file and see the text file. But maybe you can tell me exactly what you mean by it won't "allow me to open or save any kind of attachment"? Exactly what error do you see when you try to open or save an attachment? Also, where are you seeing the "size unknown" that you say is prevented some attachments from opening?
Anyhow, I will try this with your sample and others on new tb version later.
|
|
||
Comment 8•5 years ago
|
||
I went ahead and download the latest linux beta 69.0b3 and ran it with the default profile. Your sample emails seems to work fine and other emails with pdf attachments also open fine. I'll find 68.0_1 like you say are using in FreeBSD but on linux and test with that now...
|
|
||
Comment 9•5 years ago
|
||
The closest I could find in the tb build archive is 68.0b1 and I don't see any problems with it either. This is with a new profile with default settings except I disabled offline storage as you describe. Your text file and messages with pdf attachments work as expected. Maybe something unique about a FreeBSD build causing it? Did you get the package from FreeBSD project? If so, have others possibly reported a similar problem?
|
||
Updated•5 years ago
|
|
||
Updated•5 years ago
|
|
|
||
Comment 10•5 years ago
|
||
Before we declare it a regression, we should confirm it, no? Magnus, can you not open attachments with TB Daily and IMAP that you're using?
|
|
||
Comment 11•5 years ago
|
||
Works fine for me on TB 68 and a non-sync IMAP folder.
|
|
||
Comment 12•5 years ago
|
||
Works fine for me.
Regression and status are different things. This is an UNCONFIRMED regression.
|
|
Reporter | |
Comment 13•5 years ago
|
||
Thanks for the sample file. I have only had a chance to try it on 60.8.0 and, as you say, it worked OK and I could open the zip file and see the text file. But maybe you can tell me exactly what you mean by it won't "allow me to open or save any kind of attachment"? Exactly what error do you see when you try to open or save an attachment? Also, where are you seeing the "size unknown" that you say is prevented some attachments from opening?
I'm seeing no error: the attachment are listed as usual in the lower part of the window under the message body; they usually have the size just right of the filename, but in my case there's "size unknown". Right clicking on them brings up the usual menu (Open, Save as, Detach, Delete), but all elements are greyed out and can't be clicked.
Maybe something unique about a FreeBSD build causing it? Did you get the package from FreeBSD project? If so, have others possibly reported a similar problem?
I've asked on the FreeBSD mailing lists if anyone is experiencing the same problem or can try to reproduce it. I'll keep you updated.
|
|
||
Comment 14•5 years ago
|
||
Gene, since the user is using TB 68 on a non-sync folder, I'm sure the new cache debugging can help diagnose the problem. Also we need to check that the cache size preferences don't have any funny values.
|
|
||
Comment 15•5 years ago
•
|
||
Also we need to check that the cache size preferences don't have any funny values.
In tb's config editor, check to make sure these are default (not bold and the values are as shown):
browser.cache.memory.capacity 200000
browser.cache.memory.enable true
browser.cache.memory.max_entry_size 25000
Since you made a new profile it is doubtful that these have changed from default. However, if the attachment is larger than 25M bytes there may be issues.
See https://wiki.mozilla.org/MailNews:Logging for info on generating a log file. It doesn't mention how-to for freebsd but I assume it is similar to macos or linux, e.g, in a terminal do.:
export MOZ_LOG=IMAP:5,timestamp,IMAPCache:5
export MOZ_LOG_FILE=/tmp/imap.log
thunderbird
I've added IMAPCache as Jorg suggested. Before running tb with the env vars set, I would suggest creating a not sync'd folder with just a problem email in it, then shutdown tb and restart with env vars set and access the problem email and try to open the problem attachment. Then shut down tb and attach the log using the "Attach file" link above. This may tell us something.
|
|
||
Comment 16•5 years ago
|
||
(In reply to Andrea Venturoli from comment #6)
Created attachment 9088238 [details]
Test.emlMessage with zip file attached which I cannot open/download.
Maybe you mean I should unzip the the attachment and re-attach the unzipped file? When I unzip the the attachment it is more than 30M in size which exceeds the 25M max cache entry size. That really shouldn't actually be a problem except it should just be too big to cache and should always be fetched from imap server when accessed. But I will test it like that.
|
|
||
Comment 17•5 years ago
|
||
Seems to work for me even with a 40M attachment (which is what the attachment unzips to). I am testing with personal trunk build that was recently pulled and updated. I did have to find an imap server that would allow such a large email (my personal dovecot) since some (gmail) refused to save it.
I do see a lot of imap fetches for the email due to no offline storage and exceeding the cache entry allocation, but otherwise, it works fine and I can open the attached a.txt file into a text editor (vim) or save it (nothing grayed out and sizes are shown).
Note: Even when I cut down the size of the attached a.txt to 20M I see more fetching than there should be. But this is a known problem that I have been working on and need to get back to: bug 1454542.
Also, I noticed that the .txt attachment doesn't display inline. I know that is fairly recent change. It is bug 1509709 which was not in 60.*. I guess there is now a pref you can set if you really want txt attachments to view inline.
|
|
||
Comment 18•5 years ago
•
|
||
If I reduce the number of lines in a.txt down to 10M it is still larger than 25M bytes on the wire so not cached and always fetched from imap. If I reduce it to 7M lines it is less than 25M bytes on the wire so it is cached. I sent the message to gmail account and in both cases with trunk build I see the attachment size but it is the actual file size and not the "wire" size transferred via imap. However, when I check gmail on 60.8 I see the "size unknown" but the attachments still seem to be downloadable.
Note: Each line of the a.txt is just a 'y' character, not 'yes' as seems to be implied in comment 5. Never mind, I see what cmd 'yes' does now.
|
|
Reporter | |
Comment 19•5 years ago
|
||
I can confirm:
browser.cache.memory.capacity=200000
browser.cache.memory.enable=true
browser.cache.memory.max_entry_size=25000
WRT to the logs you requested:
I enabled TB logging, created a new mailbox, a new ThunderBird profile and configured it; I sent it a new message with a 64kB attachment and it showed as 32kB!!! Saving it, however produced the correct 64kB file. I'm attaching the log.
I quit TB, disabled logging and rerun it: now the attachment was marked correctly as 64kB (???).
Since I previously said a new profile didn't solve, I tried again: creating a new profile, connecting it to one of my existing mailbox and descending into some folders, I cannot save attachments.
I'm so puzzled...
In any case, back to my old profile, I did some research WRT the minimal attachment size which will trigger the problem:
_ dd if=/dev/zero of=a.pdf count=1 bs=20480
_ send a.pdf to myself
_ the attachment can be saved (of couse I cannot open it since it's not a valid PDF).
With "dd if=/dev/zero of=a.pdf count=1 bs=21540", "size unknown" is displayed and the attachment cannot be accessed.
I could not reliably determine the exact threshold: probably it's not really the attachment size that matters, but the whole message size, so some small difference in headers can change the behaviour when the file is between 20 and 21 kB (corresponding to a received message size around 30kB).
Incidentally, I was also surprised to see that the problem goes away if I name the attachment a.dat (instead of a.pdf)!!!
If in a single message I attach both a.pdf and a.dat (same zero filled content), I can save a.dat (whose size TB correctly identifies), but not a.pdf (which is, again, marked "size unknown").
Back to the 30kB limit, I saw mail.imap.mime_parts_on_demand_threshold defaults to 30000. Raising this, raises the size of PDFs I can open/save.
So, unless raising it to 10000000 has some nasty side-effects, this could solve my problem.
I would also expect setting mail.imap.mime_parts_on_demand=false would achieve, but it doesn't.
Instead, setting mail.server.default.mime_parts_on_demand=false does.
(I tried this before and asserted on the ML it didn't work... either I had done something wrong or an update has changed something meanwhile).
Last thing for now, I realized FreeBSD's 68.0_1 package is build from RC3. Now 68.0_2 is out, which is based on RC6. I'm currently building it.
|
|
Reporter | |
Comment 20•5 years ago
|
||
|
|
||
Comment 21•5 years ago
|
||
I need to look closer at the log you attached and it shows the whole message being fetched. And it is fetched twice which shouldn't occur if it had been saved to ram cache after the first fetch. But that's not really the problem you report. But in the case of Attachment 9088664 [details] (IMAP log) you say it worked ok.
I tried the attachment made with "dd if=/dev/zero of=a.pdf count=1 bs=21540" and it worked ok on 60.8 but haven't tried 68.* or later yet.
I think the two variations on "mime_parts_on_demand" are there for some sort of backward compatibility. Looking at the code, the value for (newer) mail.server.default.mime_parts_on_demand, if it exists, is use when a message is displayed and supersedes the (older) mail.imap.mime_parts_on_demand. With the former, you can also define a server-specific value like mail.server.server2.mime_parts_on_demand so you can vary the setting between servers if you care. (Documentation on this is embedded in a bug report that I can't seem to find.)
Anyhow, disabling parts on demand or setting the threshold really high will cause the whole message to fetch in one piece and will probably solve your immediate problem. However, I would still like to know what is going on :).
Not familiar with freebsd. Are you building freebsd or just tb for freebsd?
|
|
Reporter | |
Comment 22•5 years ago
|
||
I think I solved, so I'm in no hurry: please take your time with the logs.
Even if I can now work properly, I'm still willing to perform any test you think might be useful.
As I announced, I upgraded to 68.0_2 aka RC6: leaving aside some new troubles :-( (which are OT now), I still need the settings above.
Strangely the were reset to default and I had to rechange them.
If mail.imap.mime_parts_on_demand is obsolete, shouldn't it go away? Just for clarity...
I'm building both FreeBSD itself and packages (like ThunderBird) myself: too long to tell here why :), but, in any case, I'm currently using no patches and no nonstandard options, so I don't think this matters.
|
|
Reporter | |
Comment 23•5 years ago
|
||
Just for the record: someone else came up having this problem on FreeBSD's mailing list.
I pointed him here and the same workaround I used seems to have solved for him too.
|
|
||
Comment 24•5 years ago
|
||
Andrea, I will just go ahead and close this with "works for me" since I can't seem to duplicate the problem and it's possible it is unique to the freebsd build. However, I will keep an eye out for this problem in the future. Feel free to report more on this if you get more information. Thanks.
|
|
Reporter | |
Comment 25•5 years ago
|
||
I've just had a report from a customer of mine who was experiencing the same problem on MacOS (10.13), so this is NOT a FreeBSD specific problem.
Of course she was using the official Mozilla-built package.
The same workaround solved for her.
|
|
||
Comment 26•5 years ago
|
||
(In reply to Andrea Venturoli from comment #25)
I've just had a report from a customer of mine who was experiencing the same problem on MacOS (10.13), so this is NOT a FreeBSD specific problem.
Of course she was using the official Mozilla-built package.
The same workaround solved for her.
Do you know if this user was on tb 60 or the new tb 68? There has been a change and other problems found due to a change that I wasn't aware of when I last posted to this bug that affects attachments. But it is only in 68. See Bug 1579341 and the bug that made the attachment change, Bug 1345167. This may or may not be related so I will look closer at this again.
Just to be sure, your workaround is to disable the fetch on demand, right?
|
|
Reporter | |
Comment 27•5 years ago
|
||
Yes to both questions.
She is using 68.0 and had no such problem with 60.8.0 (I upgraded it, hoping to solve other minor issues, but that ended up being a bad move).
Workaround was to set mail.server.default.mime_parts_on_demand to false.
(I had a look at those other two bugs, but I'm not so expert to find the correlations. Sorry.)
|
|
||
Comment 28•5 years ago
|
||
(In reply to Andrea Venturoli from comment #27)
Yes to both questions.
She is using 68.0 and had no such problem with 60.8.0 (I upgraded it, hoping to solve other minor issues, but that ended up being a bad move).
Workaround was to set mail.server.default.mime_parts_on_demand to false.
Thanks, that's good to know.
(I had a look at those other two bugs, but I'm not so expert to find the correlations. Sorry.)
No problem. Just mentioned them for reference.
|
|
||
Comment 29•5 years ago
|
||
Wayne, alta88 has an update patch for bug 1579341 that I haven't tried yet. The original change by alta88 that first appeared in 68.0 may be causing this bug, but not sure. However, I haven't been able to duplicate the comment 0 problem (which is about the same as the support question you sent me).
Yes, disabling on-demand fetches will fix this, but it is not really an acceptable fix.
|
||
Comment 31•5 years ago
|
||
Reports are coming in to the Support Forum regarding issues with Attachments that have a 'SIZE UNKNOWN' and not being able to select to open or save etc as all options are greyed out.
Workaround advice to set 'mail.server.default.mime_parts_on_demand' to false seems to working.
I have passed this info on to two different people in as many days. One was using a Linux OS and another Windows 10.
User reports:
"Since v68.0/1 email attachments report "Size Unknown" and cannot be downloaded or viewed....My O/S is Arch Linux. "
User reports:
"Since updating to Thunderbird 68.1.0 , I am unable to open or save any attachments. When right clicking on the attachment file all options are greyed out and so is the save button in the bottom right hand corner.... I'm using a PC Windows 10. It was happening with all attachments not just PDF."
User provided an image which shows a pdf document of 'Size unknown'.
|
||
Comment 32•5 years ago
•
|
||
Re: bug 1581553 (duplicate)
The mail.server.default.mime_parts_on_demand=false workaround works on my Windows 10 system. In my case the bug only occurred with one message (with a PDF attachment) in one account. If I copied the message to other IMAP accounts (including another one by the same email provider) it had no problem. There were also other messages with PDF attachments that were unaffected.
|
|
||
Comment 33•5 years ago
|
||
This is really a duplicate of bug 1581553 and both will be resolved in dependent bug 1579341. At least that's the plan.
See bug 1581553 comment 11
|
|
||
Comment 34•5 years ago
•
|
||
Making this a duplicate of bug 1579341 as per the previous comment. The fix will ship in TB 70 beta 2 and TB 68.1.1.
Description
•