Looked deeper into the problem. There are several problems. 1. When the problem with the mbox file is found, a flag is set for the message marking it as "not offine" which means the offline store (in this case the mbox file) is invalid for the message due to the bad header line containing "blacklist". This causes the message to be fetched again but this time instead of saving it to offline store, it just is saved in memory cache. However, this also fails because the the code here https://searchfox.org/comm-central/rev/08111d84bf909f3ecf367618f823ee605658472b/mailnews/imap/src/nsImapProtocol.cpp#9453 thinks the URL is "local only" meaning it is stored for offline. This causes Cancel() to be called which "dooms" the just obtained cache entry. So the problem is at this point it is not know that the message was marked as not online. So if I comment out the "if (localOnly)" block starting here https://searchfox.org/comm-central/rev/08111d84bf909f3ecf367618f823ee605658472b/mailnews/imap/src/nsImapProtocol.cpp#9449 that problem is fixed and the full message is stored to memory cache. 2. With the full message in memory cache, it is then read. However, this also fails for the same reason it failed when read from offline store: The first header line is seen as invalid. This is checked here: https://searchfox.org/comm-central/rev/08111d84bf909f3ecf367618f823ee605658472b/mailnews/imap/src/nsImapProtocol.cpp#9350 So if I comment out the "if (shouldUseCacheEntry)" block that "peeks" at the 1st 100 bytes in cache, this is also fixed. 3. With these two change and with autosync still enabled, the message is fetched correctly and the picture appears. However, autosync apparently doesn't know the message was fetched and stored to memory cache so it still downloads the message every minute or so. This causes the mbox file to grow. 4. If I turn off autosync, for some reason the first access attempts to read from memory cache. Since nothing was stored to memory cache before the attempted read, cache read fails and the picture still does not appear like before but the full message does get stored to memory cache when the message is fetched. If you move off of the message and then come back, the full message is successfully read from memory cache and the full message with the picture is displayed now. So there are several issue to resolve here to do a correct fix.
Bug 1702692 Comment 23 Edit History
Note: The actual edited comment in the bug view page will always show the original commenter’s name and original timestamp.
Looked deeper into this I see several problems. 1. When the problem with the header in the mbox file is found, a flag is set for the message marking it as "not offline" which means the offline store (in this case the mbox file) is invalid for the message due to the invalid header line containing "blacklist". This causes the message to be fetched again but this time instead of saving it to offline store, it just is saved in memory cache. However, this also fails because the the code here https://searchfox.org/comm-central/rev/08111d84bf909f3ecf367618f823ee605658472b/mailnews/imap/src/nsImapProtocol.cpp#9453 thinks the URL is "local only" meaning it is stored for offline. This causes Cancel() to be called which "dooms" the just obtained cache entry. So the problem is that it is not know that that message was marked as "not offline". So if I comment out the "if (localOnly)" block starting here https://searchfox.org/comm-central/rev/08111d84bf909f3ecf367618f823ee605658472b/mailnews/imap/src/nsImapProtocol.cpp#9449 that problem is fixed (actually hidden) and the full message is stored to memory cache. Of course this is not a valid "fix". 2. With the full message in memory cache, it is then read. However, this also fails for the same reason it failed when read from offline store: The first header line is seen as invalid. This is checked here: https://searchfox.org/comm-central/rev/08111d84bf909f3ecf367618f823ee605658472b/mailnews/imap/src/nsImapProtocol.cpp#9350 So if I comment out the "if (shouldUseCacheEntry)" block that "peeks" at the 1st 100 bytes in cache, this is also fixed. (Again, not a valid fix.) 3. With these two change and with autosync still enabled, the message is fetched correctly and the picture appears. However, autosync apparently doesn't know the message was fetched and stored to memory cache so it still downloads the message every minute or so. This causes the mbox file to grow. 4. If I turn off autosync, for some reason the first access attempts to read from memory cache. Since nothing was stored to memory cache before the attempted read, cache read fails and the picture still does not appear like before but the full message does get stored to memory cache when the message is fetched. If you move off of the message and then come back, the full message is successfully read from memory cache and the full message with the picture is displayed now. So there are several issue to resolve here to do a correct fix.
Looking deeper into this I see several problems. 1. When the problem with the header in the mbox file is found, a flag is set for the message marking it as "not offline" which means the offline store (in this case the mbox file) is invalid for the message due to the invalid header line containing "blacklist". This causes the message to be fetched again but this time instead of saving it to offline store, it just is saved in memory cache. However, this also fails because the the code here https://searchfox.org/comm-central/rev/08111d84bf909f3ecf367618f823ee605658472b/mailnews/imap/src/nsImapProtocol.cpp#9453 thinks the URL is "local only" meaning it is stored for offline. This causes Cancel() to be called which "dooms" the just obtained cache entry. So the problem is that it is not know that that message was marked as "not offline". So if I comment out the "if (localOnly)" block starting here https://searchfox.org/comm-central/rev/08111d84bf909f3ecf367618f823ee605658472b/mailnews/imap/src/nsImapProtocol.cpp#9449 that problem is fixed (actually hidden) and the full message is stored to memory cache. Of course this is not a valid "fix". 2. With the full message in memory cache, it is then read. However, this also fails for the same reason it failed when read from offline store: The first header line is seen as invalid. This is checked here: https://searchfox.org/comm-central/rev/08111d84bf909f3ecf367618f823ee605658472b/mailnews/imap/src/nsImapProtocol.cpp#9350 So if I comment out the "if (shouldUseCacheEntry)" block that "peeks" at the 1st 100 bytes in cache, this is also fixed. (Again, not a valid fix.) 3. With these two change and with autosync still enabled, the message is fetched correctly and the picture appears. However, autosync apparently doesn't know the message was fetched and stored to memory cache so it still downloads the message every minute or so. This causes the mbox file to grow. 4. If I turn off autosync, for some reason the first access attempts to read from memory cache. Since nothing was stored to memory cache before the attempted read, cache read fails and the picture still does not appear like before but the full message does get stored to memory cache when the message is fetched. If you move off of the message and then come back, the full message is successfully read from memory cache and the full message with the picture is displayed now. So there are several issue to resolve here to do a correct fix.