Closed
Bug 428147
Opened 16 years ago
Closed 14 years ago
Using same thunderbird directory in windows and linux is now broken
Categories
(MailNews Core :: Database, defect)
Tracking
(Not tracked)
RESOLVED
DUPLICATE
of bug 462665
People
(Reporter: jd1008, Assigned: Bienvenu)
Details
User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.8.1.13) Gecko/20080325 Fedora/2.0.0.13-1.fc7 Firefox/2.0.0.13 Build Identifier: 1.5.0.14, 2.0.0.6, 2.0.0.12 On Linux (I have dual boot), /sda1 is mounted read/write after installing: ntfs-3g-1.1120-1.fc7 ntfsprogs-1.13.1-8.fc7 fuse-kmdl-2.6.23.15-80.fc7-2.7.3-8_9.fc7 fuse-libs-2.7.3-8_9.fc7 ntfsprogs-gnomevfs-1.13.1-8.fc7 fuse-2.7.3-8_9.fc7 I set up a sym link in my home dir .thunderbird -> /sda1/Documents and Settings/Application Data/Thunderbird I fire up thunderbird in. and enter my email account address (gmail account) and download my mew messages. Clicking on each new message, it is either empty, or it contains a fragment of a 2 y/o message. Always same fragment. Sometimes, same message shows multiple times in the Inbox, and each instance is identical to the other instances of same message. Booting back into windows, the msf gets re-populated, and the messages which were downloaded under the linux version of t-bird, are all there and they are OK. Since I cannot afford to have my email strewn all over creation, you need to make the format of the Inbox and Inbox.msf identical for Windows and Linux. If you have any ideas how I can get around this problem, and still use same directory for thunderbird in windows and linux, please let me know. Best regards, JD Reproducible: Always Steps to Reproduce: 1. On linux, mount your windows partition read/write 2. in your home dir, create a symlink .thunderbird which points to /mountPoint/Documents and Settings/Application Data/Thunderbird 3. fire up thunderbird, and login to your email account. 4. Send a few emails to yourself. 5. download the new email messages. 6. try to read them. Actual Results: Empty message bodies, Duplicated Inbox entries for noew messages sometimes message bodies are fragments of an old message body. When this is the case, always the same fragment appears as the body of the message(s). Expected Results: Normal and good message in Inbox I tried it with 1.5.0.14, 2.0.0.6, 2.0.0.12 and all exhibit the same behavior.
Comment 1•16 years ago
|
||
It works for me. When I follow your steps 1 - 6, I can read the messages normally. I do not see any difference in the file formats. To investigate further, you could rename your Inbox, then download some test messages into a new Inbox. If the problem persists, you could attach the files in this bug report to illustrate how the formats differ.
Status: UNCONFIRMED → RESOLVED
Closed: 16 years ago
Resolution: --- → WORKSFORME
Bug is still not resolved because you did not have an Inbox folder as large as mine - mine is 2.6GB big. I saved my Inbox, and created an empty Inbox, and sent myself new messages of 1-4 KB in size. No problems at all. Then I took another small folder containing about 100 messages, and catenated it repeatedly to a new file, until the new file size was just a hair under 2.6GB. Then I fired up Firefox and used the web to send myself a few messages. Then clicked on "Get Mail" in Firefox, and the messages came in. But when I clicked on them, I was being shown the the body of some other message - actually, seemed like a combination of other messages, none of which were the new message. However, if I "tail" the Inbox for about 500 lines, I can see the message looks just fine (headers and body). In other words, ntfs-3g did not screw up at all. The message is intact and in the folder. It is indeed Thunderbird that is screwing up!! So, the BUG is still there.
Status: RESOLVED → UNCONFIRMED
Resolution: WORKSFORME → ---
Comment 3•16 years ago
|
||
(In reply to comment #2) > Then I took another small folder containing about 100 messages, > and catenated it repeatedly to a new file, until the new > file size was just a hair under 2.6GB. Catenated on what OS? With line mode? Or with binary mode? On what OS "Rebuild Index"(.msf recreation) was done? One of next can occur, depends on last data in file, when Tb's mail box file(==Unix Mbox file) is catenate by utility or command. (a) non_CRLF/non_LF + "From - ..." (b) any + [CRLF] + "From - ..." (c) non_CRLF + [LF] + "From - ..." (d) [CRLF] + [LF] + "From - ..." When (a), on both OS, "From -" can not be unix mbox separator line. When (b), on both OS, "From -" is treated as unix mbox separator line by Tb. When (c)/(d), on Win, "From -" is probably not unix mbox separator line for Tb. Sorry but I don't know about (c)/(d) on Linux. Please note that mail folder file of Tb(==Unix Mbox file) is never be simply a text file, even though it can be read/written in text mode. And, Tb accesses the Unix Mbox file in binary mode, so manual breaking of line break character in Unix Mbox file can easily cause problem. If you need to test with big mbox, do next procedure. (1) Copy a unix mbox file as a mail folder file of Tb (say folde-A) (2) Create folder-B for big mbox on Tb (3) Copy all mails in folder-A to folder-B, using Thunderbird (4) Repeat (3) till required size
Hello??? I dont agree at all. Mail headers are STANDARD!! On both windows AND Linux, the message header that start a new message is (for example) From - Thu Jan 26 16:49:09 2008 I see this is the case for both Linux and Windows. Catenation is done via the cat command. The cat command has no knowledge of line mode. It just reads bytes and writes bytes regardless of file contents. The tty driver, which interprets what constitutes a line, has nothing to do with the cat command. You asked on What OS? My bug report stated: "On Linux (I have dual boot)," ...etc Please let the experts of the internals of TB address this issue. Regards, JD
Comment 5•16 years ago
|
||
(In reply to comment #4) > You asked on What OS? My bug report stated: "On Linux (I have dual boot)," ...etc Where is your description about "catenation is executed under what OS and how the catenation is done"? Since you say "catenate", "cat command on Linux" becomes top of guesses. But you say "dual boot", so "copy (/B) file+file+...file filex on MS Win" case is to be ruled out first. This is the reason why I asked "how catenated on what OS". > Mail headers are STANDARD!! I never talked about "Mail header". Where did I refer to "Mail header"? I refered to line-break character(last bytes of of the catenated file) and "From - ..." line(first line in the catenated file) in Unix Mbox file only, didn't I? > Please let the experts of the internals of TB address this issue. I'm never be an expert of the internal of Thunderbird. But it's required to clarify what is going on in what environment in what situation, before asking "experts of the internal of Thunderbird" for analysis of problem on you. At least, (a) evidence that no user error is involved, (b) evidence that culprit other than Tb's flaw in code doesn't exist, and (c) base that the phenomenon was caused by Tb's flaw in code, are required to ask for help to "experts of the internal of Thunderbird". > Catenation is done via the cat command. > The cat command has no knowledge of line mode. "cat command on Linux" seems to execute binary copy. What is last 2-bytes of the catenated file? [CRLF](0x0D0A)? Or other?
Comment 6•16 years ago
|
||
(In reply to comment #2) Other questions. > Then clicked on "Get Mail" in Firefox, and the messages came in. What do you mean by "Get mail in Firefox"? What phenomenon is "the messages came in" on Firefox? What relation to this bug does Firefox have? Confirmation of "new mail for test" via Web Mail Interface by Firefox? > But when I clicked on them, I was being shown the the body of some other message > - actually, seemed like a combination of other messages, none of which were the new message. Phenomenon on Firefox? Or Thunderbird? If Thunderbird, after download of the new mails? Or before download? If thunderbird, was "catenate" executed while Tb was active? Or after shutdown of Tb? > However, if I "tail" the Inbox for about 500 lines, I can see the message looks just fine (headers and body). Was the "tail" is executed while Tb is active? Or after shutdown of Tb?
Comment 7•16 years ago
|
||
Oh sorry, I misunderstood "tail" command on Linux. It was to view last 500 lines in file. (I misunderstood as "give tail to a file", "append data") > the new file size was just a hair under 2.6GB. Wraparound of offset is suspectable. (32bit but signed initially, and changed to unsigned to support up to 4GB file) "Order Received" column of message list pane is the offset value when local mail folder. (UID when IMAP, article number when NNTP) What is displayed in "Order Received" column of newest mail? What happens when sorted by "Order Received" column? Does problem occur even after explicit "Rebuild Index" on Linux? ("Rebuild Index" via folder properties) Please note that previous question is to rule out "rebuild index is not done after file content change" case. Another question to clarify your environment. > gmail POP3? Gmail IMAP? (probably POP3, because you tested with local file)
I also suspect the same problem. Like I stated previously, the new messages are either displayed as empty or contain a fragment of a message near the start of the Inbox file. Re: Order of messages: Always displayed by ASCENDING date, so that latest messages are at bottom. Re: rebuild of Inbox.msf : Yes I did it every time - and it takes quiet a long time. Re: account type: It is a pop3 gmail account. Also, to add insult to injury, the Fedora distro build of TB 2.0.0.12 (i386) rpm just exits with status 1. It never starts up; The x86_64 build rpm, starts up, but nothing works. Clicking on GetMail is a no-op. Makes me wonder why the behavior of TB from the distro is so hosed?
Comment 9•16 years ago
|
||
Some other questions to clarify environment and situation. > User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); (snip) > Build Identifier: 1.5.0.14, 2.0.0.6, 2.0.0.12 64-bit version Tb? Or 32-bit version Tb on 64-bit Linux? (Probably 64-bit Tb, because you say "The x86_64 build rpm") When problem started to occur? After upgrade of OS or Tb or something else? Or since initial use of 64-bit Tb on 64-bit Fedora? > Booting back into windows, the msf gets re-populated, What phenomenon do you mean by "re-populated"? It sounds to imply "not get re-populated when Linux". Does it mean next? "Automatic/internal "Rebuild Index" during restart of Tb is ; - invoked when Tb on MS Win uses ".msf" which was updated by Tb on Linux - not invoked when Tb on Linux uses ".msf" which was updated by Tb on MS Win > Re: Order of messages: Always displayed by ASCENDING date, so that latest messages are at bottom. Oh, not "Order of messages". "Order Received" column. Click right most button at column header row of message list pane, and display "Order Received" column (==offset value when local mail folder). What value is displayed in "Order Received" column for the mail at bottom? Positive/greater than 2GB? Or positive/small value? Or negative value? Because you don't say "mail list pane display(Subject: etc.) for the mail at bottom is corrupted), "Rebuild Index" looks to handle file greater than 2GB properly. So, I think suspect is logic to obtain mail data based on offset value.
Comment 10•16 years ago
|
||
FYI.
Following is list of bugs of : open, "2GB" in summary, OS=Linux.
Possibly file handler's bug when Linux(Tb is a victim).
> Bug 278738 file:// directory not shown containing a file larger than 2GB
> Bug 384592 Files over 2GB are shown as zero-size in file open dialog
> Bug 383446 upload of huge (~2GB) files through file input fails silently
Reporter | ||
Comment 11•16 years ago
|
||
I dont think TB is a victim. I think some TB modules which is responsible for openning the mail folders is not compiled with full LARGEFILE support. Now, getting back to you question about the Received Order, All messages after the message with the Received order of 2143617487 have negative numbers. Now for the wraparound problem, all new messages have a Received Order of Zero (0) So, what is the msf file used for? Is it used for lseek'ing to the start of the message? If so, the program that is building the msf file at the start of TB, must be using a signed integer, causing the mail client to lseek to 0 (Zero) for new messages, thus displaying the wrong message body. Are any seasoned TB developers reading this? Please PLEASE respond !
Comment 12•16 years ago
|
||
(In reply to comment #11) > Are any seasoned TB developers reading this? Please PLEASE respond ! To JD(bug opener): Be quiet, please. Please don't shout. Before asking "seasoned TB developers" for help, get back up of ".msf" file please, in order that developers can see ".msf" content. (1) Back up of ".msf" after "Rebuild Index" on MS Win (2) Back up of ".msf" after "Rebuild Index" on Linux Changing to Product=Core. I don't know which issue, mail db or file handler, so set Component=MailNews:Database initially. Please change to appropriate one.
Assignee: nobody → bienvenu
Component: General → MailNews: Database
Product: Thunderbird → Core
QA Contact: general → database
Reporter | ||
Comment 13•16 years ago
|
||
I would appreciate it if you would just shut up your comments and leave this bug I have submitted to those who are knowledgeable and capable. So, go and do something else, OK? This is about as polite as I can respond to your dumb comment.
Comment 14•16 years ago
|
||
(In reply to comment #13) > I would appreciate it if you would just shut > up your comments and leave this bug I have submitted > to those who are knowledgeable and capable. I, and I think the majority of seasoned bugzilla experts would agree, would appreciate it if you would read the Bugzilla Etiquette page before making further comments: https://bugzilla.mozilla.org/page.cgi?id=etiquette.html. Several questions: 1. Are you using the /same/ versions on both Windows and Linux partitions? 2. Do you still see this on Trunk TB? 3. Does this problem exist only for files >=2GiB in size? If so, then this is merely a (known) problem with large file sizes. The workaround is simple: don't use 2GiB+ mboxes. (In reply to comment #11) > So, what is the msf file used for? Is it used for lseek'ing to > the start of the message? Finally, the .msf is the message summary file, which caches simple information about messages (file offsets, subject, author, date, threading information, etc.) to avoid opening and reading an mbox file. > Are any seasoned TB developers reading this? Please PLEASE respond ! I am not exactly a "seasoned" TB developer, but that doesn't disqualify me from helping.
Reporter | ||
Comment 15•16 years ago
|
||
Answers to your questions: 1. Yes I am using TB 2.0.0.12 2. I am sorry, I am not familiar with "Trunk" TB. Do you mean building building it myself from CVS? If so, no. I used the Link http://www.mozilla.com/en-US/thunderbird/ which automatically selects the latest TB for my platform. 3. Yes. Thank you for the tip re; TGB Inbox'es. I find it strange that numerous other programs on 32 bit Linux have no problem handling 8GB and larger files without problems. Presently, I have started using Evolution, which avoids the whole size issue and creates a seperate file for each message. I am sure I am going to run into other issues with evolution, such as security...etc. Guess I will have to wait and see. Best regards, JD
Updated•16 years ago
|
Product: Core → MailNews Core
Comment 16•15 years ago
|
||
JD are you willing to test beta of version 3? http://en-us.www.mozillamessaging.com/en-US/thunderbird/early_releases/downloads/
Whiteboard: closeme 2009-06-01
Reporter | ||
Comment 17•15 years ago
|
||
I tried the URL http://en-us.www.mozillamessaging.com/en-US/thunderbird/early_releases/downloads/ it leads to: http://gateway.2wire.net/xslt?PAGE=HURL07 Server not found Firefox can't find the server at gateway.2wire.net. * Check the address for typing errors such as ww.example.com instead of www.example.com * If you are unable to load any pages, check your computer's network connection. * If your computer or network is protected by a firewall or proxy, make sure that Firefox is permitted to access the Web. ----------------------------------------------------------------------------- Anyway, I have completely gotten around the problem by splitting my Inbox into 2 files: Inbox and Inbox.0 (making sure I broke it at message boundary). I think that the inability of tbird to handle files bigger than 2 or 4 GB is extremely lame!!! Linux has been addressing gigantic files for a long time! tbird is still living in the past! I would not blame it on some external factor like some libs of some third party package. I think it is a flaw and it has to be fixed.
Comment 18•15 years ago
|
||
(In reply to comment #17) > I tried the URL > http://en-us.www.mozillamessaging.com/en-US/thunderbird/early_releases/downloads/ > > it leads to: > http://gateway.2wire.net/xslt?PAGE=HURL07 > > Server not found gozer, might this be another bad mirror? http://en-us.www.mozillamessaging.com/en-US/thunderbird/early_releases/downloads/ works (not surprising), but http://gateway.2wire.net/ fails
Comment 19•15 years ago
|
||
(In reply to comment #18) > (In reply to comment #17) > > I tried the URL > > http://en-us.www.mozillamessaging.com/en-US/thunderbird/early_releases/downloads/ > > > > it leads to: > > http://gateway.2wire.net/xslt?PAGE=HURL07 > > > > Server not found > > gozer, might this be another bad mirror? Yes, but sounds like bouncer is not noticing these are not there. Might want to poke MoCo IT about this.
Reporter | ||
Comment 20•15 years ago
|
||
I finally got my network back and I downloaded the t bird3.0 beta 2 I crashed and popped up a death banner saying: We're Sorry Thunderbird had a problem and crashed. To help us diagnose and fix the problem, you can send us a crash report.
Comment 21•15 years ago
|
||
(In reply to comment #19) > (In reply to comment #18) > > (In reply to comment #17) > > > I tried the URL > > > http://en-us.www.mozillamessaging.com/en-US/thunderbird/early_releases/downloads/ > > > > > > it leads to: > > > http://gateway.2wire.net/xslt?PAGE=HURL07 > > > > > > Server not found > > > > gozer, might this be another bad mirror? > > Yes, but sounds like bouncer is not noticing these are not there. Might want to > poke MoCo IT about this. Or rather, your DSL modem is being silly. From IRC: justdave: gateway.2wire.net is the default internal hostname on 2Wire DSL modems. So the original mozillamessaging.com page was probably cached, but the download links were not, and the DSL modem tried to redirect you somewhere.
Reporter | ||
Comment 22•15 years ago
|
||
The DSL modem (at&t uverse) had decided to disable all http outgoing requests because of a bug in the firmware of the modem, which is made by 2wire.com. So, let's forget about this glitch. Neither the URL nor the cached page had anything to do with it. The damned thing (modem) gave me the same error page after I went back and tried other URL's (which I should have done the first time, and not had filed the error about the URL. Sorry, all!
Updated•15 years ago
|
Status: UNCONFIRMED → RESOLVED
Closed: 16 years ago → 15 years ago
Resolution: --- → INVALID
Whiteboard: closeme 2009-06-01
Reporter | ||
Comment 23•15 years ago
|
||
I insist that bug is valid as far as inbox file size is concerned. This bug has to be addressed. It is a lame thing that TB cannot handle inbox folder, or any folder, larger than 2GB. TB must be rebuilt so that any variables dealing with message index numbers or file offsets or sizes use 64 bit data types FOR 64 bit platforms. The 64bit Linux distros which create the TB rpm for x86_64 platforms, do not bother to fix the internal design of TB and make it a truly 64 bit app (again, as far as folder file size, message index numbers...etc are concerned). This has to be fixed by the TB internals team. 32 bit limitations on any app is not only lame, it is also inexcusable with todays cpu's and compilers.
Status: RESOLVED → VERIFIED
Resolution: INVALID → WONTFIX
Comment 24•15 years ago
|
||
my bad. I closed this based on the most recent comments, not the original issue. Which, I will assert, if you want to see addressed you will want show some respect to m-wada. m-wada, there is a reasonable dupe here, no?
Status: VERIFIED → UNCONFIRMED
Resolution: WONTFIX → ---
Whiteboard: dupme
Comment 25•15 years ago
|
||
still needs to be duped iirc
Comment 26•14 years ago
|
||
If I understand this issue correctly, the root problem is that the file limit for Thunderbird on Linux is 2GiB. This is what bug 462665 will fix.
Status: UNCONFIRMED → RESOLVED
Closed: 15 years ago → 14 years ago
Resolution: --- → DUPLICATE
Assignee | ||
Comment 27•14 years ago
|
||
the 2gb limit on liux/mac has been fixed, so that it's on par with the 4gb limit on windows. See Bug 538512
Reporter | ||
Comment 28•14 years ago
|
||
Folks, even a 4GB limit is lame! It means that TB is using 32 bit unsiged data type for message offsets into the inbox. I am currently using FC12 32 bit (i686), and firefox-3.5.8-1.fc12.i686 I recombined my Inbox's 3 fragments into a single Inbox of 3138023630 bytes, and started firefox. No problems. All messages seem to be there and are perfectly readable. However, I do not know what will happen when I exceed 4GB size. Personally, I do not want go through another fiasco as I did when I first submitted this bug. I will shutdown firefox, delete my big Inbox, and restore my last small Inbox of 1.5GB.
Assignee | ||
Comment 29•14 years ago
|
||
(In reply to comment #28) > Folks, even a 4GB limit is lame! Hence bug 462665
Comment 30•14 years ago
|
||
Cleanup *dupeme* whiteboard flag from bugs that are marked as Resolved Duplicate!
Whiteboard: dupme
You need to log in
before you can comment on or make changes to this bug.
Description
•