484795 - compact folder changes/corrupts order received value of local mail folder (column name of "Order Received" is misleading. Column name "Mail Offset" is better?)

Status: RESOLVED WONTFIX

(Thunderbird :: Folder and Message Lists, defect)

Description

[clive darr]

User-Agent:       Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-GB; rv:1.9.0.7) Gecko/2009021906 Firefox/3.0.7 FirePHP/0.2.4
Build Identifier: version 2.0.0.21 (20090302)

20,000 emails 
order received range from 300000 to 80000000
compact folder and 
order received range changes to 1000 to 80000000


Reproducible: Always

Steps to Reproduce:
1. get 20000 emails
2. delete first 1000 and last 1000 emails
3. make a note of first and last order received
4. compact folder 
Actual Results:  
first and last order received values have changed !

Expected Results:  
first and last order received values should not change

Comment 1

[WADA:World Anti-bad-Duping Agency]

Local mail folder?(POP3 or "Local Folders")? IMAP folder?

Value in "Order Received" column is: 
  Local mail folder : Offset of mail data in mail folder file(Unix Mbox)
                      (UIDL of the mail is saved in X-UIDL: header)
  IMAP older        : UID of mail
  Newsgroup         : ARTICLE number of news article

"Compact of local mail folder" is removal of mail data with MSG_FLAG_EXPUNGED 
bit=ON in X-Mozilla-Status: header(==mail data of deleted mail).
( See http://www.eyrich-net.org/mozilla/X-Mozilla-Status.html?en )
So offset in mail folder file becomes different by "compact folder" operation.
"Order Received" is surely misleading column name when local mail folder.

If IMAP or Newsgroup, it's server error or result of server side configuration change(restructuring of mail/news data base), because it means that UID or ARTICLE number is changed at server.

Comment 2

[clive darr]

its a local folder getting emails from googlemail via POP

maybe just renaming column to "mail offset" would fix it !

Comment 3

[Ulf Zibis]

Please don't change this behavior!
Yes, maybe the column naming is misleading, but the info about offset is valuable when manually inspecting or repairing Mbox files.

Possible solution:
Have 2 columns, "Order Received" + "Mail Offset"

Comment 4

[Wayne Mery (:wsmwk)]

Thoughts on column name of "Order Received" is misleading, column name "Mail Offset" is better?  Might upset those who are accustomed to seeing Order Received.

Comment 5

[Blake Winton (:bwinton) (:☕️)]

"Mail Offset" confuses even me, and seems like an implementation detail that people shouldn't depend on or care about.  (For that matter, what does it even mean for Maildir?)

Comment 6

[Ulf Zibis]

It *is* an implementaton detail, so why not name it as that. From my side you could also name it:
MBOX offset
MBOX-file offset
internal start address
... or any similar combinations
Having 2 columns would help for all needs, also for inspecting IMAP offline-store mbox files.
A numbering like IMAP UID for local folders could be established by calculating the ordinal on the fly while opening the folder.

Comment 7

[:aceman]

But the column also means Order received. That it changes value at compact time is an implementation detail, but even after compact it still shows the relative order of the messages as received. Why do you need the absolute values of the column to be permanent?

Also, the value actually is not Message offset in all circumstances. In some cases it is computed differently (see e.g. bug 793865). So the column renaming would not be correct again.

I think Order received is a universal name that holds true after all changes of the implementation we currently have in queue.

Comment 8

[Kent James (:rkent)]

(In reply to :aceman from comment #7)
> But the column also means Order received. That it changes value at compact
> time is an implementation detail, but even after compact it still shows the
> relative order of the messages as received. Why do you need the absolute
> values of the column to be permanent?
>
 
I agree. When you say "Order received" you also need to add "compared to what?" We're just comparing here to a moving target, but the net result is still "order received" compared to something.

I would WONTFIX this.

Comment 9

[Wayne Mery (:wsmwk)]

per comment 8

Comment 10

[B.J. Herbison]

(In reply to :aceman from comment #7)
> But the column also means Order received. That it changes value at compact
> time is an implementation detail, but even after compact it still shows the
> relative order of the messages as received. Why do you need the absolute
> values of the column to be permanent?

The problem is the column is permanent. After a compact new messages are added with "order re3ceived" values which interleave with older messages.

But WONTFIX is appropriate as Order Received is fixed in bug 1183490.

Comment 11

[:aceman]

(In reply to B.J. Herbison from comment #10)
> The problem is the column is permanent. After a compact new messages are
> added with "order re3ceived" values which interleave with older messages.
> 
> But WONTFIX is appropriate as Order Received is fixed in bug 1183490.

Yes, that was incorrect behaviour and should be fixed now.