Closed Bug 918140 Opened 11 years ago Closed 9 years ago

mis-interprets disk "hard quota" (blocks vs bytes confusion)

Categories

(Core Graveyard :: File Handling, defect)

x86
Linux
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED
mozilla36

People

(Reporter: psz, Assigned: psz)

References

Details

Attachments

(1 file, 1 obsolete file)

User Agent: Mozilla/5.0 (X11; Linux i686 on x86_64; rv:24.0) Gecko/20100101 Firefox/24.0 (Beta/Release)
Build ID: 20130910160258

Steps to reproduce:

Having upgraded from Thunderbird 17.0.8 to 24.0 on Debian Linux, at
startup all users are getting the alert message:

  The folder Inbox is full, and can't hold any more messages. To make room for more
  messages, delete any old or unwanted mail and compact the folder.

This happens regardless of mailbox sizes, even for new users with empty
inboxes.

For now I have reverted to Thunderbird 17.0.8 .

Please let me know if I should test something.

Cheers, Paul

Paul Szabo   psz@maths.usyd.edu.au   http://www.maths.usyd.edu.au/u/psz/
School of Mathematics and Statistics   University of Sydney    Australia
Summary: Thunderbird 24 on Linux: The folder Inbox is full ... → Thunderbird 24 on Linux: mis-understands quota and says "The folder Inbox is full ..."
Seems that the problem is that Thunderbird24 mis-interprets disk quota.
The disk quota on my home directory was set so:
$ quota
Disk quotas for user psz (uid 1001): 
     Filesystem  blocks   quota   limit   grace   files   quota   limit   grace
      ...
      /dev/sda4  676984  6000000 12000000            5443  500000 1000000        
and that caused Thunderbird to say "The folder Inbox is full ...".
To avoid that message and allow Thunderbird to work, one could set
quotas to much larger e.g.:
      /dev/sda4  676812  600000000 1200000000            5443  500000 1000000        
or to "infinite":
      /dev/sda4  660056       0       0            5376  500000 1000000        

Cheers, Paul
Summary: Thunderbird 24 on Linux: mis-understands quota and says "The folder Inbox is full ..." → Thunderbird 24 on Linux: mis-interprets disk quota and says "The folder Inbox is full ..."
Attached patch quota.patch (obsolete) — Splinter Review
The calculation of space left until "hard quota" is wrong. The formula
  fs_buf.F_BSIZE * (dq.dqb_bhardlimit - dq.dqb_curspace)
does not work because:
 - quota blocks (e.g. dqb_bhardlimit) are in 1kB units, regardless of
   filesystem block size
 - current usage dqb_curspace is in bytes, not blocks. (Despite man page
   and despite comments in /usr/include/sys/quota.h. In the "old days"
   that element of struct dqblk was named dqb_curblocks, and it may have
   contained blocks.)
I propose the attached patch to resolve this issue.

Cheers, Paul

Paul Szabo   psz@maths.usyd.edu.au   http://www.maths.usyd.edu.au/u/psz/
School of Mathematics and Statistics   University of Sydney    Australia
Linux i686 on x86_64:
same behaviour on Thunderbird 24.5.0. & Thunderbird 30.0b1.
I had to downgrade from Thunderbird 31.2. to TB 10.0. to use diskquotas with Debian Wheezy. 

Like OP described, with diskquotas activated, it is not possible to download emails. 
TB stopps with a message: "mailbox full" or something.

I need quotas for the homes of 40 users. Disabling diskquotas is very annoying!
Status: UNCONFIRMED → NEW
Ever confirmed: true
Summary: Thunderbird 24 on Linux: mis-interprets disk quota and says "The folder Inbox is full ..." → mis-interprets disk quota and says "The folder Inbox is full ..."
Refreshing the patch that nobody noticed...
Attachment #808121 - Attachment is obsolete: true
Attachment #8522932 - Flags: review?(stransky)
Attachment #8522932 - Flags: feedback?(mjt+mozilla)
Hm. Now this is interesting.  I don't really know, I just tried to read the manpage and following it... ;)

I just looked at the linux quota toolset sources -- indeed, they treat dqb_curspace in bytes and dqb_*limit in 1024-byte (1<<10) blocks, despite what the manpage and the header file says.

Well, what can I tell?  The patch looks correct.  Thank you for finding this.

/mjt
Attachment #8522932 - Flags: feedback?(mjt+mozilla) → feedback+
Comment on attachment 8522932 [details] [diff] [review]
bug918140_quotas.patch

See http://stackoverflow.com/questions/2506288/detect-block-size-for-quota-in-linux

Actually the block size comes from <sys/mount.h> and its defined as:

#define BLOCK_SIZE  1024

there's also a macro fs_to_dq_blocks() in <sys/quota.h> which converts FS block size to quota block size. But IMHO the BLOCK_SIZE from <sys/mount.h> would be enough here. 

So r+ with that change, but I'm not a maintainer of this module. Maybe benjamin?
Attachment #8522932 - Flags: review?(stransky) → review?(benjamin)
Comment on attachment 8522932 [details] [diff] [review]
bug918140_quotas.patch

I'll rubberstamp this, based on Martin's feedback.
Attachment #8522932 - Flags: review?(benjamin) → review+
This does need a real commit message.
https://hg.mozilla.org/integration/mozilla-inbound/rev/4d55c412cacd
Assignee: nobody → psz
Status: NEW → ASSIGNED
Summary: mis-interprets disk quota and says "The folder Inbox is full ..." → mis-interprets disk "hard quota" (blocks vs bytes confusion)
Dear Magnus,

Why did you change the title? As is now:
 - suggests that thunderbird uses "soft" quotas correctly, that it has
   an issue with the "hard" one specifically: but that is probably not
   so, as I do not think it uses soft quotas at all.
 - the confusion is not just about blocks vs bytes, but also about block
   sizes.

Why did you assign this bug to me? Do you expect me to do something
about it? - As I see things, I did my duty, provided a patch; you
re-jiggled my patch, pointing out your own failure at noticing it
(and deflecting blame, saying "nobody noticed": certainly untrue!).

---

Your patch in the "commit"(?)
https://hg.mozilla.org/integration/mozilla-inbound/rev/4d55c412cacd
seems wrong: calculates
  BLOCK_SIZE * dq.dqb_bhardlimit
(twice, slightly wasteful, but that is not the issue): both BLOCK_SIZE
and dqb_bhardlimit may be 32-bit integers, their product may produce
overflow in 32-bit arithmetic, and then be "wrong" in the 64-bit
comparison or difference (with the 64-bit dq.dqb_curspace).
Please arrange for the (BLOCK_SIZE * dq.dqb_bhardlimit) calculation to
be done in 64-bit arithmetic.

Cheers, Paul

Paul Szabo   psz@maths.usyd.edu.au   http://www.maths.usyd.edu.au/u/psz/
School of Mathematics and Statistics   University of Sydney    Australia
Somewhat refuting my previous comment about the need to do the
  BLOCK_SIZE * dq.dqb_bhardlimit
calculation in 64-bit.
On my Debian wheezy machine, both the (identical) files
  /usr/include/sys/quota.h
  /usr/include/i386-linux-gnu/sys/quota.h
set dqb_bhardlimit as 64-bit when _LINUX_QUOTA_VERSION is 2 or higher,
and have dqb_curspace then; otherwise, for _LINUX_QUOTA_VERSION < 2,
have 32-bit quantities and dqb_curblocks instead.
Since Thunderbird uses dqb_curspace always, maybe there are no Linux
distros left with older quota versions (or anyway Thunderbird may not
work well there).
But then, other distros may use different quota systems.
(At least this "bug" will not affect me... hooray!)
(In reply to Paul Szabo from comment #12)
> Why did you change the title? As is now:

I changed it to be more generic, but tried not to go into the exact details. Feel free to adjust it if you want.

> Why did you assign this bug to me? Do you expect me to do something
> about it? - As I see things, I did my duty, provided a patch; you

I assigned it to you because you provided the patch. As you can see, I landed it on the integration branch. Someone will merge it to mozilla-central in a while, and they will mark the bug as fixed then. Nothing more you have to do.

> re-jiggled my patch, pointing out your own failure at noticing it
> (and deflecting blame, saying "nobody noticed": certainly untrue!).

"nobody noticed" referred to that the patch was left sitting in the bug without action. To avoid that patches need to be flagged for review...
Component: Untriaged → File Handling
Product: Thunderbird → Core
Version: 24 → Trunk
https://hg.mozilla.org/mozilla-central/rev/4d55c412cacd
Status: ASSIGNED → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla36
Depends on: 1106551
Product: Core → Core Graveyard
You need to log in before you can comment on or make changes to this bug.