Closed Bug 817185 Opened 12 years ago Closed 11 years ago

[email] Future messages don't always show up

Categories

(Firefox OS Graveyard :: Gaia::E-Mail, defect, P2)

defect

Tracking

(blocking-basecamp:+)

VERIFIED FIXED
B2G C3 (12dec-1jan)
blocking-basecamp +

People

(Reporter: squib, Assigned: squib)

Details

If you have a message with a date in the future, we end up starting out scrolled past it. If a folder only has messages from the future, we don't show *anything*. We should fix this, since time zone issues and other clock issues can result in us having messages with future dates.

The culprit is endTS in getMessagesInImapDateRange. We need to give it a huge value instead of Date.now()
What is the standard behaviour. Do we have to use the receiving date and time ?
Flags: needinfo?(bugmail)
There are two dates normally used for e-mail:

- Composed: The date the message indicates it was composed, provided by the "Date" header in the message.  IMAP servers expose this as the DATE.

- Received: The date the mail server received the message, as indicated by the Received: header provided by the receiving SMTP server (or when imported into the server.)  IMAP servers expose this as the INTERNALDATE.  ActiveSync specs email:DateReceived to be this.

Thunderbird uses the "Composed" date for IMAP because it does not know how to get the INTERNALDATE from the server; this is a known/lived-with but.


Since both IMAP and ActiveSync both allegedly used received date time, the only source of 'future' *should be* from clock skew between the device and the server.  It would not be surprising for there to be a few seconds of such clock skew, but it should exist.  The current implementation makes a bad call by using Date.now(), as that clock skew will cause us to do the wrong thing.

The bigger question is how many servers don't obey these specs and might be putting messages further into the future because they use the 'composed' date.  Jim, are you seeing small time-skew messages, or big ones like when spammers send messages frmo a few days in the future according to the 'Composed' date?
Flags: needinfo?(bugmail)
Uh, but either way I recommend we take this fix which should be a 1-line code fix, albeit with a unit test that will probably be a bit longer.
(In reply to Andrew Sutherland (:asuth) from comment #2)
> Thunderbird uses the "Composed" date for IMAP because it does not know how
> to get the INTERNALDATE from the server; this is a known/lived-with but.

s/but/bug/
 
> It would not be surprising for there to be a few seconds of
> such clock skew, but it should exist.

Strike "but it should exist"; my main point is that it is highly unlikely that both the device and the server clocks will be precisely synchronized.
Waiting for Jim feedback
Flags: needinfo?(squibblyflabbetydoo)
(In reply to Andrew Sutherland (:asuth) from comment #2)
> The bigger question is how many servers don't obey these specs and might be
> putting messages further into the future because they use the 'composed'
> date.  Jim, are you seeing small time-skew messages, or big ones like when
> spammers send messages frmo a few days in the future according to the
> 'Composed' date?

I mostly see small amounts of skew (Hotmail's server clock seems to run a minute or so fast relative to mine). However, I did at one point enter my time zone incorrectly, and it of course caused several hours of skew. Obviously similar time zone issues could happen on the sender's end as well.
Flags: needinfo?(squibblyflabbetydoo)
(In reply to Jim Porter (:squib) from comment #6)
> However, I did at one point enter my
> time zone incorrectly, and it of course caused several hours of skew.

I think this should just be apparent skew because of the time formatting; all values in the database should be in UTC.
blocking-basecamp: ? → +
Priority: -- → P2
(In reply to Andrew Sutherland (:asuth) from comment #7)
> (In reply to Jim Porter (:squib) from comment #6)
> > However, I did at one point enter my
> > time zone incorrectly, and it of course caused several hours of skew.
> 
> I think this should just be apparent skew because of the time formatting;
> all values in the database should be in UTC.

Sorry. To elaborate, I meant that I entered the right time, but put it in the wrong timezone, resulting in the wrong UTC time.
Assignee: nobody → squibblyflabbetydoo
Status: NEW → ASSIGNED
Mass Modify: All un-milestoned, unresolved blocking-basecamp+ bugs are being moved into the C3 milestone. Note that the target milestone does not mean that these bugs can't be resolved prior to 12/10, rather C2 bugs should be prioritized ahead of C3 bugs.
Target Milestone: --- → B2G C3 (12dec-1jan)
PR up here: https://github.com/mozilla-b2g/gaia-email-libs-and-more/pull/95

I still need to improve the tests a bit more, but everything is pretty close to done.
Merged: https://github.com/mozilla-b2g/gaia-email-libs-and-more/pull/95
        https://github.com/mozilla-b2g/gaia/pull/6928
Status: ASSIGNED → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
In Unagi version 20130102070202, emails with future dates are still being scrolled past when loading or refreshing the inbox.  However, otherwise empty email accounts will shot the emails normally.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
This bug ended up getting fixed as a message filtering bug, scrolling was not addressed; please file a new bug on the scrolling issue so it can be appropriately triaged, etc.
Status: REOPENED → RESOLVED
Closed: 12 years ago11 years ago
Resolution: --- → FIXED
Verified as fixed in build 20130114073222 on Unagi.
Status: RESOLVED → VERIFIED
You need to log in before you can comment on or make changes to this bug.