Closed Bug 826071 Opened 8 years ago Closed 7 years ago

[Email] New messages appear above top of message list; AKA message list is always scrolled past new messages


(Firefox OS Graveyard :: Gaia::E-Mail, defect, P1)

Gonk (Firefox OS)


(blocking-b2g:leo+, b2g18 verified)

1.1 QE2 (6jun)
blocking-b2g leo+
Tracking Status
b2g18 --- verified


(Reporter: ckreinbring, Assigned: gaye)



(Keywords: late-l10n, Whiteboard: [dupetome], c= , [TD-24266] , MiniWW)


(2 files)

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20100101 Firefox/17.0
Build ID: 20121128204232

Steps to reproduce:

Unagi version 20130102070202

Original defect:

Prerequisites: Time zone on device set for earlier than the current location's time zone (eg Continent = America, City = Nome).
Email account set up on the device.

1. From another email address, send mail to the account set up on the device.
2. On the device, launch Email and observe the email shown at the top of the screen.
3. Attempt to scroll up to see more emails.
4. From the email address in step 1, send another email to the account on the device.
5. Tap the Refresh softkey (circular arrow in the lower left corner) and watch for the email sent in the previous step.
6. Attempt to scroll up to see more emails.

Actual results:

When the email app loads or is refreshed, the messages that are marked as from a future time are already scrolled past.

Expected results:

When the email app loads or is refreshed, the last message received by the account is shown at the top of the screen, regardless of when it was marked.
Thanks for filing this as a separate bug!

(In reply to ckreinbring from comment #0)
> Prerequisites: Time zone on device set for earlier than the current
> location's time zone (eg Continent = America, City = Nome).
> Email account set up on the device.

Based on your steps to reproduce, I believe the time zone is not relevant to what you are witnessing.  Messages are stored in the database with UTC timestamps.  We did have a bug where we filtered our database results for future messages, but that was not a timezone issue but a clock skew issue.

> Expected results:
> When the email app loads or is refreshed, the last message received by the
> account is shown at the top of the screen, regardless of when it was marked.

So, in terms of loading, this expectation is something that we currently try to meet some of the time, but is something that we are going to be entirely unable to meet once we land the fixes for bug 822882.

Specifically, we do one of three things when opening a folder.  If the last sync was more than an hour ago (or four hours for non-inboxes) we try and synchronize before showing the user any messages.  If the last sync was recent enough, we show what the database knows immediately and then try and start the sync (as a 'refresh').  Since the sync is an asynchronous process and we don't want to drive the user crazy by having the message list jump around, this basically means that the user may not be seeing the most recent messages.

When I finish implementing bug 822882 and we land it, we will never make the user wait for synchronization to happen, and everything will be a 'refresh'.  This means that the scrolling situation will get worse.

I had filed bug 800402 on potentially providing better visual feedback about such things.

In terms of refreshing, we could make the 'refresh' button skip to the newest known messages before triggering the refresh, but that would be a different set of semantics than I implemented based on the 1 sentence definition in the interaction diagrams.  Also, the whole async operation/scrolling issue would still stand.  I think a different button metaphor like a big up arrow to convey jumping to the new messages would be more appropriate for that.
(This is going to be a popular bug; dupe to it.)
Summary: [B2G][Email] Future messages don't always show up, are usually scrolled past when app is loaded or refreshed → [Email] New messages appear above top of message list; AKA message list is always scrolled past new messages
Whiteboard: [dupetome]
OS: All → Gonk (Firefox OS)
Hardware: All → ARM
Still repros on Unagi version 20130208070301
Gaia: cff23a80c41b7de223d27c6a6f1f82f95c9c5f6b
Kernel: Dec 5
Casey, as alluded to in comment 1, this is going to start being a bigger problem now that bug 822882 is up for review and likely to land soon.  We may need a bigger UX fix than the unread count discussed on bug 800402.

To be very clear about the situation, the following is how e-mail sync works under bug 822882.  This is also how things work right now if you open the Inbox within an hour of when it was last synchronized:
1) The UI displays the most recent messages from the database.
2) We try and talk to the server to refresh the state of those messages (detect flag changes, deletions) and hear about new messages.
3) As we hear about messages we didn't know about, we will insert them into the list.  This will change the scroll height of the message list and will result in the scroll bar indicator on the right-side becoming visible for a second or two.  "New" messages will be inserted above the top-most message in the list; the user will have to scroll up to see them.

I think we will very likely need the unread indicator since the scroll-bar can be subtle, but is there anything else you would like me to do at the same time?
Flags: needinfo?(kyee)
Repros on Unagi version 20130308070202
Gaia: 87801b12cd3aac153fae84844da8f1e6455ea679
Kernel Date: Dec 5
Could we scroll to the top anytime we receive new messages?   I feel that loosing your place in the message list is probably less important than seeing that you have received new email?
Flags: needinfo?(kyee)
We can do anything.  There are just a lot of parameters to decide on:

** When do we scroll?
W1) After the first newer message is received.  This implies that messages are fetched newest-to-oldest by a refresh rather than oldest-to-newest.  We currently fetch oldest-to-newest because it avoids the potential for gaps and makes a little more sense with the current UI where the user needs to scroll up manually.

Specifically, if we have the set of known messages [known A, known B, known C], and the refresh is going to eventually get us 3 new messages where A is the newest and C the oldest so that the resulting set is [new A, new B, new C, known A, known B, known C], our IMAP sync algorithm can decide to fetch things A-C or C-A.  We could display our little 'synchronizing messages' notice with spinner where we believe the current insertion points are, or do nothing to help make things more clear.

The bigger issue is that if we lose network access during synchronization and we only have synchronized 'A', it may not be clear to the user that there simply are some unsynchronized messages.  We could address this with some type of UX affordance that indicates there is known to be a gap in synchronized messages.

W2) After all new messages have been received.  Note that for ActiveSync accounts, this is basically what we have because messages come in large batches.

** How do we scroll?
(Keep in mind that we currently can't truly smooth animate scrolling so we'll need to do some trickery with transform.)

S1) Instantaneously scroll us to the top without animation.  This would not be too disurptive with W1, but for ActiveSync which is W2-ish and there might be 20 new messages, this could be more disruptive.

S2) Unrealistic animated scroll to convey that we are scrolling the user to the top, but not accurately reflecting the number of messages.  Specifically, If we have new messages A-Z, and we are displaying known A-F (because of display limitations), we would only animate past new A-new F rather than past new A-new Z.  Effectively, a card transition rather than scrolling.  Makes more sense with W2 than W1.

S3) Realistic scrolling.  Actually scroll past all of new A-new Z or whatever we have.

S4) Some weird other effect like zooming out and then zooming back in, flipping the screen, cross-fading, etc.

** What do we do about recent touches / user input / mode switches:
Things that would be weird for the user and potential solutions:

- Going to touch a message and having the scrolling make it so they touch the wrong message.  This could be addressed by suppressing user input for some small fraction of a second during/after the scroll so we can be sure that any taps the user makes are based on what they are seeing now.

- The user having started a message compose or search mode, and then returning.  Do we make the change then?

- The user having entered edit mode.  Do we make the change when they return?

** Other possible affordances:

Q1) We could use an internal notification to say "there are N" new messages.  We could update this as the messages come in.  If the user clicks on it, we do our scroll deal then.  Places for the affordance would be in the header, in a little indicator that slides out just under the header, or our banner/toaster mechanism that lives at the bottom of the page.  There could be an 'x' to get rid of the message about new messages.

Q2) We still haven't actually implemented the unread count.  So this would be Q1-ish, but we make clicking on the unread count take you to the top and maybe have some visual flourish when there are new unread mesages.  Clicking on the unread count might not be discoverable, and there could be issues from mis-touches from the compose button.  We'd probably want to sensitize most of the header with the folder name.
Flags: needinfo?(kyee)
Repros on Unagi version 20130404070202
Gaia: a845be046c5d3cb077e3c78f963ca5c079e7ab3d
Kernel Date: Dec 5
UX reviewed this and is going to work on some proposals, based on Andrew's suggestions and thoughts, for how this might work this week. We'll keep you posted.
Whiteboard: [dupetome] → [dupetome], u=user c=System s=tribe
Assignee: nobody → shane
Flags: needinfo?(kyee)
Still repros on Inari 1.0.1 commercial RIL and Leo 1.1 mozilla RIL.

Inari Build ID: 20130503070205
Kernel Date: Feb 21
Gaia: 3e232bce289c9e156d92553e752616cba284bc8f
Version #: 18.0

Leo Build ID: 20130503070204
Kernel Date: Mar 15
Gaia: b0aca0dd1e2955e11190ede725e1fb9ee596438b
Version #: 18.0
This document was created by Modern Tribe and subsequently edited to display only options that were reviewed and accepted by UX. If anyone needs the original please contact myself or Shane. Thanks!
Duplicate of this bug: 869824
Whiteboard: [dupetome], u=user c=System s=tribe → [dupetome], u=user c=System s=tribe, [TD-24266]
Target Milestone: --- → 1.1 QE2
blocking-b2g: --- → leo?
Priority: -- → P1
Assignee: shane → nobody
blocking-b2g: leo? → leo+
Assignee: nobody → gaye
Attached file Pull request
Attachment #752975 - Flags: feedback?(bugmail)
Attachment #752975 - Attachment mime type: text/plain → text/html
Comment on attachment 752975 [details]
Pull request

I've provided feedback on the pull request.  A lot of feedback because this is a very complicated feature!  You're getting thrown into the deep-end on this one! :)

In retrospect, I perhaps should have left the feedback here.  If you have any questions about my responses, please put the feedback? flag back on here or set needinfo or something so that I can know to check the pull request again.  Although I don't hate github like Fabrice, I don't keep up with my github spam like I do with my bugmail.
Attachment #752975 - Flags: feedback?(bugmail)
I realized my proposal for the new message count was wrong; new mechanism is on the pull request now in the same grouping of line comments.
Whiteboard: [dupetome], u=user c=System s=tribe, [TD-24266] → [dupetome], c=, [TD-24266]
Whiteboard: [dupetome], c=, [TD-24266] → [dupetome], c= , [TD-24266]
Whiteboard: [dupetome], c= , [TD-24266] → [dupetome], c= , [TD-24266] , MiniWW
Comment on attachment 752975 [details]
Pull request

I've completed my review pass on the gaia and gelam ( pull requests.  This is looking good, and major props for digging into this very complicated system which also has a smidge of technical debt here and there ;)

Please request review again via the flag when you want me to do another review pass.  I check bugmail much much more frequently than github mail.
Attachment #752975 - Flags: feedback+
gaye - are you following this up per asuth's feedback+?

we hope to have this fixed and landed by the end of the week.
Flags: needinfo?(gaye)
(In reply to Wayne Chang [:wchang] from comment #17)
> gaye - are you following this up per asuth's feedback+?

Yep, he is.  I just finished another mini-review pass, and things are making good progress!
Flags: needinfo?(gaye)
Yep things are chugging along. The feature works and the frontend is tested, but the backend needs to be tested and Andrew has a few small points that I still need to address.
Attachment #752975 - Flags: review?(bugmail)
Merged de6fa6572ccf27a3ddf7f78b3f9b28453021bf0e :)
Comment on attachment 752975 [details]
Pull request

r=asuth and congrats and great work on your first major e-mail feature patch!
Attachment #752975 - Flags: review?(bugmail)
Attachment #752975 - Flags: review+
Attachment #752975 - Flags: feedback+
Flags: in-moztrap?
Flags: in-moztrap? → in-moztrap-
Please see STR in comment 0. This is a valid bug worthy of adding a test case.
Flags: in-moztrap- → in-moztrap?
Just attempted to repro, and while the email list does not scroll to the first new email, there is now a banner at the top notifying the user of new emails.  Is this the intended fix?
(In reply to ckreinbring from comment #25)
>   Is this the intended fix?

Yes, see the attachment in comment #11 for design/requirements.
In that case, verifying as fixed since it matches the requirements in comment 11.

Leo 1.1 mozilla RIL
Build ID: 20130729070226
Gaia: 7aaffc8ccb6cf7ddd1e97943c108f1cb9eae5de0
Platform Version: 18.1
Just had to change the tracking flag.
Closed: 7 years ago7 years ago
New Test Case created to cover bug
Flags: in-moztrap? → in-moztrap+
You need to log in before you can comment on or make changes to this bug.