971603 - [tarako]Email App Loads Messages Indefinitely

Status: RESOLVED WORKSFORME

(Firefox OS Graveyard :: Gaia::E-Mail, defect, P3)

Description

[Mason Chang [Inactive] [:mchang]]

While loading the test account in the email app, The Email app seems to load new messages forever, causing a memory leak.

Comment 1

[Andrew Sutherland [:asuth] (he/him)]

The sync process is deterministic; the only thing that should really be impacted by low-memory situations is if our garbage production is causing GC's at an abnormally high rate and that is slowing us down.  Mason, if you could provide the following info, that would be handy.  See https://wiki.mozilla.org/Gaia/Email/RequiredBugInfo for details in some caes on how to get the info, etc.:
- Account domain
- Account type
- Number of messages/message density
- Network connection speed
- Did the screen turn off during sync
- Logcat

Comment 2

[Mason Chang [Inactive] [:mchang]]

Attachment: logcat w/ letting the screen shut off

Comment 3

[Mason Chang [Inactive] [:mchang]]

Attachment: logcat w/ APZ enabled

Comment 4

[Mason Chang [Inactive] [:mchang]]

Attachment: logcat w/ APZ disabled

To answer your other questions, I used the partner provided credential account [e-mail].

This was over a 2g network. 

I have attached a logcat w/ and without the screen shutting off. If i just keep the screen alive by scrolling a little bit, we keep downloading messages.

Comment 5

[Mason Chang [Inactive] [:mchang]]

Attachment: Video of Scrollbar Stuck At bottom

This occurs over wifi also. Just noticed something weird, the scrollbar jumps down to the bottom, then starts downloading messages forever, almost like it thinks we're constantly scrolling down. Please see the attached video ~the 21s mark.

Comment 6

[Andrew Sutherland [:asuth] (he/him)]

It looks like there are a few things going on:

1) We are trying to synchronize ~1286 messages from a single day because the test account is subscribed to a lot of bugmail because a day is our minimum sync granularity.  This places a fundamental floor on our memory usage, etc.  Bug 823384 covers improving this.

2) We are not holding any wake-locks while synchronizing so if the screen goes off, stuff is likely to stop happening as some combination of connection loss, process suspension, and loss of high priority status due to not being in the foreground could get us memkilled.  This is bug 971183.  Because of the large number of messages involved, quick fix timeout-based wake-locks are unlikely to actually resolve this problem, but could work in general.  (A simple wake-lock tied to the sync request could work, but has bad battery life connotations if our sync breaks.)

3) As you note in the video, all of the very large number of messages we are synchronizing are showing up in the DOM.  I've filed bug 972328 on that (known but previously not filed) issue.  My suggested fix there involves some refactoring, but it's an easy stem-the-bleeding fix (on a different bug than that one which involved a lot of typing!) to just let the front-end issue shrink requests all the time.


While these are all real bugs, I think using a test account with message densities in excess of what Andreas gets is not a realistic testing scenario for a limited capacity device.

Comment 7

[Nicholas Nethercote [inactive]]

MemShrink:P3 until we have data on a more normal workload.

Comment 8

[bhavana bajaj [:bajaj]]

Leaving a NI on :mason/Brian to help retest this as discussed in triage today. Keeping comment #6 in mind was curious if there is a recovery path here ? Go back to homescreen and open email app again  ? Are we stuck on downloading new messages forever? What is the threshold number of messages that we are good with ?

Could this be a major issue for FTE when new users are trying to sync-up their email ?

Comment 9

[Andrew Sutherland [:asuth] (he/him)]

(In reply to bhavana bajaj [:bajaj] from comment #8)
> Could this be a major issue for FTE when new users are trying to sync-up
> their email ?

I don't have recent specific memory numbers, but if anyone wants a better understanding of the e-mail app's approach here and how it impacts memory, check out https://wiki.mozilla.org/Gaia/Email/Implementation/MailSynchronization.

The executive summary is that the primary things that matters to us is the number of messages per day.  We can handle a folder with 100,000 messages in it for IMAP, but those messages need to be spread out over 2 years, not 2 months.  (And there are of course things we can do to optimize this, but we do start running up against limitations of the IMAP protocol and start needing to increase our network usage.)

Comment 10

[Brian Huang [:brianhuang][:brhuang]]

William, please help to check this.

Comment 11

[Edward Chen[:edchen]]

This case seems like #979164 (#839273), 

Hi Andrew, could you check both of tickets are the same issues.

Comment 12

[Andrew Sutherland [:asuth] (he/him)]

(In reply to Edward Chen[:edchen] from comment #11)
> This case seems like #979164 (#839273), 
> 
> Hi Andrew, could you check both of tickets are the same issues.

They are different; in bug 839273 (note: to get bugzilla to autolink, you need to do "bug NNNN" not "#NNNN", see http://globau.wordpress.com/2014/02/14/markup-within-bugzilla-comments/) sync breaks completely and fails to return the mutex, also breaking message display/all future operations on the folder.  As described on this bug in the logcat and video, sync works and messages are being emitted into the DOM, but eventually we sync too much/put too much in the DOM and crash.

Comment 13

[Mason Chang [Inactive] [:mchang]]

Just flashed the newest tarako build today and wifi doesn't work again. Will get a sim card this weekend and try again on cell data.

Comment 14

[William Hsu [:whsu]]

Interesting problem~
Thanks for Andrew's sharing (Comment 6).
I think this bug was a legacy problem.
Andrew and Bhavana both are right (Comment 6 and Comment 7).
One is from developer's opinion, the other is from user's opinion.
Yes, it might be a system limitation of our implementation.
But, If I am a general user, I don't want to understand the technologic problem and I just care about if I can use my mobile device to send/receive emails.

It is unreasonable to tell Firefox OS user that you are using a cheaper device so you cannot send/receive emails as normal as other mobile devices.

So, looking into this problem. What did we have?
We had excellent developer here and powerful OS, but we had a limited capacity device with poor UI presentation.
So, we can change the rule (email quantity first, not date first) and turn the UI to fulfill user's requirement.
For example, we can temporarily retrieve the 100 mails.(It will relate to comment 9)
If it causes heavy loading for system, we can decrease the quantity of synced emails.
Just like the sliding window. Firefox OS can dynamically adjust the number of retrieved emails.
(It is a ideal but we may have hardcode first)
We can append a "Read more" button on the bottom of inbox.
If user want to read more emails, user can tap it to get more mails.

My two cents. Any thought?

I will also try to reproduce it on the latest Tarako build to check if we have workaround solution.
Thanks. Have a nice weekend! :)

Comment 15

[William Hsu [:whsu]]

Keep needinfo "whsu" to remind myself to check this problem.

Comment 16

[William Hsu [:whsu]]

> Andrew and Bhavana both are right (Comment 6 and Comment 7).
Sorry for the typo. It was Comment 6 and Comment 8.

Comment 17

[Mason Chang [Inactive] [:mchang]]

I just tried on my Tarako with two different accounts and multiple different folders. My own personal mozilla account and the jn.rlly test account. All accounts seemed to sync only the past 5 days.

Inbox in the past 5 days: 24 messages
My bugmail folder: 22 messages
fxos-perf-bugmail folder: 25 messages
jn.rlly test account: 28 messages

All of them worked fine. Maybe someone who has an email account with 100 messages in a single day can try?

Comment 18

[[:fabrice] Fabrice Desré]

I tested using an edge data connection with my bugmail account (it gets more than 100 mails per day) and everything worked fine.

Reopen if you can reproduce.

Comment 19

[William Hsu [:whsu]]

(In reply to Mason Chang [:mchang] from comment #17)
> I just tried on my Tarako with two different accounts and multiple different
> folders. My own personal mozilla account and the jn.rlly test account. All
> accounts seemed to sync only the past 5 days.
> 
> Inbox in the past 5 days: 24 messages
> My bugmail folder: 22 messages
> fxos-perf-bugmail folder: 25 messages
> jn.rlly test account: 28 messages
> 
> All of them worked fine. Maybe someone who has an email account with 100
> messages in a single day can try?

Hi, Mason,

Could I have your help?
If you still can reproduce this bug, please disable "APZ" and reproduce this bug again to see if you still can reproduce this bug.
If you cannot reproduce it, it might relates to APZ function.

By the way, after I disabled APZ, email UI worked as usual while I reproduced this bug today.
But if I enabled APZ, I can reproduce similar symptoms.

* Method to disable APZ
Step 1. Long press the power on button until the shoutdown menu displayed.
Step 2. Tap "Disable APZ for Apps"
Step 3. Please restart the device.


* Build Information:
 - Gaia      7f434a8799b8c7b2b419ea118b2de7c2682c170d
 - Gecko     8da7b6293d63b140932893fed6a45f91bfa93631
 - BuildID   20140310075556
 - Version   28.0

Comment 20

[Andrew Sutherland [:asuth] (he/him)]

I think Mason and Fabrice are saying they see no bug.  The number of message synchronized in the initial sync is the expected outcome as per https://wiki.mozilla.org/Gaia/Email/Implementation/MailSynchronization.  More messages can always be retrieved by pressing the "get more messages" button.

Comment 21

[Mason Chang [Inactive] [:mchang]]

I just tried without APZ, and cannot reproduce regardless if APZ is enabled or disabled on the two test accounts.