Closed Bug 815348 Opened 12 years ago Closed 9 years ago

Tracking: Meet memory-usage acceptance criteria

Categories

(Core :: General, defect)

Product:
Core
Component:
General
Platform:
ARM
Gonk (Firefox OS)
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED WORKSFORME

People

(Reporter: cjones, Assigned: gal)

References

(Depends on 1 open bug,
URL
)

Details

(Keywords: meta) 

See the URL for the acceptance criteria.  The document will evolve.
The model device used for tests is the "otoro", which is
 - HVGA
 - 16-bit display
 - has 256MB physical RAM
 - running the ics_chocolate branch of gonk
These are intermittently not-broken enough for me to complete the test.  Not blocking this work anymore though they're still bad bugs.
No longer depends on: 815368, 815377
Guys, can we run these tests[1] as part of the smoketests?  We went from passing all three on Dec 6 to failing all of them today.

The regressions (bug 820702, bug 820704) are not related to memory usage, they're just extremely basic user scenarios that are regressing.  This is noise that makes it hard to actually test the memory-usage signal.

[1] https://wiki.mozilla.org/B2G/Memory_acceptance_criteria#Workloads
No longer depends on: 820704
We just went from all-pass to all-fail again, all the failures for crashes not related to memory usage.  This is bad and wrong.
Back to no more non-memory-usage-related failures \o/

Can we help keep it that way by adding these tests to our smoketests?
That's tough - the smoketests are already lengthy and time-consuming.

Is it possible to automate this test?
Ok. Will see if QA will run it at least weekly.
Weekly isn't enough to be useful.  Don't bother.
I think these memory tests can be automated with gaia-ui-test framework (probably need a little help with shell script)
Can I work on this?
We don't have a meaningful and reliable way to automate many of the checks in these tests.  I proposed a way over a year ago, but we haven't had bandwidth to implement that yet.

With the technology we currently have, you could automate the tests for a particular gaia SHA1, but then on the next commit the automation would break, with high probability.  And then the tests would fail for reasons not related to the tests themselves until someone had time to fix the tests/automation.  That would probably take around a week, and we'd be back where we started with comment 8.

So unless you're committed to that maintenance burden or can find someone who is, I would also say "don't bother" ;).
Blocks: 827809
Keywords: meta
Aaaaand we're back to failing all the tests.  Sad.
Can you try backing out bug 838615?
(In reply to Chris Jones [:cjones] [:warhammer] from comment #14)
> (Right bug #?)

Yes.  It's possible that we're sending processes into the background and giving them a very low CPU priority, which is causing them to take longer to start up, which is causing them to be killed.
We're back to failing for non-memory-management-related bugs, which is a start I guess.
I'm tired of fighting for blocking on exotic use cases like "can launch the dialer".

Someone else will need to pick this up.
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → WONTFIX
That someone is lucky me I guess.
Status: RESOLVED → REOPENED
Resolution: WONTFIX → ---
Assignee: nobody → gal
In the very last run of the memory tests, we went all pass.  Nice work, jlebar!

From here on out we'll transition away from unrealistic workloads like launching a few apps and onto our extensive automation, dogfooding, and QA testing.
I forgot to add: this was the first all-pass for a long time, so bug 836654 is almost certainly the fix.
Is this bug serving any useful purpose now?  It hasn't been touched in 9 months.  Bug 797189 is still open as a kind of catch-all for memory issues.
Status: REOPENED → RESOLVED
Closed: 11 years ago → 9 years ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.