Closed Bug 815348 Opened 7 years ago Closed 4 years ago
Tracking: Meet memory-usage acceptance criteria
See the URL for the acceptance criteria. The document will evolve.
The model device used for tests is the "otoro", which is - HVGA - 16-bit display - has 256MB physical RAM - running the ics_chocolate branch of gonk
Depends on: 815355
Depends on: 815368
Depends on: 815377
These are intermittently not-broken enough for me to complete the test. Not blocking this work anymore though they're still bad bugs.
Depends on: 820702
Depends on: 820704
Guys, can we run these tests as part of the smoketests? We went from passing all three on Dec 6 to failing all of them today. The regressions (bug 820702, bug 820704) are not related to memory usage, they're just extremely basic user scenarios that are regressing. This is noise that makes it hard to actually test the memory-usage signal.  https://wiki.mozilla.org/B2G/Memory_acceptance_criteria#Workloads
No longer depends on: 820704
Depends on: 822398
Depends on: 823962
Depends on: 824106
We just went from all-pass to all-fail again, all the failures for crashes not related to memory usage. This is bad and wrong.
Depends on: 824526
Depends on: 824819
Depends on: 824869
Back to no more non-memory-usage-related failures \o/ Can we help keep it that way by adding these tests to our smoketests?
That's tough - the smoketests are already lengthy and time-consuming. Is it possible to automate this test?
Ok. Will see if QA will run it at least weekly.
Weekly isn't enough to be useful. Don't bother.
I think these memory tests can be automated with gaia-ui-test framework (probably need a little help with shell script) Can I work on this?
We don't have a meaningful and reliable way to automate many of the checks in these tests. I proposed a way over a year ago, but we haven't had bandwidth to implement that yet. With the technology we currently have, you could automate the tests for a particular gaia SHA1, but then on the next commit the automation would break, with high probability. And then the tests would fail for reasons not related to the tests themselves until someone had time to fix the tests/automation. That would probably take around a week, and we'd be back where we started with comment 8. So unless you're committed to that maintenance burden or can find someone who is, I would also say "don't bother" ;).
Depends on: 827165
Depends on: 834059
Depends on: 834883
Depends on: 836509
Depends on: 836638
Depends on: 837927
Depends on: 840252
Depends on: 840257
Aaaaand we're back to failing all the tests. Sad.
Can you try backing out bug 838615?
(Right bug #?)
(In reply to Chris Jones [:cjones] [:warhammer] from comment #14) > (Right bug #?) Yes. It's possible that we're sending processes into the background and giving them a very low CPU priority, which is causing them to take longer to start up, which is causing them to be killed.
Depends on: 839256
We're back to failing for non-memory-management-related bugs, which is a start I guess.
Depends on: 840698
Depends on: 841105
Depends on: 841600
Depends on: 841604
I'm tired of fighting for blocking on exotic use cases like "can launch the dialer". Someone else will need to pick this up.
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → WONTFIX
That someone is lucky me I guess.
Status: RESOLVED → REOPENED
Resolution: WONTFIX → ---
In the very last run of the memory tests, we went all pass. Nice work, jlebar! From here on out we'll transition away from unrealistic workloads like launching a few apps and onto our extensive automation, dogfooding, and QA testing.
I forgot to add: this was the first all-pass for a long time, so bug 836654 is almost certainly the fix.
Is this bug serving any useful purpose now? It hasn't been touched in 9 months. Bug 797189 is still open as a kind of catch-all for memory issues.
Status: REOPENED → RESOLVED
Closed: 7 years ago → 4 years ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.