Closed
Bug 1015427
Opened 10 years ago
Closed 10 years ago
make test-perf not running in same order on datazilla
Categories
(Firefox OS Graveyard :: Gaia::PerformanceTest, defect, P1)
Tracking
(Not tracked)
RESOLVED
FIXED
2.0 S3 (6june)
People
(Reporter: mchang, Assigned: hub)
References
Details
(Keywords: perf, Whiteboard: [c=automation p=1 s= u=])
Attachments
(1 file)
+++ This bug was initially created as a clone of Bug #1015426 +++ From bug 1013387, bug 1013381, we had false alarm memory regressions because make test-perf increases the size of system.* values after every run. Because the music and email app were run in different order between the two revisions, the system.* values went up when in reality it was because make test-perf didn't execute the tests in the same order, causing a false alarm. Either reset the system between tests or run make test-perf in the same order every time.
|
Assignee | |
Comment 2•10 years ago
|
||
Sadly the current design of the make test-perf doesn't make for a deterministic order in which app tests are run as it scan through the filesystem. Maybe this is where we should implement a definite order and use that as a baseline.
Flags: needinfo?(hub)
|
|
Assignee | |
Updated•10 years ago
|
Component: Performance → Gaia::PerformanceTest
|
|
Assignee | |
Updated•10 years ago
|
Blocks: gaia-perf-measure
|
|
Assignee | |
Updated•10 years ago
|
Assignee: nobody → hub
Whiteboard: [c=automation p= s= u=] → [c=automation p=2 s= u=]
|
Reporter | |
Updated•10 years ago
|
|
|
Assignee | |
Updated•10 years ago
|
Status: NEW → ASSIGNED
|
|
Assignee | |
Comment 3•10 years ago
|
||
Yuren, while this fix is for make test-perf, it will impact in some way the integration test. We just sort the app list to keep the order deterministic on the testing slaves.
Attachment #8430950 -
Flags: review?(yurenju.mozilla)
|
|
Assignee | |
Updated•10 years ago
|
Attachment #8430950 -
Flags: review?(eperelman)
|
||
Updated•10 years ago
|
Attachment #8430950 -
Flags: review?(eperelman) → review+
|
||
Updated•10 years ago
|
Attachment #8430950 -
Flags: review?(yurenju.mozilla) → review+
|
|
Assignee | |
Comment 4•10 years ago
|
||
Merged https://github.com/mozilla-b2g/gaia/commit/d9874b2e74065c43f421bed0c97a02bfd662eb32
Status: ASSIGNED → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
|
|
Assignee | |
Updated•10 years ago
|
Whiteboard: [c=automation p=2 s= u=] → [c=automation p=1 s= u=]
|
|
Assignee | |
Updated•10 years ago
|
Target Milestone: --- → 2.0 S3 (6june)
You need to log in
before you can comment on or make changes to this bug.
Description
•