Closed
Bug 980519
Opened 12 years ago
Closed 12 years ago
Experiment with other instance types for Android 2.3 jobs
Categories
(Infrastructure & Operations Graveyard :: CIDuty, task)
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: armenzg, Assigned: gbrown)
References
Details
The jobs run rather slow. We want to see what would happen if we run them on faster instances or multi-cores ones.
Let's get you a loaner.
https://tbpl.mozilla.org/?tree=Ash&jobname=Android%202.3
Once we have a better candidate we should write a patch to do the required buildbot changes.
<rail> armenzg: ... we may need to introduce another platform name for this because buildbot is dumb
| Reporter | ||
Comment 1•12 years ago
|
||
As per email, I've upgraded this instance from m1.medium to c3.2xlarge.
Adding rail for him to be aware with this experiment.
| Assignee | ||
Comment 2•12 years ago
|
||
c3.2xlarge, R1 running via mozharness:
top - 10:27:52 up 1 day, 4:10, 2 users, load average: 6.10, 4.92, 2.60
Tasks: 171 total, 2 running, 169 sleeping, 0 stopped, 0 zombie
Cpu(s): 44.2%us, 1.2%sy, 0.0%ni, 54.5%id, 0.0%wa, 0.0%hi, 0.0%si, 0.1%st
Mem: 15339148k total, 5815076k used, 9524072k free, 77884k buffers
Swap: 0k total, 0k used, 0k free, 4089484k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2575 cltbld 20 0 3484m 896m 22m R 360 6.0 24:42.95 emulator64-arm
746 cltbld 20 0 291m 129m 26m S 12 0.9 0:49.26 Xvfb
1186 cltbld 20 0 346m 12m 9868 S 1 0.1 0:06.54 metacity
2723 cltbld 20 0 17340 1344 960 R 0 0.0 0:00.15 top
----------
top - 11:13:07 up 1 day, 4:55, 2 users, load average: 4.32, 4.72, 4.86
Tasks: 171 total, 2 running, 169 sleeping, 0 stopped, 0 zombie
Cpu(s): 56.4%us, 1.8%sy, 0.0%ni, 41.6%id, 0.0%wa, 0.0%hi, 0.0%si, 0.2%st
Mem: 15339148k total, 5872012k used, 9467136k free, 81316k buffers
Swap: 0k total, 0k used, 0k free, 4089844k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2575 cltbld 20 0 3556m 932m 22m R 454 6.2 199:06.16 emulator64-arm
746 cltbld 20 0 291m 129m 26m S 16 0.9 6:37.48 Xvfb
1186 cltbld 20 0 346m 12m 9868 S 2 0.1 0:51.47 metacity
26 root 20 0 0 0 0 S 0 0.0 0:00.19 ksoftirqd/5
--------------
11:22:24 INFO - REFTEST TEST-LOAD | http://10.0.2.2:8854/tests/image/test/reftest/apng/bug411852-1-ref.png | 507 / 3554 (14%)
11:22:24 INFO - REFTEST TEST-KNOWN-FAIL(EXPECTED RANDOM) | http://10.0.2.2:8854/tests/image/test/reftest/apng/delaytest.html?bug411852-1.png | image comparison (==)
11:22:24 INFO - REFTEST INFO | Loading a blank page
11:22:24 INFO - REFTEST TEST-END | http://10.0.2.2:8854/tests/image/test/reftest/apng/delaytest.html?bug411852-1.png
11:22:24 INFO - REFTEST TEST-START | http://10.0.2.2:8854/tests/image/test/reftest/apng/delaytest.html?bug546272.png
11:22:24 INFO - REFTEST TEST-LOAD | http://10.0.2.2:8854/tests/image/test/reftest/apng/delaytest.html?bug546272.png | 508 / 3554 (14%)
11:22:24 WARNING - TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/image/test/reftest/apng/delaytest.html?bug546272.png | application ran for longer than allowed maximum time
11:22:24 INFO - INFO | automation.py | Application ran for: 1:00:51.307762
11:22:25 INFO - Total resource usage - Wall time: 3715s; CPU: 48.0%; Read bytes: 1245184; Write bytes: 294600704; Read time: 572; Write time: 1276600
11:22:25 INFO - install - Wall time: 22s; CPU: 13.0%; Read bytes: 0; Write bytes: 7294976; Read time: 0; Write time: 17276
11:22:25 INFO - run-tests - Wall time: 3693s; CPU: 48.0%; Read bytes: 1040384; Write bytes: 286666752; Read time: 496; Write time: 1259012
11:22:25 INFO - stop-emulators - Wall time: 0s; CPU: Can't collect data; Read bytes: 0; Write bytes: 0; Read time: 0; Write time: 0
| Assignee | ||
Comment 3•12 years ago
|
||
top - 11:36:42 up 1 day, 5:19, 2 users, load average: 3.56, 3.34, 3.45
Tasks: 172 total, 2 running, 170 sleeping, 0 stopped, 0 zombie
Cpu0 : 43.3%us, 1.4%sy, 0.0%ni, 55.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu1 : 41.6%us, 0.7%sy, 0.0%ni, 57.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.3%st
Cpu2 : 43.1%us, 0.0%sy, 0.0%ni, 56.6%id, 0.0%wa, 0.0%hi, 0.0%si, 0.3%st
Cpu3 : 52.7%us, 1.4%sy, 0.0%ni, 45.5%id, 0.0%wa, 0.0%hi, 0.0%si, 0.3%st
Cpu4 : 58.2%us, 1.4%sy, 0.0%ni, 40.1%id, 0.0%wa, 0.0%hi, 0.0%si, 0.3%st
Cpu5 : 42.5%us, 1.7%sy, 0.0%ni, 55.4%id, 0.0%wa, 0.0%hi, 0.0%si, 0.3%st
Cpu6 : 39.9%us, 0.7%sy, 0.0%ni, 59.4%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu7 : 41.6%us, 1.4%sy, 0.0%ni, 56.7%id, 0.0%wa, 0.0%hi, 0.0%si, 0.3%st
Mem: 15339148k total, 5874052k used, 9465096k free, 83104k buffers
Swap: 0k total, 0k used, 0k free, 4207028k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2784 cltbld 20 0 3483m 833m 22m R 370 5.6 9:07.16 emulator64-arm
746 cltbld 20 0 291m 130m 27m S 12 0.9 8:08.47 Xvfb
1186 cltbld 20 0 346m 12m 9868 S 2 0.1 1:03.25 metacity
1259 cltbld 9 -11 271m 4280 3048 S 0 0.0 0:05.91 pulseaudio
2822 cltbld 20 0 17340 1352 960 R 0 0.0 0:00.68 top
| Assignee | ||
Comment 4•12 years ago
|
||
For comparison, an R3 run on c3.2xlarge:
12:35:23 INFO - REFTEST TEST-PASS | http://10.0.2.2:8854/tests/layout/reftests/position-dynamic-changes/vertical/toauto-topA-heightN-bottomN.html?padding_parent | image comparison (==)
12:35:23 INFO - REFTEST INFO | Loading a blank page
12:35:23 INFO - REFTEST TEST-END | http://10.0.2.2:8854/tests/layout/reftests/position-dynamic-changes/vertical/toauto-topA-heightN-bottomN.html?padding_parent
12:35:23 INFO - REFTEST TEST-START | http://10.0.2.2:8854/tests/layout/reftests/position-dynamic-changes/vertical/toauto-topA-heightA-bottomN.html?padding_parent
12:35:23 INFO - REFTEST TEST-LOAD | http://10.0.2.2:8854/tests/layout/reftests/position-dynamic-changes/vertical/toauto-topA-heightA-bottomN.html?padding_parent | 81 / 3189 (2%)
12:35:23 WARNING - TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/position-dynamic-changes/vertical/toauto-topA-heightA-bottomN.html?padding_parent | application ran for longer than allowed maximum time
12:35:23 INFO - INFO | automation.py | Application ran for: 1:00:46.737086
| Assignee | ||
Comment 5•12 years ago
|
||
jsreftests on c3.2xlarge:
13:51:18 INFO - REFTEST TEST-LOAD | http://10.0.2.2:8854/jsreftest/tests/jsreftest.html?test=js1_5/Regress/regress-360969-05.js | 1932 / 6734 (28%)
13:51:18 WARNING - TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/jsreftest/tests/jsreftest.html?test=js1_5/Regress/regress-360969-05.js | application ran for longer than allowed maximum time
13:51:18 INFO - INFO | automation.py | Application ran for: 1:01:20.542796
13:51:19 INFO - Total resource usage - Wall time: 3760s; CPU: 51.0%; Read bytes: 0; Write bytes: 406016000; Read time: 0; Write time: 1557784
13:51:19 INFO - install - Wall time: 21s; CPU: 13.0%; Read bytes: 0; Write bytes: 61440; Read time: 0; Write time: 8
13:51:19 INFO - run-tests - Wall time: 3738s; CPU: 52.0%; Read bytes: 0; Write bytes: 398024704; Read time: 0; Write time: 1534840
13:51:19 INFO - stop-emulators - Wall time: 0s; CPU: Can't collect data; Read bytes: 0; Write bytes: 0; Read time: 0; Write time: 0
top - 14:25:44 up 1 day, 8:08, 2 users, load average: 7.73, 4.58, 2.82
Tasks: 172 total, 2 running, 170 sleeping, 0 stopped, 0 zombie
Cpu0 : 47.6%us, 0.0%sy, 0.0%ni, 52.0%id, 0.3%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu1 : 47.6%us, 1.4%sy, 0.0%ni, 51.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu2 : 47.6%us, 1.4%sy, 0.0%ni, 50.7%id, 0.0%wa, 0.0%hi, 0.0%si, 0.3%st
Cpu3 : 98.7%us, 1.3%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu4 : 46.3%us, 1.0%sy, 0.0%ni, 52.4%id, 0.0%wa, 0.3%hi, 0.0%si, 0.0%st
Cpu5 : 48.0%us, 0.7%sy, 0.0%ni, 51.4%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu6 : 47.4%us, 0.7%sy, 0.0%ni, 51.9%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu7 : 47.1%us, 1.0%sy, 0.0%ni, 51.9%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 15339148k total, 6141012k used, 9198136k free, 93296k buffers
Swap: 0k total, 0k used, 0k free, 4443352k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
3156 cltbld 20 0 3484m 841m 22m R 426 5.6 15:59.61 emulator64-arm
746 cltbld 20 0 291m 130m 27m S 14 0.9 25:22.69 Xvfb
3270 cltbld 20 0 357m 33m 18m S 3 0.2 0:05.75 xpcshell
1186 cltbld 20 0 346m 12m 9868 S 2 0.1 3:16.35 metacity
1259 cltbld 9 -11 271m 4280 3048 S 0 0.0 0:17.19 pulseaudio
3011 cltbld 20 0 17340 1352 960 R 0 0.0 0:12.33 top
3254 cltbld 20 0 61896 11m 1096 S 0 0.1 0:00.23 python
| Assignee | ||
Comment 6•12 years ago
|
||
M1 on c3.2xlarge:
15:48:02 INFO - 1274 INFO Passed: 31197
15:48:02 INFO - 1275 INFO Failed: 0
15:48:02 INFO - 1276 INFO Todo: 63
15:48:02 INFO - 1277 INFO Slowest: 35209ms - /tests/content/base/test/test_bug682592.html
15:48:02 INFO - 1278 INFO SimpleTest FINISHED
15:48:02 INFO -
15:48:02 INFO - INFO | automation.py | Application ran for: 0:10:28.016660
vs. normal m3.medium (ash):
09:23:09 INFO - 1274 INFO Passed: 31203
09:23:09 INFO - 1275 INFO Failed: 0
09:23:09 INFO - 1276 INFO Todo: 63
09:23:09 INFO - 1277 INFO Slowest: 209635ms - /tests/content/base/test/test_bug682592.html
09:23:09 INFO - 1278 INFO SimpleTest FINISHED
09:23:09 INFO - INFO | automation.py | Application ran for: 0:35:41.721845
| Reporter | ||
Comment 7•12 years ago
|
||
Changed now to be a m3.large.
| Assignee | ||
Comment 8•12 years ago
|
||
m3.large, R1 running via mozharness:
top - 09:35:31 up 2:46, 1 user, load average: 2.36, 1.36, 0.67
Tasks: 139 total, 2 running, 137 sleeping, 0 stopped, 0 zombie
Cpu0 : 92.0%us, 4.3%sy, 0.0%ni, 3.7%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu1 : 91.7%us, 5.3%sy, 0.0%ni, 3.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 7629428k total, 5576772k used, 2052656k free, 147680k buffers
Swap: 0k total, 0k used, 0k free, 4079108k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1629 cltbld 20 0 3139m 710m 22m R 178 9.5 2:40.44 emulator64-arm
659 cltbld 20 0 242m 122m 27m S 8 1.6 0:06.45 Xvfb
1741 cltbld 20 0 357m 33m 18m S 6 0.4 0:00.97 xpcshell
1105 cltbld 20 0 346m 12m 9868 S 1 0.2 0:00.97 metacity
1627 cltbld 20 0 17340 1312 960 R 0 0.0 0:00.81 top
-----------------
top - 09:44:31 up 2:55, 1 user, load average: 2.80, 2.50, 1.57
Tasks: 139 total, 2 running, 137 sleeping, 0 stopped, 0 zombie
Cpu0 : 86.3%us, 6.5%sy, 0.0%ni, 7.2%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu1 : 86.3%us, 5.5%sy, 0.0%ni, 8.2%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 7629428k total, 5755440k used, 1873988k free, 147684k buffers
Swap: 0k total, 0k used, 0k free, 4079344k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1629 cltbld 20 0 3139m 883m 22m R 174 11.9 18:04.15 emulator64-arm
659 cltbld 20 0 242m 122m 27m S 8 1.6 0:48.20 Xvfb
1105 cltbld 20 0 346m 12m 9868 S 1 0.2 0:07.57 metacity
1725 cltbld 20 0 61780 11m 1092 S 0 0.2 0:00.58 python
1 root 20 0 24332 2316 1364 S 0 0.0 0:00.39 init
-----------------
10:36:08 INFO - REFTEST TEST-LOAD | http://10.0.2.2:8854/tests/image/test/reftest/pngsuite-oddsizes/s39i3p04.png | 241 / 3554 (6%)
10:36:08 INFO - REFTEST TEST-LOAD | http://10.0.2.2:8854/tests/image/test/reftest/pngsuite-oddsizes/s39_3p04.html | 241 / 3554 (6%)
10:36:08 INFO - REFTEST TEST-KNOWN-FAIL | http://10.0.2.2:8854/tests/image/test/reftest/pngsuite-oddsizes/s39i3p04.png | image comparison (==)
10:36:08 INFO - REFTEST INFO | Loading a blank page
10:36:08 WARNING - TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/image/test/reftest/pngsuite-oddsizes/s39i3p04.png | application ran for longer than allowed maximum time
10:36:08 INFO - INFO | automation.py | Application ran for: 1:01:07.094051
10:36:10 INFO - Total resource usage - Wall time: 3717s; CPU: 88.0%; Read bytes: 1302528; Write bytes: 294555648; Read time: 936; Write time: 1137124
10:36:10 INFO - install - Wall time: 24s; CPU: 58.0%; Read bytes: 16384; Write bytes: 79720448; Read time: 20; Write time: 395928
10:36:10 INFO - run-tests - Wall time: 3693s; CPU: 89.0%; Read bytes: 1097728; Write bytes: 198160384; Read time: 504; Write time: 636604
10:36:10 INFO - stop-emulators - Wall time: 0s; CPU: Can't collect data; Read bytes: 0; Write bytes: 0; Read time: 0; Write time: 0
| Assignee | ||
Comment 9•12 years ago
|
||
Now m3.medium.
| Reporter | ||
Comment 10•12 years ago
|
||
From https://bugzilla.mozilla.org/show_bug.cgi?id=980520#c3 it seems that you don't need the machine again.
Did we significantly better results on other instances?
| Assignee | ||
Comment 11•12 years ago
|
||
m3.medium, R1 running via mozharness:
top - 15:12:35 up 1 day, 9:50, 1 user, load average: 1.38, 1.17, 0.60
Tasks: 135 total, 2 running, 133 sleeping, 0 stopped, 0 zombie
Cpu0 : 36.6%us, 6.0%sy, 0.0%ni, 24.4%id, 4.0%wa, 0.0%hi, 0.1%si, 28.9%st
Mem: 3840420k total, 3821444k used, 18976k free, 91756k buffers
Swap: 0k total, 0k used, 0k free, 2678332k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
8041 cltbld 20 0 2652m 474m 22m R 91.3 12.7 2:30.51 emulator64-arm
664 cltbld 20 0 226m 122m 26m S 1.0 3.3 45:54.20 Xvfb
1014 cltbld 20 0 95880 10m 1880 S 1.0 0.3 10:33.58 x11vnc
1169 cltbld 9 -11 271m 3984 2696 S 0.7 0.1 11:59.44 pulseaudio
8141 cltbld 20 0 66468 16m 4444 S 0.7 0.4 0:00.48 python
----------------
08:37:55 INFO - REFTEST TEST-LOAD | http://10.0.2.2:8854/tests/image/test/reftest/pngsuite-ancillary/cdsn2c08.png | 146 / 3554 (4%)
08:37:55 INFO -
08:37:55 WARNING - TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/image/test/reftest/pngsuite-ancillary/cdsn2c08.png | application ran for longer than allowed maximum time
08:37:55 INFO - INFO | automation.py | Application ran for: 1:02:14.502592
08:37:57 INFO - Total resource usage - Wall time: 3843s; CPU: 99.0%; Read bytes: 17743872; Write bytes: 307716096; Read time: 6700; Write time: 1104932
08:37:57 INFO - install - Wall time: 59s; CPU: 100.0%; Read bytes: 0; Write bytes: 197644288; Read time: 0; Write time: 880212
08:37:57 INFO - run-tests - Wall time: 3784s; CPU: 99.0%; Read bytes: 17526784; Write bytes: 109764608; Read time: 6584; Write time: 224648
| Assignee | ||
Comment 12•12 years ago
|
||
R1 via mozharness on my ix loaner:
14:13:03 INFO - REFTEST TEST-LOAD | http://10.0.2.2:8854/tests/layout/reftests/w3c-css/submitted/variables/variable-supports-62.html | 1058 / 3554 (29%)
14:13:03 INFO - REFTEST TEST-LOAD | http://10.0.2.2:8854/tests/layout/reftests/w3c-css/submitted/variables/support/color-green-ref.html | 1058 / 3554 (29%)
14:13:03 WARNING - TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/w3c-css/submitted/variables/variable-supports-62.html | application ran for longer than allowed maximum time
14:13:03 INFO - INFO | automation.py | Application ran for: 1:00:55.757441
14:13:04 INFO - Total resource usage - Wall time: 3715s; CPU: 13.0%; Read bytes: 8192; Write bytes: 522919936; Read time: 464; Write time: 716564
14:13:04 INFO - install - Wall time: 22s; CPU: 13.0%; Read bytes: 0; Write bytes: 78622720; Read time: 0; Write time: 46476
14:13:04 INFO - run-tests - Wall time: 3694s; CPU: 13.0%; Read bytes: 8192; Write bytes: 441323520; Read time: 464; Write time: 669928
| Assignee | ||
Comment 13•12 years ago
|
||
R1 via remotereftest.py on my w540 laptop:
REFTEST TEST-LOAD | http://10.0.2.2:8854/tests/layout/reftests/bidi/unicode-bidi-plaintext-textarea-2.html | 1626 / 3554 (45%)
TEST-UNEXPECTED-FAIL | http://10.0.2.2:8854/tests/layout/reftests/bidi/unicode-bidi-plaintext-textarea-2.html | application ran for longer than allowed maximum time
INFO | automation.py | Application ran for: 1:00:47.541895
| Assignee | ||
Comment 14•12 years ago
|
||
Let's summarize these results in terms of "how long would it take to run all plain-reftests on each platform?" This will be a gross approximation: If R1 was 10% complete after 1 hour, I'll assume it would take 10 hours to complete R1, and an additional 10 hours to complete each of R2 and R3, for a total of 30 hours.
m3.medium 75 hours
m3.large 50 hours
c3.2xlarge 22 hours
ix 11 hours
w540 7 hours
We cannot use my laptop -- that's just an interesting result.
We could use ix slaves. Allowing for some variation in per-chunk run times, I imagine we could run reftests in about 16 chunks.
We probably cannot reasonably use aws for reftests (unless profiling/optimization improves these results dramatically).
| Assignee | ||
Comment 15•12 years ago
|
||
Let's also look at mochitests, comparing a single run of each chunk on m3.medium to one on m3.large:
m3.medium m3.large
M1 0:37:00 0:12:50
M2 0:32:39 0:10:29
M3 0:35:32 0:11:11
M4 1:02:09 0:22:44
M5 1:01:20 0:20:41
M6 0:44:50 0:15:52
M7 0:42:23 0:13:53
M8 0:31:01 0:10:11
The m3.large run times are about 1/3 of the m3.medium times and I understand that m3.large costs about 2x m3.medium -- it seems to me that running these on m3.large would reduce our overall costs.
Notice the top results (comments 8 and 11 especially): the emulator fully utilizes the cpus on m3.large, but does not on m3.medium (less memory?).
Also, most of the tests (mochitests, crashtests, robocop) disabled for 2.3 are caused by timeouts, or show evidence of Android services not keeping up with requests. I wonder if these problems would be lessened on m3.large.
So although a "bigger" aws instance type will not help us run reftests, I think it would be better to run the remaining tests on m3.large. Armen (or appropriate rel-eng proxy) -- can we run Android 2.3 tests on m3.large?
Flags: needinfo?(armenzg)
| Reporter | ||
Comment 16•12 years ago
|
||
(adding kmoir so she can be in the loop once she's back from EclispeCon)
Running on a different instance type than our regular test EC2 might not be an easy task.
rail: what would it be involved?
gbrown:
* I'm happy to run the reftests on the Linux ix machines since it is an under-utilized pool of machines.
* m3.large seems a better choice than m3.medium
What are the m1.medium wall time numbers? (Our current setup)
Do we know what is the reason for the slow jobs?
Have any of the numbers improved after we deployed a newer version of the Mesa libs? (2 weeks ago)
Flags: needinfo?(armenzg) → needinfo?(rail)
| Assignee | ||
Comment 17•12 years ago
|
||
(In reply to Armen Zambrano [:armenzg] (Release Engineering) (EDT/UTC-4) from comment #16)
> (adding kmoir so she can be in the loop once she's back from EclispeCon)
> What are the m1.medium wall time numbers? (Our current setup)
Oh, are we running m1.medium currently? I thought they were m3.medium.
Anyway, the wall time numbers on ash seem very close to my m3.medium observations.
> Do we know what is the reason for the slow jobs?
No. I am hoping to find time to profile reftests again (dminor did earlier, but that did not point to a solution and we never followed up).
> Have any of the numbers improved after we deployed a newer version of the
> Mesa libs? (2 weeks ago)
No. All the current times seem about the same as they were at the end of February.
Comment 18•12 years ago
|
||
(In reply to Armen Zambrano [:armenzg] (Release Engineering) (EDT/UTC-4) from comment #16)
> (adding kmoir so she can be in the loop once she's back from EclispeCon)
>
> Running on a different instance type than our regular test EC2 might not be
> an easy task.
> rail: what would it be involved?
* Buildbot has no idea about the instance types we use. We'll need different naming for these instances.
* Changing the instance type for all slaves may be problematic, see bug 969590.
Flags: needinfo?(rail)
| Assignee | ||
Comment 19•12 years ago
|
||
(In reply to Geoff Brown [:gbrown] from comment #14)
> Let's summarize these results in terms of "how long would it take to run all
> plain-reftests on each platform?"
>
> m3.medium 75 hours
> m3.large 50 hours
> c3.2xlarge 22 hours
> ix 11 hours
> w540 7 hours
Between the March 13 and March 18 merges to Ash, there was a dramatic improvement in reftest performance. Ash run-times which were equivalent to m3.medium now run 3 to 4 times faster and ix runs about twice as fast:
Ash (like m3.medium?) 24 hours
ix 6 hours
| Assignee | ||
Updated•12 years ago
|
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
Updated•8 years ago
|
Component: Platform Support → Buildduty
Product: Release Engineering → Infrastructure & Operations
Updated•6 years ago
|
Product: Infrastructure & Operations → Infrastructure & Operations Graveyard
You need to log in
before you can comment on or make changes to this bug.
Description
•