Closed Bug 1030419 Opened 10 years ago Closed 4 years ago

Intermittent Android 4.3 (aws) /tests/layout/style/test/test_media_queries.html | Test timed out.

Categories

(Core :: CSS Parsing and Computation, defect, P5)

x86_64
Linux
defect

Tracking

()

RESOLVED DUPLICATE of bug 1579272

People

(Reporter: gbrown, Assigned: gbrown)

References

Details

test_media_queries.html has been running reliably on Android 2.3, in an emulator run on ix slaves. We are now moving Android 2.3 mochitests to aws slaves. On aws, test_media_queries.html intermittently fails: https://tbpl.mozilla.org/php/getParsedLog.php?id=42473078&full=1&branch=ash#error0 14:14:43 INFO - 177 INFO TEST-START | /tests/layout/style/test/test_media_queries.html 14:14:43 INFO - 178 INFO TEST-INFO | dumping last 100 message(s) 14:14:43 INFO - 179 INFO TEST-INFO | if you need more context, please use SimpleTest.requestCompleteLog() in your test 14:14:43 INFO - 180 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | Media query 'not all and (min-resolution: 39dpcm)' should apply after cloning 14:14:43 INFO - 181 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (scan)' should not change serialization 14:14:43 INFO - 182 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (scan: progressive)' should not change serialization 14:14:43 INFO - 183 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (scan:interlace)' should not change serialization 14:14:43 INFO - 184 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query '(scan)' should not change serialization 14:14:43 INFO - 185 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | Media query '(scan)' should NOT apply after cloning 14:14:43 INFO - 186 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query '(scan: progressive)' should not change serialization 14:14:43 INFO - 187 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | Media query '(scan: progressive)' should NOT apply after cloning 14:14:43 INFO - 188 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query '(scan: interlace)' should not change serialization 14:14:43 INFO - 189 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | Media query '(scan: interlace)' should NOT apply after cloning 14:14:43 INFO - 190 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'not all and (scan)' should not change serialization 14:14:43 INFO - 191 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | Media query 'not all and (scan)' should apply after cloning 14:14:43 INFO - 192 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'not all and (scan: progressive)' should not change serialization 14:14:43 INFO - 193 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | Media query 'not all and (scan: progressive)' should apply after cloning 14:14:43 INFO - 194 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'not all and (scan: interlace)' should not change serialization 14:14:43 INFO - 195 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | Media query 'not all and (scan: interlace)' should apply after cloning 14:14:43 INFO - 196 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (grid)' should not change serialization 14:14:43 INFO - 197 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (grid: 0)' should not change serialization 14:14:43 INFO - 198 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (grid: 1)' should not change serialization 14:14:43 INFO - 199 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (grid: 1)' should not change serialization 14:14:43 INFO - 200 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query '(grid)' should not change serialization 14:14:43 INFO - 201 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | Media query '(grid)' should NOT apply after cloning 14:14:43 INFO - 202 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query '(grid: 0)' should not change serialization 14:14:43 INFO - 203 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | Media query '(grid: 0)' should apply after cloning 14:14:43 INFO - 204 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query '(grid: 1)' should not change serialization 14:14:43 INFO - 205 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | Media query '(grid: 1)' should NOT apply after cloning 14:14:43 INFO - 206 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query '(grid: 2)' should not change serialization 14:14:43 INFO - 207 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | Media query '(grid: 2)' should NOT apply after cloning 14:14:43 INFO - 208 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query '(grid: -1)' should not change serialization 14:14:43 INFO - 209 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | Media query '(grid: -1)' should NOT apply after cloning 14:14:43 INFO - 210 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-scrollbar-start-backward)' should not change serialization 14:14:43 INFO - 211 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-scrollbar-start-forward)' should not change serialization 14:14:43 INFO - 212 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-scrollbar-end-backward)' should not change serialization 14:14:43 INFO - 213 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-scrollbar-end-forward)' should not change serialization 14:14:43 INFO - 214 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-scrollbar-thumb-proportional)' should not change serialization 14:14:43 INFO - 215 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-images-in-menus)' should not change serialization 14:14:43 INFO - 216 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-images-in-buttons)' should not change serialization 14:14:43 INFO - 217 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-overlay-scrollbars)' should not change serialization 14:14:43 INFO - 218 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-windows-default-theme)' should not change serialization 14:14:43 INFO - 219 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-mac-graphite-theme)' should not change serialization 14:14:43 INFO - 220 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-mac-lion-theme)' should not change serialization 14:14:43 INFO - 221 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-mac-yosemite-theme)' should not change serialization 14:14:43 INFO - 222 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-windows-compositor)' should not change serialization 14:14:43 INFO - 223 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-windows-classic)' should not change serialization 14:14:43 INFO - 224 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-windows-glass)' should not change serialization 14:14:43 INFO - 225 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-touch-enabled)' should not change serialization 14:14:43 INFO - 226 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-swipe-animation-enabled)' should not change serialization 14:14:43 INFO - 227 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-scrollbar-start-backward: 0)' should not change serialization 14:14:43 INFO - 228 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-scrollbar-start-forward: 0)' should not change serialization 14:14:43 INFO - 229 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-scrollbar-end-backward: 0)' should not change serialization 14:14:43 INFO - 230 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-scrollbar-end-forward: 0)' should not change serialization 14:14:43 INFO - 231 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-scrollbar-thumb-proportional: 0)' should not change serialization 14:14:43 INFO - 232 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-images-in-menus: 0)' should not change serialization 14:14:43 INFO - 233 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-images-in-buttons: 0)' should not change serialization 14:14:43 INFO - 234 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-overlay-scrollbars: 0)' should not change serialization 14:14:43 INFO - 235 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-windows-default-theme: 0)' should not change serialization 14:14:43 INFO - 236 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-mac-graphite-theme: 0)' should not change serialization 14:14:43 INFO - 237 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-mac-lion-theme: 0)' should not change serialization 14:14:43 INFO - 238 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-mac-yosemite-theme: 0)' should not change serialization 14:14:43 INFO - 239 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-windows-compositor: 0)' should not change serialization 14:14:43 INFO - 240 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-windows-classic: 0)' should not change serialization 14:14:43 INFO - 241 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-windows-glass: 0)' should not change serialization 14:14:43 INFO - 242 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-touch-enabled: 0)' should not change serialization 14:14:43 INFO - 243 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-swipe-animation-enabled: 0)' should not change serialization 14:14:43 INFO - 244 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-scrollbar-start-backward: 1)' should not change serialization 14:14:43 INFO - 245 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-scrollbar-start-forward: 1)' should not change serialization 14:14:43 INFO - 246 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-scrollbar-end-backward: 1)' should not change serialization 14:14:43 INFO - 247 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-scrollbar-end-forward: 1)' should not change serialization 14:14:43 INFO - 248 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-scrollbar-thumb-proportional: 1)' should not change serialization 14:14:43 INFO - 249 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-images-in-menus: 1)' should not change serialization 14:14:43 INFO - 250 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-images-in-buttons: 1)' should not change serialization 14:14:43 INFO - 251 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-overlay-scrollbars: 1)' should not change serialization 14:14:43 INFO - 252 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-windows-default-theme: 1)' should not change serialization 14:14:43 INFO - 253 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-mac-graphite-theme: 1)' should not change serialization 14:14:43 INFO - 254 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-mac-lion-theme: 1)' should not change serialization 14:14:43 INFO - 255 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-mac-yosemite-theme: 1)' should not change serialization 14:14:43 INFO - 256 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-windows-compositor: 1)' should not change serialization 14:14:43 INFO - 257 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-windows-classic: 1)' should not change serialization 14:14:43 INFO - 258 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-windows-glass: 1)' should not change serialization 14:14:43 INFO - 259 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-touch-enabled: 1)' should not change serialization 14:14:43 INFO - 260 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-swipe-animation-enabled: 1)' should not change serialization 14:14:43 INFO - 261 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-windows-theme: aero)' should not change serialization 14:14:43 INFO - 262 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-windows-theme: aero-lite)' should not change serialization 14:14:43 INFO - 263 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-windows-theme: luna-blue)' should not change serialization 14:14:43 INFO - 264 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-windows-theme: luna-olive)' should not change serialization 14:14:43 INFO - 265 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-windows-theme: luna-silver)' should not change serialization 14:14:43 INFO - 266 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-windows-theme: royale)' should not change serialization 14:14:43 INFO - 267 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-windows-theme: generic)' should not change serialization 14:14:43 INFO - 268 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-windows-theme: zune)' should not change serialization 14:14:43 INFO - 269 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-windows-theme: garbage)' should not change serialization 14:14:43 INFO - 270 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-os-version: windows-xp)' should not change serialization 14:14:43 INFO - 271 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-os-version: windows-vista)' should not change serialization 14:14:43 INFO - 272 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-os-version: windows-win7)' should not change serialization 14:14:43 INFO - 273 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-os-version: windows-win8)' should not change serialization 14:14:43 INFO - 274 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query '(-moz-is-glyph)' should not change serialization 14:14:43 INFO - 275 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-is-glyph)' should not change serialization 14:14:43 INFO - 276 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'not all and (-moz-is-glyph)' should not change serialization 14:14:43 INFO - 277 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'only all and (-moz-is-glyph)' should not change serialization 14:14:43 INFO - 278 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query '(-moz-is-glyph:0)' should not change serialization 14:14:43 INFO - 279 INFO TEST-PASS | /tests/layout/style/test/test_media_queries.html | cloning query 'all and (-moz-is-glyph:0)' should not change serialization 14:14:43 INFO - 280 INFO TEST-UNEXPECTED-FAIL | /tests/layout/style/test/test_media_queries.html | Test timed out. 14:14:43 INFO - 281 INFO TEST-INFO | MEMORY STAT vsize after test: 425168896 14:14:43 INFO - 282 INFO TEST-INFO | MEMORY STAT residentFast after test: 174690304 14:14:43 INFO - 283 INFO TEST-INFO | MEMORY STAT heapAllocated after test: 33629824 14:14:43 INFO - 284 INFO TEST-END | /tests/layout/style/test/test_media_queries.html | finished in 320264ms
Do we know how many CPU cycles per second of system clock advance we're guaranteed in this testing setup? (In particular, is it a number greater than 0?)
Flags: needinfo?(gbrown)
See Also: → 1030432
I do not know of any such guarantee. But, except for a handful of exceptions, all of the other mochitests seem to run reliably on Android 2.3 on aws, usually just a little slower than on ix.
Flags: needinfo?(gbrown)
Was this test almost at the timeout threshold before? Is the problem here that it's intermittently crossing just over the boundary, or intermittently hanging or taking massively longer? (One of the reasons we explicitly stopped running tests on VMs about 5 years ago was that our test timeouts were in system clock time, and the VMs we were using didn't seem to offer any guarantee about how much work we could do per any advance in wall clock time, so we had low frequency intermittent timeouts of just about everything.)
Whiteboard: [leave open] [test disabled on Android 2.3]
I don't think this test should be disabled without a better understanding of why it's timing out and whether the platform on which it's running is actually suitable for running our test suites. Could you at least answer the questions in comment 3?
Flags: needinfo?(gbrown)
Sorry David. I have not communicated well or even sufficiently here. Let me try to clarify what I'm doing and how we got here. Historically, we have run our Android tests on Tegra boards (Android 2.2) and Panda boards (Android 4.0). The older Tegra boards are now in short supply, prone to failure, and cannot be replaced. Independent of test considerations, the Firefox for Android team has decided to end support for Android 2.2, making Android 2.3 the earliest supported version. For some months now, we have worked to set up a new test environment based on the Android emulator: Android 2.3 is run in the Android arm emulator, and the emulator is run on an in-house ix slave running Linux. This removes the need and cost of purchasing, housing, cabling, etc device boards. As on any new test platform, there have been numerous new cases of test failures -- some of those have been investigated and resolved, but others remain. Bug 979921 tracks tests failures on the new Android 2.3 platform. Many of the remaining failures are consequences of bug 1025968 (SwapBuffers crash seen in many robocop tests) and bug 1020227 (deadlock in opensl_stream_destroy). A few weeks ago, we thought that most of the test failures and all other issues were sorted out and we started running a full suite of Android 2.3 tests on trunk trees on the emulator, on ix slaves. Test failure rates have been quite low (typically 10x lower than Android 2.2 tegra rates). Test run times for reftests have been about 3x slower than tegras, while most other tests, like mochitests, have run times similar to tegras. However, when we tried to introduce Android 2.3 *armv6* tests on trunk trees, we ran into a capacity problem: We don't have enough ix slaves to service the test load. To avoid the ix capacity issue, we looked at running on aws (preferred also for scalability and cost). Running reftests on emulator-on-aws seems impractical -- they take at least 4x as long as on ix. But other test types run in about the same time (looking at the average time for test jobs), or up to 50% longer (we had to increase mochitest chunks from 8 to 12 for aws). Running {mochitest, robocop, xpcshell} on aws seems practical: With a few exceptions, like this test, tests run reliably and just a little slower on average than on ix. If we disable or fix this test and a few others on Android 2.3 on aws, we can run Android 2.3 and Android 2.3 armv6 tests on trunk trees, retire the Android 2.2 tests on Tegras, and the Firefox for Android team can end support for Android 2.2 in Firefox 33, or perhaps even 32. An alternative is to order more ix slaves, but there is both cost and delay involved in that decision and that seems unwarranted if only a handful of tests fail on aws. I wish we could have identified this failure earlier in this project, but the ix capacity issue and switch to aws was a last minute surprise. Now there is a lot of pressure to get these tests running on trunk ASAP. Now, as for test_media_queries... > Was this test almost at the timeout threshold before? Is the problem here that it's intermittently > crossing just over the boundary, or intermittently hanging or taking massively longer? It is taking much longer on 2.3-aws compared to 2.3-ix or 2.2-tegra: 2.3-aws: test_media_queries.html | finished in 320264ms (timeout) 2.3-ix: test_media_queries.html | finished in 62252ms 2.2-tegra: test_media_queries.html | finished in 25131ms On 2.3-aws, it fails frequently (40%?). When it passes, it takes a long time, but typically 60 seconds less than the timeout threshold: 2.3-aws: test_media_queries.html | finished in 266679ms (pass) 2.3-aws: test_media_queries.html | finished in 243644ms (pass) 2.3-aws: test_media_queries.html | finished in 260503ms (pass) There are several retries of Android 2.3 Armv6 Opt mochitest-12 at https://tbpl.mozilla.org/?tree=Ash&rev=c135c74b985e, showing both failures and passes. Let me know if I can provide any other information -- I want to see this running again too.
Flags: needinfo?(gbrown)
Priority: -- → P5
See Also: → 1118001
Whiteboard: [leave open] [test disabled on Android 2.3] → [leave open] [test disabled on Android 2.3] [test disabled on Android 4.3]
Android 2.3 is no longer supported in Firefox 48+. Test manifests were updated in bug 1251013. This test remains disabled on Android 4.3.
Whiteboard: [leave open] [test disabled on Android 2.3] [test disabled on Android 4.3] → [test disabled on Android 4.3]
Summary: Intermittent Android 2.3 (aws) /tests/layout/style/test/test_media_queries.html | Test timed out. → Intermittent Android 4.3 (aws) /tests/layout/style/test/test_media_queries.html | Test timed out.

This test has been running on android since bug 1579272.

Assignee: nobody → whole.grains
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → DUPLICATE
Whiteboard: [test disabled on Android 4.3]
You need to log in before you can comment on or make changes to this bug.