981898 - Android 2.3 /tests/content/media/test/test_mediarecorder_record_immediate_stop.html (and others) | Test timed out.

Status: RESOLVED FIXED

(Core :: Audio/Video: Recording, defect)

Description

[Geoff Brown [:gbrown]]

On the Android 2.3 emulator, several tests in content/media/tests fail frequently and have been disabled for the initial run of these tests.

https://tbpl.mozilla.org/php/getParsedLog.php?id=34456080&tree=Ash&full=1#error0

7:44:03     INFO -  686 INFO TEST-START | /tests/content/media/test/test_mediarecorder_record_4ch_audiocontext.html
17:44:03     INFO -  687 ERROR TEST-UNEXPECTED-FAIL | /tests/content/media/test/test_mediarecorder_record_4ch_audiocontext.html | check blob has data
17:44:03     INFO -  688 INFO TEST-INFO | MEMORY STAT vsize after test: 451207168
17:44:03     INFO -  689 INFO TEST-INFO | MEMORY STAT residentFast after test: 155455488
17:44:03     INFO -  690 INFO TEST-INFO | MEMORY STAT heapAllocated after test: 30314884
17:44:03     INFO -  691 INFO TEST-END | /tests/content/media/test/test_mediarecorder_record_4ch_audiocontext.html | finished in 7971ms
17:44:03     INFO -  692 INFO TEST-START | /tests/content/media/test/test_mediarecorder_record_audiocontext.html
17:44:03     INFO -  693 INFO TEST-INFO | MEMORY STAT vsize after test: 452255744
17:44:03     INFO -  694 INFO TEST-INFO | MEMORY STAT residentFast after test: 154439680
17:44:03     INFO -  695 INFO TEST-INFO | MEMORY STAT heapAllocated after test: 29905940
17:44:03     INFO -  696 INFO TEST-END | /tests/content/media/test/test_mediarecorder_record_audiocontext.html | finished in 5960ms
17:44:03     INFO -  697 INFO TEST-START | /tests/content/media/test/test_mediarecorder_record_immediate_stop.html
17:44:03     INFO -  698 INFO TEST-INFO | /tests/content/media/test/test_mediarecorder_record_immediate_stop.html | Started Mon Feb 10 2014 17:14:51 GMT-0800 (PST) (1392081291.321s)
17:44:03     INFO -  699 ERROR TEST-UNEXPECTED-FAIL | /tests/content/media/test/test_mediarecorder_record_immediate_stop.html | Test timed out.
17:44:03     INFO -  700 INFO TEST-INFO | MEMORY STAT vsize after test: 452128768
17:44:03     INFO -  701 INFO TEST-INFO | MEMORY STAT residentFast after test: 154705920
17:44:03     INFO -  702 INFO TEST-INFO | MEMORY STAT heapAllocated after test: 31136472
17:44:03     INFO -  703 INFO TEST-END | /tests/content/media/test/test_mediarecorder_record_immediate_stop.html | finished in 318984ms
17:44:03     INFO -  704 INFO TEST-START | /tests/content/media/test/test_mediarecorder_record_no_timeslice.html
17:44:03     INFO -  705 INFO TEST-INFO | /tests/content/media/test/test_mediarecorder_record_no_timeslice.html | Started Mon Feb 10 2014 17:20:10 GMT-0800 (PST) (1392081610.599s)
17:44:03     INFO -  706 ERROR TEST-UNEXPECTED-FAIL | /tests/content/media/test/test_mediarecorder_record_no_timeslice.html | Test timed out.
17:44:03     INFO -  707 INFO TEST-INFO | MEMORY STAT vsize after test: 448983040
17:44:03     INFO -  708 INFO TEST-INFO | MEMORY STAT residentFast after test: 155013120
17:44:03     INFO -  709 INFO TEST-INFO | MEMORY STAT heapAllocated after test: 29597436
17:44:03     INFO -  710 INFO TEST-END | /tests/content/media/test/test_mediarecorder_record_no_timeslice.html | finished in 329133ms
17:44:03     INFO -  711 INFO TEST-START | /tests/content/media/test/test_mediarecorder_record_nosrc.html
17:44:03     INFO -  712 INFO TEST-INFO | /tests/content/media/test/test_mediarecorder_record_nosrc.html | create MediaRecorder
17:44:03     INFO -  713 INFO TEST-INFO | MEMORY STAT vsize after test: 450031616
17:44:03     INFO -  714 INFO TEST-INFO | MEMORY STAT residentFast after test: 155201536
17:44:03     INFO -  715 INFO TEST-INFO | MEMORY STAT heapAllocated after test: 30320304
17:44:03     INFO -  716 INFO TEST-END | /tests/content/media/test/test_mediarecorder_record_nosrc.html | finished in 2618ms
17:44:03     INFO -  717 INFO TEST-START | /tests/content/media/test/test_mediarecorder_record_session.html
17:44:03     INFO -  718 INFO TEST-INFO | /tests/content/media/test/test_mediarecorder_record_session.html | Started Mon Feb 10 2014 17:25:44 GMT-0800 (PST) (1392081944.675s)
17:44:03     INFO -  719 ERROR TEST-UNEXPECTED-FAIL | /tests/content/media/test/test_mediarecorder_record_session.html | Test timed out.
17:44:03     INFO -  720 INFO TEST-INFO | MEMORY STAT vsize after test: 448983040
17:44:03     INFO -  721 INFO TEST-INFO | MEMORY STAT residentFast after test: 155254784
17:44:03     INFO -  722 INFO TEST-INFO | MEMORY STAT heapAllocated after test: 30294704
17:44:03     INFO -  723 INFO TEST-END | /tests/content/media/test/test_mediarecorder_record_session.html | finished in 325051ms
17:44:03     INFO -  724 INFO TEST-START | /tests/content/media/test/test_mediarecorder_record_startstopstart.html
17:44:03     INFO -  725 ERROR TEST-UNEXPECTED-FAIL | /tests/content/media/test/test_mediarecorder_record_startstopstart.html | Test timed out.
17:44:03     INFO -  726 ERROR TEST-UNEXPECTED-FAIL | (SimpleTest/TestRunner.js) | 4 test timeouts, giving up.
17:44:03     INFO -  727 ERROR TEST-UNEXPECTED-FAIL | (SimpleTest/TestRunner.js) | Skipping 139 remaining tests.

Comment 1

[Geoff Brown [:gbrown]]

Most Android 2.3 mochitests are running fine now. The content/media tests are disabled. I tried enabling them on try, but errors persist:

https://tbpl.mozilla.org/php/getParsedLog.php?id=40289291&tree=Try&full=1

Comment 2

[Geoff Brown [:gbrown]]

It seems there are similar, but perhaps less frequent, problems with content/media tests on other Android platforms, and perhaps other platforms too.

Note that for Android 2.3, many tests fail frequently, so the entire content/media/test directory is skipped on Android 2.3.

Comment 3

[JW Wang [:jwwang] [:jw_wang]]

Benjamin will look at test cases about media recorder. Once that is fixed, we should be able to enable content/media tests on Android 2.3.

Btw, we have multiple files to disable tests which is kinda confusing. (content/media/test/mochitest.ini and testing/mochitest/android23.json) Is it because "skip-if" in mochitest.ini doesn't support a specific platform version?

Comment 4

[Geoff Brown [:gbrown]]

(In reply to JW Wang [:jwwang] from comment #3)
> Benjamin will look at test cases about media recorder. Once that is fixed,
> we should be able to enable content/media tests on Android 2.3.

That's good news. We are holding off on retiring the Android 2.2/Tegra test platform until we can get these last tests running on Android 2.3, so I am eager to see this sorted out.

> Btw, we have multiple files to disable tests which is kinda confusing.
> (content/media/test/mochitest.ini and testing/mochitest/android23.json) Is
> it because "skip-if" in mochitest.ini doesn't support a specific platform
> version?

Yes, that's basically it. We certainly want to have just one place to disable tests, and mochitest.ini is the preferred location. We added support for skipping tests in mochitest.ini based on Android version, but now we are using mochitest.ini for build-time decisions, while Android version is only determined at test time -- so we had to introduce android23.json for cases like this.

Comment 5

[Maire Reavy [:mreavy]]

Hi JW -- Can you estimate how much work it would be to get these tests working for Android 2.3?  (It sounds like you're close to having a solution.)

Comment 6

[JW Wang [:jwwang] [:jw_wang]]

Hi Maire,
Benjamin is working on media recorder tests. He should be a better person to estimate the effort. The major effort is bug 969372 where we had a workaround first to keep tests running and then we will revert the workaround once we have a fix. The workaround should also work for other media recorder test failures. So if the schedule is tight, I would suggest to check-in the workaround first so that Benjamin has more time working in the proper fix.

Also the blocker of enabling media tests on Android 2.3 is bug 1020227 which I don't have a proper fix yet. The temp fix can reduce the failure rate from 8/10 to nearly 1/300, though not perfect, which should work somehow for us to enable the media tests.

Comment 7

[Benjamin Chen [:bechen]]

(In reply to JW Wang [:jwwang] from comment #6)
> Hi Maire,
> Benjamin is working on media recorder tests. He should be a better person to
> estimate the effort. The major effort is bug 969372 where we had a
> workaround first to keep tests running and then we will revert the
> workaround once we have a fix. The workaround should also work for other
> media recorder test failures. So if the schedule is tight, I would suggest
> to check-in the workaround first so that Benjamin has more time working in
> the proper fix.

I guess bug 969372 will be really fixed at next week, include code check-in, maybe next Wednesday.
If there is no more accident...

Comment 8

[Geoff Brown [:gbrown]]

Following resolution of bug 1020227, I found these were running reliably:

https://tbpl.mozilla.org/?tree=Try&rev=f0c43f76559e
https://tbpl.mozilla.org/?tree=Try&rev=11b7993b83b7

content/media/test re-enabled on Android 2.3:

https://hg.mozilla.org/integration/mozilla-inbound/rev/9a77f132eed3

Comment 9

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/9a77f132eed3