Closed Bug 1357639 Opened 7 years ago Closed 7 years ago

Add Fennec test for media control and audio focus

Categories

(Firefox for Android Graveyard :: Audio/Video, enhancement)

Product:
Firefox for Android Graveyard
Component:
Audio/Video
Type:
enhancement
Priority:
Not set
Severity:
normal

Tracking

(firefox55 fixed)

Status:
RESOLVED FIXED
Milestone:
Firefox 55
Tracking Flags:
Tracking Status
firefox55 --- fixed

People

(Reporter: alwu, Assigned: alwu)

References

Details

Attachments

(11 files)

Bug 1357639 - part1 : change tab's media playing state when the tab is controlled by media control.
59 bytes, text/x-review-board-request
sebastian
: review+
Details
Bug 1357639 - part2 : use enum for audio focus states.
59 bytes, text/x-review-board-request
sebastian
: review+
Details
Bug 1357639 - part3 : change audio focus state before notifying observers.
59 bytes, text/x-review-board-request
sebastian
: review+
Details
Bug 1357639 - part4 : add Fennec test for media control and audio focus.
59 bytes, text/x-review-board-request
sebastian
: review+
Details
Bug 1357639 - part5 : create new type test class for media playback.
59 bytes, text/x-review-board-request
sebastian
: review+
Details
Bug 1357639 - part6 : list all test tasks in the ctor.
59 bytes, text/x-review-board-request
sebastian
: review+
Details
Bug 1357639 - part7 : add check for notification's content.
59 bytes, text/x-review-board-request
sebastian
: review+
Details
Bug 1357639 - part8 : add annotiation @RobocopTarget.
59 bytes, text/x-review-board-request
sebastian
: review+
Details
Bug 1357639 - part9 : wait tab event to make sure media started successfully.
59 bytes, text/x-review-board-request
sebastian
: review+
Details
Bug 1357639 - part10 : check notification id.
59 bytes, text/x-review-board-request
sebastian
: review+
Details
Bug 1357639 - part11 : increase the waiting time for notification UI changed.
59 bytes, text/x-review-board-request
sebastian
: review+
Details 
Although we can't write UI test for media control and audio focus now, but we should add some chrome mochitests for them.
Summary: Add chrome mochitest for media control and audio focus → Add Fennec test for media control and audio focus
Finish the preliminary version, and also ensure the basic stability.
Now I'm modifying some details and will ask for a review soon.
Blocks: 1357633
Comment on attachment 8861792 [details]
Bug 1357639 - part2 : use enum for audio focus states.

https://reviewboard.mozilla.org/r/133792/#review138400
Attachment #8861792 - Flags: review?(s.kaspari) → review+
Comment on attachment 8861793 [details]
Bug 1357639 - part3 : change audio focus state before notifying observers.

https://reviewboard.mozilla.org/r/133794/#review138402
Attachment #8861793 - Flags: review?(s.kaspari) → review+
Comment on attachment 8861794 [details]
Bug 1357639 - part4 : add Fennec test for media control and audio focus.

https://reviewboard.mozilla.org/r/133796/#review138404
Attachment #8861794 - Flags: review?(s.kaspari) → review+
Comment on attachment 8862311 [details]
Bug 1357639 - part5 : create new type test class for media playback.

https://reviewboard.mozilla.org/r/134242/#review138406

Nice!
Attachment #8862311 - Flags: review?(s.kaspari) → review+
Comment on attachment 8862312 [details]
Bug 1357639 - part6 : list all test tasks in the ctor.

https://reviewboard.mozilla.org/r/134244/#review138408
Attachment #8862312 - Flags: review?(s.kaspari) → review+
Comment on attachment 8862778 [details]
Bug 1357639 - part7 : add check for notification's content.

https://reviewboard.mozilla.org/r/134618/#review138410
Attachment #8862778 - Flags: review?(s.kaspari) → review+
Comment on attachment 8859529 [details]
Bug 1357639 - part1 : change tab's media playing state when the tab is controlled by media control.

https://reviewboard.mozilla.org/r/131534/#review138398

::: mobile/android/base/java/org/mozilla/gecko/media/MediaControlService.java:338
(Diff revision 6)
> +    private void setMediaStateForTab(boolean isTabPlaying) {
> +        Tab tab = mTabReference.get();
> +        if (tab == null) {
> +            return;
> +        }
> +        tab.setIsMediaPlaying(isTabPlaying);
> +    }

Can you help me understand why we need this? Doesn't the tab code already take care of the is playing flag? It seems strange that we need to do this here too?
(In reply to Sebastian Kaspari (:sebastian) from comment #27) 
> Can you help me understand why we need this? Doesn't the tab code already
> take care of the is playing flag? It seems strange that we need to do this
> here too?

The |Tab:MediaPlaybackChange| event is only dispatched when media started or paused from page, so we forgot to set the value for it on other cases (eg. pause from control, resume from control, resume from page)
(In reply to Alastor Wu [:alwu][please needinfo? me] from comment #28)
> (In reply to Sebastian Kaspari (:sebastian) from comment #27) 
> > Can you help me understand why we need this? Doesn't the tab code already
> > take care of the is playing flag? It seems strange that we need to do this
> > here too?
> 
> The |Tab:MediaPlaybackChange| event is only dispatched when media started or
> paused from page, so we forgot to set the value for it on other cases (eg.
> pause from control, resume from control, resume from page)

Got it. Thanks!
Comment on attachment 8859529 [details]
Bug 1357639 - part1 : change tab's media playing state when the tab is controlled by media control.

https://reviewboard.mozilla.org/r/131534/#review138436

::: mobile/android/base/java/org/mozilla/gecko/media/MediaControlService.java:339
(Diff revision 7)
>          });
>  
>      }
>  
> +    private void setMediaStateForTab(boolean isTabPlaying) {
> +        Tab tab = mTabReference.get();

nit: final
Attachment #8859529 - Flags: review?(s.kaspari) → review+
Comment on attachment 8863667 [details]
Bug 1357639 - part8 : add annotiation @RobocopTarget.

https://reviewboard.mozilla.org/r/135472/#review138438
Attachment #8863667 - Flags: review?(s.kaspari) → review+
Pushed by alwu@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/ecf487cbca1b
part1 : change tab's media playing state when the tab is controlled by media control. r=sebastian
https://hg.mozilla.org/integration/autoland/rev/48e901332b4d
part2 : use enum for audio focus states. r=sebastian
https://hg.mozilla.org/integration/autoland/rev/3e7bc9a792c2
part3 : change audio focus state before notifying observers. r=sebastian
https://hg.mozilla.org/integration/autoland/rev/acb72bf2db09
part4 : add Fennec test for media control and audio focus. r=sebastian
https://hg.mozilla.org/integration/autoland/rev/916edbdfece6
part5 : create new type test class for media playback. r=sebastian
https://hg.mozilla.org/integration/autoland/rev/4b332b74aba6
part6 : list all test tasks in the ctor. r=sebastian
https://hg.mozilla.org/integration/autoland/rev/99c71dd11632
part7 : add check for notification's content. r=sebastian
https://hg.mozilla.org/integration/autoland/rev/d37e0ce6c7c9
part8 : add annotiation @RobocopTarget. r=sebastian
Backed out for frequently failing robocop's testAudioFocus:

https://hg.mozilla.org/integration/autoland/rev/7513e906184322b58e8a9010f8385ea57970f372
https://hg.mozilla.org/integration/autoland/rev/28500031e15ef8eb8c679499c2e14e3d7b948015
https://hg.mozilla.org/integration/autoland/rev/8278c46a2ecc15975d64aa5bf440910841a583ba
https://hg.mozilla.org/integration/autoland/rev/56f905e5139bfe91bd4bd7886aff7d6a6e3fbb4b
https://hg.mozilla.org/integration/autoland/rev/a884e2a71d2107c05822883c716a84e6f1648d99
https://hg.mozilla.org/integration/autoland/rev/963d8157d72df0901f90f84925d4eda2c5a228f7
https://hg.mozilla.org/integration/autoland/rev/7116303fc8693672718de413a5d6b912ed946784
https://hg.mozilla.org/integration/autoland/rev/90f4f6a0b80e9b07dc557f2bb57f20ff7061d54a

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=d37e0ce6c7c9c28a0b991cc9d2a34390a127d7a0&filter-resultStatus=testfailed&filter-resultStatus=busted&filter-resultStatus=exception&filter-resultStatus=retry&filter-resultStatus=usercancel&filter-resultStatus=runnable
Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=96033710&repo=autoland

[task 2017-05-02T19:15:09.249501Z] 19:15:09     INFO -  TEST-PASS | testAudioFocus | Given message occurred for registered event: {metadata=null, bgColor=rgba(0, 0, 0, 0), type=null, tabID=0, errorType=} - Content:DOMContentLoaded should equal Content:DOMContentLoaded
[task 2017-05-02T19:15:09.249627Z] 19:15:09     INFO -  - wait audio starts playing -
[task 2017-05-02T19:15:09.249768Z] 19:15:09     INFO -  Buffered messages logged at 19:14:53
[task 2017-05-02T19:15:09.249992Z] 19:15:09     INFO -  waitForCondition timeout after 4500 ms.
[task 2017-05-02T19:15:09.250198Z] 19:15:09     INFO -  Buffered messages finished
[task 2017-05-02T19:15:09.250534Z] 19:15:09  WARNING -  TEST-UNEXPECTED-FAIL | testAudioFocus | Should request audio focus after media started playing. - got LOST_FOCUS, expected OWN_FOCUS
[task 2017-05-02T19:15:09.251040Z] 19:15:09     INFO -  0 ERROR Exception caught during test! - junit.framework.AssertionFailedError: TEST-UNEXPECTED-FAIL | testAudioFocus | Should request audio focus after media started playing. - got LOST_FOCUS, expected OWN_FOCUS
[task 2017-05-02T19:15:09.251246Z] 19:15:09     INFO -  	at junit.framework.Assert.fail(Assert.java:50)
[task 2017-05-02T19:15:09.251391Z] 19:15:09     INFO -  	at org.mozilla.gecko.FennecMochitestAssert._logMochitestResult(FennecMochitestAssert.java:124)
[task 2017-05-02T19:15:09.251704Z] 19:15:09     INFO -  	at org.mozilla.gecko.FennecMochitestAssert.ok(FennecMochitestAssert.java:145)
[task 2017-05-02T19:15:09.251850Z] 19:15:09     INFO -  	at org.mozilla.gecko.FennecMochitestAssert.is(FennecMochitestAssert.java:150)
[task 2017-05-02T19:15:09.252063Z] 19:15:09     INFO -  	at org.mozilla.gecko.tests.testAudioFocus.testAudioFocusLoadTab(testAudioFocus.java:96)
[task 2017-05-02T19:15:09.252292Z] 19:15:09     INFO -  	at org.mozilla.gecko.tests.testAudioFocus.testAudioFocus(testAudioFocus.java:20)
[task 2017-05-02T19:15:09.252474Z] 19:15:09     INFO -  	at java.lang.reflect.Method.invokeNative(Native Method)
[task 2017-05-02T19:15:09.252768Z] 19:15:09     INFO -  	at java.lang.reflect.Method.invoke(Method.java:525)
[task 2017-05-02T19:15:09.252902Z] 19:15:09     INFO -  	at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:214)
[task 2017-05-02T19:15:09.253129Z] 19:15:09     INFO -  	at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:199)
[task 2017-05-02T19:15:09.253327Z] 19:15:09     INFO -  	at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:192)
[task 2017-05-02T19:15:09.253387Z] 19:15:09     INFO -  	at org.mozilla.gecko.tests.BaseRobocopTest.runTest(BaseRobocopTest.java:188)
[task 2017-05-02T19:15:09.253476Z] 19:15:09     INFO -  	at junit.framework.TestCase.runBare(TestCase.java:134)
[task 2017-05-02T19:15:09.253761Z] 19:15:09     INFO -  	at junit.framework.TestResult$1.protect(TestResult.java:115)
[task 2017-05-02T19:15:09.253910Z] 19:15:09     INFO -  	at junit.framework.TestResult.runProtected(TestResult.java:133)
[task 2017-05-02T19:15:09.254207Z] 19:15:09     INFO -  	at junit.framework.TestResult.run(TestResult.java:118)
[task 2017-05-02T19:15:09.254349Z] 19:15:09     INFO -  	at junit.framework.TestCase.run(TestCase.java:124)
[task 2017-05-02T19:15:09.254666Z] 19:15:09     INFO -  	at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:191)
[task 2017-05-02T19:15:09.254816Z] 19:15:09     INFO -  	at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:176)
[task 2017-05-02T19:15:09.255017Z] 19:15:09     INFO -  	at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:554)
[task 2017-05-02T19:15:09.255296Z] 19:15:09     INFO -  	at org.mozilla.gecko.FennecInstrumentationTestRunner.onStart(FennecInstrumentationTestRunner.java:64)
[task 2017-05-02T19:15:09.255679Z] 19:15:09     INFO -  	at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1701)
[task 2017-05-02T19:15:09.255901Z] 19:15:09  WARNING -  TEST-UNEXPECTED-FAIL | testAudioFocus | Exception caught - junit.framework.AssertionFailedError: TEST-UNEXPECTED-FAIL | testAudioFocus | Should request audio focus after media started playing. - got LOST_FOCUS, expected OWN_FOCUS
Flags: needinfo?(alwu)
Hi, Wes,
In comment49, the patches were back-out, but you landed the patches in comment 50 again. I'm confused what's the present state?
Thanks!
It seems something went wrong for bugzilla that I can't add NI flag...
Flags: needinfo?(wkocher)
I think I accidentally merged to m-c from a point between this landing and it getting backed out. The backout commits got pushed to m-c by tomcat a few hours ago, so this should be reopened.
Status: RESOLVED → REOPENED
status-firefox55: fixed → ---
Flags: needinfo?(wkocher)
Resolution: FIXED → ---
Target Milestone: Firefox 55 → ---
Pushed by alwu@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/6fa5520670a3
part1 : change tab's media playing state when the tab is controlled by media control. r=sebastian
https://hg.mozilla.org/integration/autoland/rev/608cca1e6059
part2 : use enum for audio focus states. r=sebastian
https://hg.mozilla.org/integration/autoland/rev/090286c03592
part3 : change audio focus state before notifying observers. r=sebastian
https://hg.mozilla.org/integration/autoland/rev/3c76fdbce387
part4 : add Fennec test for media control and audio focus. r=sebastian
https://hg.mozilla.org/integration/autoland/rev/adcc6f84e6c2
part5 : create new type test class for media playback. r=sebastian
https://hg.mozilla.org/integration/autoland/rev/6f192315ba79
part6 : list all test tasks in the ctor. r=sebastian
https://hg.mozilla.org/integration/autoland/rev/3cf46eb1a111
part7 : add check for notification's content. r=sebastian
https://hg.mozilla.org/integration/autoland/rev/7ded779bd7fc
part8 : add annotiation @RobocopTarget. r=sebastian
Backed out for frequently failing testAudioFocus on Android:

https://hg.mozilla.org/integration/autoland/rev/90becadc7978086b796ecba13dac7740d0e9899a
https://hg.mozilla.org/integration/autoland/rev/e85f2c69f55e9347ccac411feb5bfe31df0140e1
https://hg.mozilla.org/integration/autoland/rev/e0c1c4a330829d9aec44ed4aa4211d2914600733
https://hg.mozilla.org/integration/autoland/rev/32a89a5739234939dc4b8ac8819e492d20033e8b
https://hg.mozilla.org/integration/autoland/rev/a41e2c219725e9aefaa73703abb1e81551ee7faa
https://hg.mozilla.org/integration/autoland/rev/19a585810f74ba5cb3c15f9dac55e398f93597e2
https://hg.mozilla.org/integration/autoland/rev/3ea8b12db442a40d831133b6e415839d7e572f02
https://hg.mozilla.org/integration/autoland/rev/2c15da24390697149ba93c6cbae7db482531a674

Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=96633790&repo=autoland

[task 2017-05-04T17:04:40.130173Z] 17:04:40     INFO -  TEST-PASS | testAudioFocus | Should not request audio focus before media starts. - LOST_FOCUS should equal LOST_FOCUS
[task 2017-05-04T17:04:40.130489Z] 17:04:40     INFO -  - load URL with looping audio file -
[task 2017-05-04T17:04:40.130724Z] 17:04:40     INFO -  Buffered messages logged at 17:04:16
[task 2017-05-04T17:04:40.131096Z] 17:04:40     INFO -  TEST-PASS | testAudioFocus | Given message occurred for registered event: {metadata=null, bgColor=rgba(0, 0, 0, 0), type=null, tabID=0, errorType=} - Content:DOMContentLoaded should equal Content:DOMContentLoaded
[task 2017-05-04T17:04:40.131473Z] 17:04:40     INFO -  - wait audio starts playing -
[task 2017-05-04T17:04:40.131749Z] 17:04:40     INFO -  Buffered messages logged at 17:04:20
[task 2017-05-04T17:04:40.132029Z] 17:04:40     INFO -  waitForCondition timeout after 4500 ms.
[task 2017-05-04T17:04:40.132307Z] 17:04:40     INFO -  Buffered messages finished
[task 2017-05-04T17:04:40.132667Z] 17:04:40  WARNING -  TEST-UNEXPECTED-FAIL | testAudioFocus | Should request audio focus after media started playing. - got LOST_FOCUS, expected OWN_FOCUS
[task 2017-05-04T17:04:40.133184Z] 17:04:40     INFO -  0 ERROR Exception caught during test! - junit.framework.AssertionFailedError: TEST-UNEXPECTED-FAIL | testAudioFocus | Should request audio focus after media started playing. - got LOST_FOCUS, expected OWN_FOCUS
[task 2017-05-04T17:04:40.133343Z] 17:04:40     INFO -  	at junit.framework.Assert.fail(Assert.java:50)
[task 2017-05-04T17:04:40.133582Z] 17:04:40     INFO -  	at org.mozilla.gecko.FennecMochitestAssert._logMochitestResult(FennecMochitestAssert.java:124)
[task 2017-05-04T17:04:40.133808Z] 17:04:40     INFO -  	at org.mozilla.gecko.FennecMochitestAssert.ok(FennecMochitestAssert.java:145)
[task 2017-05-04T17:04:40.134054Z] 17:04:40     INFO -  	at org.mozilla.gecko.FennecMochitestAssert.is(FennecMochitestAssert.java:150)
[task 2017-05-04T17:04:40.134290Z] 17:04:40     INFO -  	at org.mozilla.gecko.tests.testAudioFocus.testAudioFocusChanged(testAudioFocus.java:93)
[task 2017-05-04T17:04:40.134544Z] 17:04:40     INFO -  	at org.mozilla.gecko.tests.testAudioFocus.testAudioFocus(testAudioFocus.java:20)
[task 2017-05-04T17:04:40.134766Z] 17:04:40     INFO -  	at java.lang.reflect.Method.invokeNative(Native Method)
[task 2017-05-04T17:04:40.135109Z] 17:04:40     INFO -  	at java.lang.reflect.Method.invoke(Method.java:525)
[task 2017-05-04T17:04:40.135317Z] 17:04:40     INFO -  	at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:214)
[task 2017-05-04T17:04:40.135569Z] 17:04:40     INFO -  	at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:199)
[task 2017-05-04T17:04:40.135836Z] 17:04:40     INFO -  	at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:192)
[task 2017-05-04T17:04:40.136039Z] 17:04:40     INFO -  	at org.mozilla.gecko.tests.BaseRobocopTest.runTest(BaseRobocopTest.java:188)
[task 2017-05-04T17:04:40.136673Z] 17:04:40     INFO -  	at junit.framework.TestCase.runBare(TestCase.java:134)
[task 2017-05-04T17:04:40.136739Z] 17:04:40     INFO -  	at junit.framework.TestResult$1.protect(TestResult.java:115)
[task 2017-05-04T17:04:40.136845Z] 17:04:40     INFO -  	at junit.framework.TestResult.runProtected(TestResult.java:133)
[task 2017-05-04T17:04:40.137032Z] 17:04:40     INFO -  	at junit.framework.TestResult.run(TestResult.java:118)
[task 2017-05-04T17:04:40.137255Z] 17:04:40     INFO -  	at junit.framework.TestCase.run(TestCase.java:124)
[task 2017-05-04T17:04:40.137514Z] 17:04:40     INFO -  	at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:191)
[task 2017-05-04T17:04:40.138123Z] 17:04:40     INFO -  	at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:176)
[task 2017-05-04T17:04:40.138204Z] 17:04:40     INFO -  	at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:554)
[task 2017-05-04T17:04:40.138273Z] 17:04:40     INFO -  	at org.mozilla.gecko.FennecInstrumentationTestRunner.onStart(FennecInstrumentationTestRunner.java:64)
[task 2017-05-04T17:04:40.138431Z] 17:04:40     INFO -  	at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1701)
[task 2017-05-04T17:04:40.138763Z] 17:04:40  WARNING -  TEST-UNEXPECTED-FAIL | testAudioFocus | Exception caught - junit.framework.AssertionFailedError: TEST-UNEXPECTED-FAIL | testAudioFocus | Should request audio focus after media started playing. - got LOST_FOCUS, expected OWN_FOCUS
Flags: needinfo?(alwu)
Comment on attachment 8864765 [details]
Bug 1357639 - part9 : wait tab event to make sure media started successfully.

https://reviewboard.mozilla.org/r/136424/#review140040
Attachment #8864765 - Flags: review?(s.kaspari) → review+
Comment on attachment 8864800 [details]
Bug 1357639 - part10 : check notification id.

https://reviewboard.mozilla.org/r/136478/#review140044

::: mobile/android/base/java/org/mozilla/gecko/media/MediaControlService.java:50
(Diff revision 2)
>      public static final String ACTION_RESUME_BY_AUDIO_FOCUS = "action_resume_audio_focus";
>      public static final String ACTION_PAUSE_BY_AUDIO_FOCUS  = "action_pause_audio_focus";
>      public static final String ACTION_START_AUDIO_DUCK      = "action_start_audio_duck";
>      public static final String ACTION_STOP_AUDIO_DUCK       = "action_stop_audio_duck";
> -    private static final int MEDIA_CONTROL_ID = 1;
> +    public static final String MEDIA_CONTROL_NOTIFICATION = "media-control-notification";
> +    private static final int MEDIA_CONTROL_ID = MEDIA_CONTROL_NOTIFICATION.hashCode();

Why hashCode? This makes the id uncontrollable and you might end up overriding other notifications we show. This might be a problem we faced with "1" too - We should start to create a central place for defining those Ids to make sure they are unique (I wonder what push uses) - anyways: hashCode() might be not the best we can do here.
Attachment #8864800 - Flags: review?(s.kaspari) → review-
Comment on attachment 8865284 [details]
Bug 1357639 - part11 : increase the waiting time for notification UI changed.

https://reviewboard.mozilla.org/r/136922/#review140048
Attachment #8865284 - Flags: review?(s.kaspari) → review+
Blocks: 1351087
Comment on attachment 8864800 [details]
Bug 1357639 - part10 : check notification id.

https://reviewboard.mozilla.org/r/136478/#review140786

Perfect :)
Attachment #8864800 - Flags: review?(s.kaspari) → review+
Pushed by alwu@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/5211eb9a9b95
part1 : change tab's media playing state when the tab is controlled by media control. r=sebastian
https://hg.mozilla.org/integration/autoland/rev/49e6be4c1e58
part2 : use enum for audio focus states. r=sebastian
https://hg.mozilla.org/integration/autoland/rev/8f3190d4e3e4
part3 : change audio focus state before notifying observers. r=sebastian
https://hg.mozilla.org/integration/autoland/rev/1d82eb12f453
part4 : add Fennec test for media control and audio focus. r=sebastian
https://hg.mozilla.org/integration/autoland/rev/9b6f4262f550
part5 : create new type test class for media playback. r=sebastian
https://hg.mozilla.org/integration/autoland/rev/4a9d90bd04af
part6 : list all test tasks in the ctor. r=sebastian
https://hg.mozilla.org/integration/autoland/rev/ac34f641df95
part7 : add check for notification's content. r=sebastian
https://hg.mozilla.org/integration/autoland/rev/99965090d3d1
part8 : add annotiation @RobocopTarget. r=sebastian
https://hg.mozilla.org/integration/autoland/rev/eb57c8ac9c33
part9 : wait tab event to make sure media started successfully. r=sebastian
https://hg.mozilla.org/integration/autoland/rev/664e9d26fe0a
part10 : check notification id. r=sebastian
https://hg.mozilla.org/integration/autoland/rev/facb44db5254
part11 : increase the waiting time for notification UI changed. r=sebastian
Hi, Sebastian,
From the pushlog on autoland [1], I found that the robocop test didn't run on all Android testing environment. Could we run it by default on more environments? or even on autophone?
Thanks!

[1] https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=facb44db5254319b24ae30b57063a5862a347299
Flags: needinfo?(s.kaspari)
I'm not sure what our options are here -> I'll redirect to gbrown. :)
Flags: needinfo?(s.kaspari) → needinfo?(gbrown)
All robocop tests normally run on 'Android 4.3 API15+ opt' and 'Android API15+ Gradle opt'. There are a couple of tests that also run on Autophone, at least on Android 4.4. 

Many robocop tests fail when run against Debug builds, so we don't normally run them there.

The [1] reference in comment 102 is for autoland, an integration branch. Not all test jobs are run against every push on integration branches, to save resources. That's why only rc3 was run in [1]...but the other robocop jobs are run on integration branches on some pushes (every once in a while).

It's easier to see the full set of robocop tests on central: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&filter-searchStr=robocop

Is that good enough?
Flags: needinfo?(gbrown)
Hi, Geoff,
Could we run the specific robocop tests which are robost enough to test on autophone?
I'm wondering whether we can use this ini [1] to indicate what test we want to run?
The reason is one of tests I added in this bug could only be tested on API 23 and above, that is about the status bar's notification test and the testing method is available on API 23.
Thanks!

[1] http://searchfox.org/mozilla-central/source/mobile/android/tests/browser/robocop/robocop_autophone.ini
Flags: needinfo?(gbrown)
That seems reasonable, as long as we have the resources. We have a limited number of devices available, so need to be careful about how long tests run.

:bc - Thoughts, concerns?
Flags: needinfo?(gbrown) → needinfo?(bob)
I've filed bug1364028 for enabling robocop test, we can discuss there.
robocop is available on try though it is not available on trychooser to prevent it from being used to DOS autophone. I haven't run robocop on autophone / try recently and I can't find the runs with out having to page through treeherder which I'm loath to do. If I remember correctly robocop was:

1. perma-orange with no one to care or fix any issues.

2. ran in 4 chunks and ate a considerable amount of time.

I've submitted a try run at:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=1073d21bb32904655a328339966dd498e60b2dc7&group_state=expanded

If we want to use the normal robocop manifest, we will need to make it green on whichever devices we want to run it on. Or we will need to create an autophone specific manifest in the way we did for Flash with rca.

We will also want to trim the number of tests so that they run in a reasonable amount of time depending on which branches you want to run it on.
Flags: needinfo?(bob)
PS. With the exception of the new geckoview_example perf tests, we won't be running tests on android-api-15-gradle.
Hi, Bob,
We should file another bug to handle those robocop fail, and ask frond-end folks put more attention about that. Now I don't want to enable *all* test, I just want to run my test on Autophone. (Reason is on the comment105)

I would also take responsibility to fix the intermittent fail if my test is not robust enough (I've already test them lots of time on the try-server). So could you help me to create a specific robocop manifest to run my tests? We can do that in bug1364028.

I don't have strong wish to put the test on all branch, we can enable it only on m-c.

Thanks!
Flags: needinfo?(bob)
I can handle the changes to autophone's manifests to get your test scheduled. You will need to create a manifest like that for robocop_autophone.ini: http://searchfox.org/mozilla-central/search?q=robocop_autophone.ini&case=true&path=

Once that is in the tree we can run it on try and see how it looks.
Flags: needinfo?(bob)
Depends on: 1364202
Hi, Bob,
I've add my tests in robocop_autophone.ini, does that mean my tests have already been run on the Autophone?
Thanks!
Flags: needinfo?(bob)
Yes, But I wouldn't do that since the robocop_autophone.ini file is restricted to where it runs due to issues with Flash on various devices and repos. I would remove your tests from robocop_autophone.in (an unfortunately named manifest) and create your own manifest that we can schedule independently of flash.
Flags: needinfo?(bob)
Depends on: 1395086
Product: Firefox for Android → Firefox for Android Graveyard
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: