947010 - Create automation test for recording status

Status: RESOLVED FIXED

(Firefox OS Graveyard :: General, defect)

Description

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

Need some test case for testing mozChromeEvent change for recording status, especially for any change made in shell.js. We might need to redirect mozChromeEvent event from b2g process to content process via specialpower API so that we verify if mozChromeEvent is generated as expected.

Comment 1

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

@ahal, I'm looking for a test framework that can execute test case in content process and verify the generated CustomEvent, i.e. mozChromeEvent in chrome process. Some one point me to use the SpecialPowers.loadChromeScript(), however, the chrome script is executed in a sandbox that cannot capture the CustomEvent. Can you give me some suggestion on which test framework I should use, or the way I can extend SpecialPower for this kind of test requirement?

Comment 2

[Andrew Halberstadt [:ahal]]

I'm not super familiar with specialpowers, but could you use SpecialPowers.addChromeEventListener()?

https://developer.mozilla.org/en-US/docs/SpecialPowers#Event_Listener_APIs

Comment 3

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

Thanks @ahal, looks like SpecialPowers.addChromeEventListener() can only capture the event that generated in content process because the event listener is attached to TabChild. However I do find out marionette test API have addChromeEventListener/sendContentEvent that exactly matched with our requirement. I'll use js marionette to write the test case.

Comment 4

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

Just put a reference to existing marionette API. It's only used in updater testcase right now.
http://dxr.mozilla.org/mozilla-central/source/testing/marionette/client/marionette/atoms/b2g_update_test.js#60

Comment 5

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

Attachment: testcase.patch

cover following 4 scenarios:
1. create single media stream
2. create multiple media stream at the same time
3. create media stream in iframe (the same origin)
4. kill content process while recording active (simulate receiving ipc:content-shutdown)

Comment 6

[Jason Smith [:jsmith]]

Comment on attachment 8355158 [details] [diff] [review]
testcase.patch

I'm not the best person to review a chrome-driven mochitest, so I'm going to redirect this. Fabrice probably could help review this though.

Comment 7

[Jason Smith [:jsmith]]

You probably also should consider kicking off a try run with these mochitests btw.

Comment 8

[[:fabrice] Fabrice Desré]

Comment on attachment 8355158 [details] [diff] [review]
testcase.patch

Review of attachment 8355158 [details] [diff] [review]:
-----------------------------------------------------------------

That looks good to me, thanks for working on that! As Jason said, please push to try first (let me know if you can't).

::: b2g/chrome/content/shell.js
@@ +1276,5 @@
>    //                               videoCount: <N>}}
>    let gRecordingActiveProcesses = {};
>  
>    let recordingHandler = function(aSubject, aTopic, aData) {
> +    dump('recording status: ' + aData);

nit: remove before landing.

::: b2g/test/mochitest/file_getusermedia_iframe.html
@@ +26,5 @@
> +    case 'stop':
> +      localStream.stop();
> +      localStream = null;
> +      break;
> +  } 

nit: trailing whitespace.

Comment 9

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

These testcase cannot run on b2g-desktop right now because b2g-desktop doesn't support OOP mochitest. All the mozChromeEvent will be pending because no mozbrowserloadstart event is triggered. I'll exclude these testcases from b2g-desktop.json for now.

Comment 10

[[:fabrice] Fabrice Desré]

(In reply to Shih-Chiang Chien [:schien] from comment #9)
> These testcase cannot run on b2g-desktop right now because b2g-desktop
> doesn't support OOP mochitest. All the mozChromeEvent will be pending
> because no mozbrowserloadstart event is triggered. I'll exclude these
> testcases from b2g-desktop.json for now.

Even without running OOP we should have mozbrowserloadstart triggered. Let's investigate that first.

Comment 11

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

I've filed Bug 957554 for investigating the mozbrowserloadstart issue.

Comment 12

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

Attachment: testcase.patch, v2

Update according to comment #8 and comment #9. Here is the try result for emulator mochitest: https://tbpl.mozilla.org/?tree=Try&rev=e2cb17272c1d.

Comment 13

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

Attachment: workaround-mozbrowserloadstart.patch

mochitest-b2g-desktop will success after applying this workaround. Here is the try result: https://tbpl.mozilla.org/?tree=Try&rev=c71df4fc7d08.

We may land testcase.patch first, then re-enable these testcases in bug 957554.

Comment 14

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

Based on my investigation in bug 957554 comment #2, I'm quite confident to say that we should re-enable these testcases on b2g-desktop after OOP is enabled on it.

Comment 15

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

Full try in https://tbpl.mozilla.org/?tree=Try&rev=a1d66b080172

Comment 16

[Andrew Halberstadt [:ahal]]

(In reply to Shih-Chiang Chien [:schien] from comment #15)
> Full try in https://tbpl.mozilla.org/?tree=Try&rev=a1d66b080172

Looks like you forgot to push your changes in that last try run.

Comment 17

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

(In reply to Andrew Halberstadt [:ahal] from comment #16)
> (In reply to Shih-Chiang Chien [:schien] from comment #15)
> > Full try in https://tbpl.mozilla.org/?tree=Try&rev=a1d66b080172
> 
> Looks like you forgot to push your changes in that last try run.
No, I don't. My changeset was pushed to TBPL in another try run in comment #12 and doesn't got changed since than. So, the TBPL only displayed the changeset I made on top of it, which is the try command. I've confirmed that my change is included in this try run from the revision log https://hg.mozilla.org/try/rev/a1d66b080172. The parent node is pointed to 91f38ef4e8fa, which is the testcase.patch.

Comment 18

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

Attachment: testcases.patch v2.1

Rebase to latest m-c and add the follow-up bug id in b2g-desktop.json. Per discussion with @fabrice today, we'll enable the test case on emulator first and enable it on b2g-desktop after bug 957554 is fixed.

try result: https://tbpl.mozilla.org/?tree=Try&rev=6fa9da1d004a

Comment 19

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

Bug 957554 is now a follow-up bug so remove it from dependent bug list.

Comment 20

[Ryan VanderMeulen [:RyanVM]]

b2g/test/mochitest seems like kind of an odd place for these tests?

Comment 21

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

(In reply to Ryan VanderMeulen [:RyanVM UTC-5] from comment #20)
> b2g/test/mochitest seems like kind of an odd place for these tests?

These test cases are for b2g/chrome/content/shell.js. How about moving these to b2g/chrome/content/test/mochitest? Do you think it is a suitable place for these tests?

Comment 22

[Ryan VanderMeulen [:RyanVM]]

SGTM if Fabrice is OK with it.

Comment 23

[[:fabrice] Fabrice Desré]

I am.

Comment 24

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

Attachment: testcases.patch v3

Move all the test cases to b2g/chrome/content/test/mochitest, carry r+. Will mark checkin-needed after try shows green.

try: https://tbpl.mozilla.org/?tree=Try&rev=091eb0a77157

Comment 25

[Shih-Chiang Chien [:schien] (UTC+8) (use ni? plz)]

Mark "checkin-needed" because this patch works well on my local environment and try is busted for all B2G Emulator builds currently (bug 963148)

Comment 26

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/integration/b2g-inbound/rev/47d35d5bc358

Comment 27

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/47d35d5bc358