Last Comment Bug 770894 - Add a testing infrastructure to test stuff related to Web Apps in mochitests
: Add a testing infrastructure to test stuff related to Web Apps in mochitests
Status: RESOLVED FIXED
:
Product: Core
Classification: Components
Component: General (show other bugs)
: Trunk
: All All
: -- normal (vote)
: mozilla17
Assigned To: Mounir Lamouri (:mounir)
:
Mentors:
: 768334 (view as bug list)
Depends on:
Blocks: 758258
  Show dependency treegraph
 
Reported: 2012-07-04 07:48 PDT by Mounir Lamouri (:mounir)
Modified: 2012-09-17 14:19 PDT (History)
13 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---
+
+


Attachments
Patch (3.75 KB, patch)
2012-07-05 07:03 PDT, Mounir Lamouri (:mounir)
no flags Details | Diff | Review
Patch (3.56 KB, patch)
2012-07-05 07:05 PDT, Mounir Lamouri (:mounir)
no flags Details | Diff | Review
Patch (4.93 KB, patch)
2012-07-15 19:33 PDT, Mounir Lamouri (:mounir)
jmaher: review+
Details | Diff | Review
Followup: Don't use "/" inside app names, and don't declare multiple apps with the same origin. (1.42 KB, patch)
2012-07-19 11:36 PDT, Justin Lebar (not reading bugmail)
mounir: review+
Details | Diff | Review

Description Mounir Lamouri (:mounir) 2012-07-04 07:48:58 PDT
The code behaving differently for Web Apps grows everyday but we still have no way to test it except manual testing with Gaia.
We should add tests in m-c for that. We will need to create fake webapps in the test profile.
Comment 1 Justin Lebar (not reading bugmail) 2012-07-04 07:53:27 PDT
It would be really nice if we could test from mochitest-plain.  I don't know if that's feasible.
Comment 2 Vivien Nicolas (:vingtetun) (:21) - (NOT reading bugmails, needinfo? please) 2012-07-04 08:27:08 PDT
At some points etienne and I have write some tests based on mochitests for Gaia, but they didn't gain much traction and they have been... sorry really... abandonned.
Comment 3 Mounir Lamouri (:mounir) 2012-07-05 01:58:35 PDT
I had something somewhat working yesterday. However, I'm not able to get Fullscreen working with that but for some reasons, nsGlobalWindow::mApp is never set (if gdb doesn't lie to me) even if the correct app is found. I will have a deeper look at that today.
Comment 4 Mounir Lamouri (:mounir) 2012-07-05 07:03:43 PDT
Created attachment 639329 [details] [diff] [review]
Patch

We can easily add new apps in the test profile.
Using an app in a test requires you to have the mozbrowser privileges (should we make that available by default for say app.example.org?) and set @mozapp to a valid manifest URL. For the moment, the content of the frame can even be in another origin, it will still be flagged as installed content. That might change though.
Comment 5 Mounir Lamouri (:mounir) 2012-07-05 07:05:21 PDT
Created attachment 639330 [details] [diff] [review]
Patch

I didn't meant to have two apps for the moment. That was only done for testing purposes (making sure the generated json is correct).
Comment 6 Kyle Huey [:khuey] (khuey@mozilla.com) 2012-07-05 10:41:58 PDT
Comment on attachment 639330 [details] [diff] [review]
Patch

Review of attachment 639330 [details] [diff] [review]:
-----------------------------------------------------------------

I'm not a test harness peer.
Comment 7 Justin Lebar (not reading bugmail) 2012-07-05 11:01:32 PDT
I'm going to need two apps for my tests.  I can add that as a follow-up patch if you'd like.
Comment 8 Mounir Lamouri (:mounir) 2012-07-15 19:33:30 PDT
Created attachment 642466 [details] [diff] [review]
Patch

I completely forgot that I had another patch with quite more applications (I'm using those in another patch in my patch queue).
Comment 9 Mounir Lamouri (:mounir) 2012-07-15 19:34:33 PDT
This should be a blocker because we need to test a lot of stuff based on apps. The security model is maybe the biggest one.
Comment 10 Jason Smith [:jsmith] 2012-07-15 22:14:07 PDT
Okay, so I'm confused a bit. A while back I know David Clarke ported over a bunch of mochitests for the mozapps API that should be in m-c right now. I know there's been some check-ins to it post the implementation as well (it's actively maintained). So what's different here? What are you trying to do that's different? And can the existing tests be reused? Why or Why Not?
Comment 11 Jason Smith [:jsmith] 2012-07-15 22:18:58 PDT
(In reply to Jason Smith [:jsmith] from comment #10)
> Okay, so I'm confused a bit. A while back I know David Clarke ported over a
> bunch of mochitests for the mozapps API that should be in m-c right now. I
> know there's been some check-ins to it post the implementation as well (it's
> actively maintained). So what's different here? What are you trying to do
> that's different? And can the existing tests be reused? Why or Why Not?

There's also work I've seen on the Marionette side of the world in regards to apps (I know I saw a perf test for launching an app somewhere in the gaia repo...). Can someone clarify the differences here and why they are needed?
Comment 12 Mounir Lamouri (:mounir) 2012-07-17 11:16:51 PDT
Comment on attachment 642466 [details] [diff] [review]
Patch

Review of attachment 642466 [details] [diff] [review]:
-----------------------------------------------------------------

Thank you for the fast review :)
Comment 13 cmtalbert 2012-07-18 16:51:59 PDT
(In reply to Jason Smith [:jsmith] from comment #10)
> Okay, so I'm confused a bit. A while back I know David Clarke ported over a
> bunch of mochitests for the mozapps API that should be in m-c right now. I
> know there's been some check-ins to it post the implementation as well (it's
> actively maintained). So what's different here? What are you trying to do
> that's different? And can the existing tests be reused? Why or Why Not?

I'll take a stab at answering this.
I don't know exactly what dclarke's mochitests do w.r.t. apps. The framework here being used inside mochitest came after that and allows mochitest to run the app runtime. This is needed because developers need a way to test their patches to the web app code in the short term. At the time we designed this hack atop mochitest (last February) we didn't know when Marionette would land,and the apps team thought they were landing sooner. So we pushed through. 

We have a more complete and far simpler Marionette based apps testing framework that is being developed as well. This will have more ability to verify attributes of an installed app that QA cares about and will be able to work with real end-user apps rather than fake apps like mochitest can use (of course the marionette based framework can also use these fake apps too). It is my hope that we move the web apps testing system to the marionette based framework (you shouldn't have to change any tests) and we remove these hacks from mochitest once we have the new system in place. But, that is how this came to be, and why we have two implementations here - one is the short term solution and one is the longer term solution.
Comment 14 :Felipe Gomes (needinfo me!) 2012-07-18 17:17:54 PDT
Note that we implemented two test suites: one which was the framework to test the webapp runtime.  But we also implemented mochitests for the mozApps API, see bug 741549

Not sure how they overlap here with what you're trying to test
Comment 15 Jason Smith [:jsmith] 2012-07-18 17:20:41 PDT
If I understand what Clint is saying, this implementation is intending to allow you to run mochitests within a web app on B2G, correct? If that's correct, then yeah, this is different the mozapps API mochitests and the desktop runtime mochitests.
Comment 16 Mounir Lamouri (:mounir) 2012-07-18 17:26:31 PDT
It's quite different: we want to be able to tests mozapps so having pre-installed apps in the profile. But it seems like tests in bug 741549 are breaking this :(
Comment 17 :Felipe Gomes (needinfo me!) 2012-07-18 17:39:45 PDT
how are those tests breaking this? can you explain more?
Comment 18 Mounir Lamouri (:mounir) 2012-07-18 17:43:33 PDT
Justin is having a look at that. He might know better.
Comment 19 Justin Lebar (not reading bugmail) 2012-07-19 09:49:19 PDT
Right now, it's failing to uninstall the apps Mounir pre-installs.  I'm not sure why it's failing, yet.

That the first step of this test is to uninstall all apps in the profile seems pretty bogus to me.  :-/
Comment 20 Justin Lebar (not reading bugmail) 2012-07-19 09:58:50 PDT
I'd ask on IRC, but it's down for me:

The webapps code identifies an app to uninstall by its origin (Webapps.jsm, uninstall function).  But looking at the apps defined in automation.py.in, we have multiple apps with the same origin.  So...that does not seem right, and is likely part of the problem.
Comment 21 Justin Lebar (not reading bugmail) 2012-07-19 11:36:21 PDT
Created attachment 643945 [details] [diff] [review]
Followup: Don't use "/" inside app names, and don't declare multiple apps with the same origin.
Comment 22 Mounir Lamouri (:mounir) 2012-07-19 21:54:18 PDT
Comment on attachment 643945 [details] [diff] [review]
Followup: Don't use "/" inside app names, and don't declare multiple apps with the same origin.

Review of attachment 643945 [details] [diff] [review]:
-----------------------------------------------------------------

Thanks Justin.
Comment 23 Mounir Lamouri (:mounir) 2012-07-19 23:38:34 PDT
*** Bug 768334 has been marked as a duplicate of this bug. ***
Comment 25 Ed Morley [:emorley] 2012-07-20 04:47:57 PDT
Followup:
https://hg.mozilla.org/integration/mozilla-inbound/rev/fe71c51ee0b9

Note You need to log in before you can comment on or make changes to this bug.