Closed Bug 989926 Opened 10 years ago Closed 10 years ago

Add a fake rild.js and libril.js to desktop build

Categories

(Firefox OS Graveyard :: General, defect)

Product:
Firefox OS Graveyard
Component:
General
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED WONTFIX

People

(Reporter: vingtetun, Unassigned)

References

Details

(Whiteboard: [dependency: marketplace-partners])

Attachments

(3 files, 13 obsolete files)

0002-Bug-989926-Integrate-a-fake-RIL-proxy-and-libril-in-.patch
44.08 KB, patch
Details | Diff | Splinter Review
0001-Bug-989926-Make-B2G-Desktop-able-to-build-with-B2G_R.patch
15.21 KB, patch
Details | Diff | Splinter Review
Screen Shot 2014-07-14 at 4.46.47 AM.png
860.48 KB, image/png
Details
Attached patch rild.desktop.wip.patch (obsolete) — — Splinter Review 
Today it's painful for Gaia developers develop features like the call screen and to write integration tests for it.

With the coming Mulet, which is the future replacement for the simulator distributed by Mozilla, we also would like to emulate the device as much as we can.

In order to help on those parts it would be helpful to have a fake rild daemon and a fake libril that use the rest of the Gecko telephony/icc/sms stack.

This POC changes the build system to let a desktop build compile with MOZ_B2G_RIL and add 2 files to b2g/, such a rild.js and libril.js. Those files are basically a local server that load a fake ril implemented in JS in order to let us fake calls.

Ideally libril.js could be implemented as an XPCOM so marionette would have access to it, and we can write a little bit of abstraction in order to control the fake RIL. But in the current shape it will let people working on the dialer write integration tests for the call screen, conference calls, and the call log.



Vicamo, I'm asking feedback? on this patch for 2 reasons.


First, I know there is a lot of work involved to create a fake RIL and I have been told that some would prefer to use the Android emulator for that, as there is already a fake RIL implementation maintained by the community.

Sadly, I'm not sure this is a viable solution on a daily basis for front-end developers working on top of Firefox or b2g-desktop, or for third party developers using the Simulator (r2d2b2g).

What do you think ?


Secondly, and more related to the WIP, I'm curious about the changes to the build system I made. As of today they are pretty dirty as I was only trying to be able to compile a desktop build with the MOZ_B2G_RIL flag.

It seems like a bunch of the code that lives under the dom/system/gonk folder is not that much gonk specific, and if we proceed with that libril.js idea, can be moved somewhere else. But I don't have good ideas about what the dom/system folder structure should be.
Attachment #8399317 - Flags: feedback?(vyang)
I hacked a bit on this one, and I've been able to add REQUEST_OPERATOR handling. I'm about to hack better SIM card detection also.
Attached patch rild-js.patch (obsolete) — — Splinter Review 
Please find attached an improvement of the patch:
 - triggeting SIM status changed event
 - implemeing OPERATOR
 - early hackish implementation of SIM_IO, for EF_ICCID
Attached patch rild-js.patch (obsolete) — — Splinter Review 
Now with more SIM card stuff :)
Attachment #8399567 - Attachment is obsolete: true
Vivien,

I have to say I'm quite complex on this topic.  I would like to see RIL DOM APIs becomes available on other platforms, but I don't want to have fake/emulated libril/rild things on platforms other than Gonk.

Kyle Machulis also told me once that he would like to have a fake Node.js based rild for automation test purpose, and it turns out it's possible to run Gaia tests on B2G emulator.  That's a better idea for me.  Really.  We, Mozilla, take great effort to improve AOSP emulator and enrich its features inclusive of NFC and Bluetooth.  We'll constantly work on it because we need something to act as a practical environment for our own test cases.

And you want an environment for easier, more convenient development, too.  That's all we want to do.  For all these proposals, re-implement rild, I care only one question: what's RIL on Foo platform like?  RIL is currently a privilege of Gonk.  It's not available on any other platform now.  The libril/rild thing you see is from AOSP.  It's of Mozilla Gonk.  It doesn't follow that all other systems should have the same arch.  But, yes, it will be easier to copy what we currently have.

I would also like to remind you that RIL is not the only thing that has these problems.  Someday you will probably also want to test Bluetooth, FM Radio, NFC.  So, what's the solution for Bluetooth?  Are we going to rewrite Bluetooth emulation for B2G-desktop?  What are the solutions for FM & NFC?  Are we going to have everything emulated on Mulet?  Or, let's turn on Bluetooth API on desktop since we have DBus based Bluetooth support?  Yes, then one can plug in his Bluetooth dungle, and his app sees Bluetooth becoming available.  He can test his app with all the functions provided by Bluetooth API.  As for NFC, can we connect a NFC card to Gecko via pcsc-lite?

So, the problem becomes: what does Mulet mean to us?  If that's a emulation based project, we can even skip libril/rild thing and have a control panel that gives you direct control to all RIL states you want.  You can have a checkbox for Airplane Mode.  You can have an input box and a submit button to simulate receiving SMS messages and a list box to show sent ones as well.  You don't really have to implement everything down to libril & rild.  Just create an folder dom/system/mulet and have everything inside simulate behaviours of Gonk RIL.

(In reply to Vivien Nicolas (:vingtetun) (:21) from comment #0)
> Created attachment 8399317 [details] [diff] [review]
> rild.desktop.wip.patch
> 
> Today it's painful for Gaia developers develop features like the call screen
> and to write integration tests for it.
> 
> With the coming Mulet, which is the future replacement for the simulator
> distributed by Mozilla, we also would like to emulate the device as much as
> we can.
> 
> In order to help on those parts it would be helpful to have a fake rild
> daemon and a fake libril that use the rest of the Gecko telephony/icc/sms
> stack.
> 
> This POC changes the build system to let a desktop build compile with
> MOZ_B2G_RIL and add 2 files to b2g/, such a rild.js and libril.js. Those
> files are basically a local server that load a fake ril implemented in JS in
> order to let us fake calls.
> 
> Ideally libril.js could be implemented as an XPCOM so marionette would have
> access to it, and we can write a little bit of abstraction in order to
> control the fake RIL. But in the current shape it will let people working on
> the dialer write integration tests for the call screen, conference calls,
> and the call log.
> 
> Vicamo, I'm asking feedback? on this patch for 2 reasons.
> 
> First, I know there is a lot of work involved to create a fake RIL and I
> have been told that some would prefer to use the Android emulator for that,
> as there is already a fake RIL implementation maintained by the community.
>
> Sadly, I'm not sure this is a viable solution on a daily basis for front-end
> developers working on top of Firefox or b2g-desktop, or for third party
> developers using the Simulator (r2d2b2g).
> 
> What do you think ?
> 
> 
> Secondly, and more related to the WIP, I'm curious about the changes to the
> build system I made. As of today they are pretty dirty as I was only trying
> to be able to compile a desktop build with the MOZ_B2G_RIL flag.

The biggest change about MOZ_B2G_RIL and related build system lines is in bug 920551.  In that bug, the original intention is to remove RIL completely from Flatfish -- B2G tablet.  So we have only RIL available on Gonk && Phone.  In bug 947116, we should remove MOZ_B2G_RIL flag.  This means: 1) for platforms other than Gonk, unless it has RIL backend implemented and detected at runtime, RIL is still disabled, 2) for Flatfish, RIL backend is implemented but can't be detected at runtime, so RIL is still disabled.

> It seems like a bunch of the code that lives under the dom/system/gonk
> folder is not that much gonk specific, and if we proceed with that libril.js
> idea, can be moved somewhere else. But I don't have good ideas about what
> the dom/system folder structure should be.

Everything under dom/system/gonk _is_ Gonk specific.  There is no exception.  Yes, RIL is Gonk specific.  We have not yet port RIL to platforms other than Gonk.
Comment on attachment 8399317 [details] [diff] [review]
rild.desktop.wip.patch

Review of attachment 8399317 [details] [diff] [review]:
-----------------------------------------------------------------

::: configure.in
@@ -7305,5 @@
> -    if test -n "$_PLATFORM_HAVE_RIL"; then
> -        AC_DEFINE(MOZ_B2G_RIL)
> -    else
> -        AC_MSG_ERROR([b2g-ril cannot be enabled because target platform doesn't support it.])
> -    fi

Flatfish has no RIL, so this is not acceptable.

::: dom/system/moz.build
@@ +13,5 @@
>  elif toolkit == 'cocoa':
>      DIRS += ['mac']
>  elif toolkit == 'android':
>      DIRS += ['android']
> +DIRS += ['gonk']

Certainly not acceptable, either.
Attachment #8399317 - Flags: feedback?(vyang) → feedback-
Attached patch rild-js.patch (obsolete) — — Splinter Review 
Randomized signal strength
Attachment #8399986 - Attachment is obsolete: true
Adding a couple of stuff.
Attachment #8400000 - Attachment is obsolete: true

        Attachment #8400483 -
        Attachment is obsolete: true

    
    

    
  


  

        Attachment #8400601 -
        Attachment is obsolete: true

    
    

    
  


  

        Attachment #8400914 -
        Attachment is obsolete: true

    
    

    
  


  

        Attachment #8401177 -
        Attachment is obsolete: true

    
    

    
  
Comment on attachment 8401182 [details] [diff] [review]
0001-Hacking-a-rild-and-libril-in-JS.patch

Review of attachment 8401182 [details] [diff] [review]:
-----------------------------------------------------------------

::: dom/system/gonk/SystemWorkerManager.cpp
@@ +22,2 @@
>  #include "nsIWifi.h"
> +#endif

I'd really like to make myself clear again.  I DO NOT want this to happen.  All the lines added in this patch that tries to make dom/system/gonk bits also compiled on platforms other than Gonk are _NOT_ acceptable to me.  You should either share necessary parts by moving them into some shared folder, or create dom/system/mulet and place whatever you need inside.  Thank you.

    
  
Blocks: 943528

    
    

    
  
(In reply to Vicamo Yang [:vicamo][:vyang] from comment #12)
> Comment on attachment 8401182 [details] [diff] [review]
> 0001-Hacking-a-rild-and-libril-in-JS.patch
> 
> Review of attachment 8401182 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> ::: dom/system/gonk/SystemWorkerManager.cpp
> @@ +22,2 @@
> >  #include "nsIWifi.h"
> > +#endif
> 
> I'd really like to make myself clear again.  I DO NOT want this to happen. 
> All the lines added in this patch that tries to make dom/system/gonk bits
> also compiled on platforms other than Gonk are _NOT_ acceptable to me.  You
> should either share necessary parts by moving them into some shared folder,
> or create dom/system/mulet and place whatever you need inside.  Thank you.

Vicamo,

please do not worry about the hack in the build system with the gonk folder. I didn't want to land it as if. It was more a hack in the build system to see if it is doable to use a js to fake ril.

When I asked you the feedback that was to understand exactly what we can move out of the gonk/ specific folder to move it to a shared folder and what we can not. So basically what I was asking is 'what could be a good folder structure in dom/system/' so the Mulet can use them.

I will do a bigger answer in a few, but please don't see the Proof-of-concept as a final patch.

    
    

    
  


  

        Attachment #8401182 -
        Attachment is obsolete: true

    
    

    
  
In the Developer Tools, we would like to be able to change the mcc/mnc values (see bug 943528). We could monkeypatch the MozMobile* APIs, but using libril would be much cleaner. And we could provide a telephony-tool in the App Manager.

    
    

    
  
We should definitely move forward with this approach.

The default build environment that most of our gaia developers are using is going to be Mulet. It's important that in that environment that we can use the same APIs as are available on devices.

Not only do we want to be able to use the same APIs, but we should also reuse as much as possible of the same implementation of those APIs. Writing a whole new implementation for Mulet will mean that the implementation on devices and the implementation in Mulet will have different bugs and behave differently in various edge cases. And they will get out of sync any time that we update the API running on devices.

I know that having a fake rild implemented in JS will have the same problem. I.e. the fake JS rild and the real C++ rild will behave differently in edge cases. However it'll be less different than if we implement the full separate DOM API.

I totally agree that one day we might want an implementation of the telephony API which runs on desktop. And that this API would likely use a totally different architecture and not use libril/rild. However that is likely a long way out, and we need our gaia developers to be productive until then.

But we should definitely only enable the fake rild in Mulet. Until we have Mulet running we can enable it on B2G desktop builds, but we should disable that as soon as Mulet ships. The fake rild should definitely not be included in stingray or flatfish builds.


I'm absolutely open to different approaches than the one proposed in this patch. However we need to know:

* How do we make the different approach work in desktop builds like Mulet?
* Who would be able to work on it?
* When can we expect it to be done?
* How much of the code would be different from the code that runs on devices?

Note that the intent here is not to create a working implementation of the RIL related APIs on desktop. It's to enable gaia developers to write code that works on devices.

    
    

    
  
(In reply to Jonas Sicking (:sicking) from comment #17)
> * How do we make the different approach work in desktop builds like Mulet?
> * Who would be able to work on it?
> * When can we expect it to be done?
> * How much of the code would be different from the code that runs on devices?

There are a few approaches I've thought about. I'll start listing them here, though it's probably going to be multiple comments.

Idea: Re-instate network socket per rilproxy process, connect directly to phone from desktop

Description:

This is what I had going way back when I first wrote the original rild code, and what I used to test it on desktop. Basically, you can have adb forward the /socket/rilproxy socket up out to a network port, and desktop, via a pref to flip between socket types to connect to, can just connect to that network port. That allows you to develop using the real radio on the cellphone, except with code running on desktop. This got yanked during the beginnings of DSDS since figuring out how to expose multiple ports that way was going to be an issue and I don't think anyone used it anyways.

The problems lie in the fact that you need a phone, and you need to have it on and be on the network. Technically we could also just start up an emulator and do this trick to the simulated ril in it, but that seems like a ton of work for not much payoff.

Q&A:

* How do we make the different approach work in desktop builds like Mulet? 

Just put the pref and network access back in. Pretty simple.

* Who would be able to work on it?

Anyone familiar with RIL interaction, unixsocket, etc. Me, tzimmermann, vyang, hsinyi, etc. 

* When can we expect it to be done?

Shouldn't take too long but I'm not aware of how difficult it would be to get it going with DSDS

* How much of the code would be different from the code that runs on devices?

None. This would just be hosting the phone socket on desktop, nothing more. Even with the other solutions I'll be posting about (writing a simulator in node.js, or in gecko), this would be viable because with this there /is no simulator/. So when you have a phone and want to use a real radio, use this. When you don't (like, say, in integration testing, which is what my original thoughts were in relation to), us this.

    
    

    
  
Idea: Implement ril simulator in node.js, make it part of gaia (tl;dr I don't think this is the right way to go anymore, but I'm putting it here for history sake)

Description:

This is the idea I pitched back in February. The gaia-centricness of it was because I really needed to test some ICC stuff in gaia, and mocks suck. Seemed like gaia was the place to put it because we could just make it a node.js module. I hadn't really thought about using it for development, more for packet replays for testing. We'd just put a chain of RIL parcels together, make a json out of them, replay that through the system. It'd still require the network stuff I mentioned in Comment #18, as the ril in gecko would need to be able to connect to the ril simulator somehow.

Q&A

* How do we make the different approach work in desktop builds like Mulet?

Detaches implementation from gecko. Not sure if that'd be a good idea or not.

* Who would be able to work on it?

That's where this starts to suck. If we're going to use node.js, it'd require node.js api experience. We have a ton of that, but I think the venn diagrams of "gecko/ril internals knowledge" and "node.js knowledge" has very, very few points of intersection.

* When can we expect it to be done?

The answer for this and the mulet embedded simulator or going to be the same: Maybe one sprint to get the communications up and running, and god knows how many sprints to get it working right. What we're planning on doing here is basically setting up a GIGANTIC state machine where we simulate the shared state we get from sim, the radio towers, the switches, everything that makes up what happens when you ship something out to the radio. That is not an amount of work to be underestimated.

> * How much of the code would be different from the code that runs on devices?

All the DOM stuff would be similar. That's where it ends though. To do this with node.js, we'd have to rewrite our parcel parser and what not.

    
    

    
  
Idea: Implement RIL simulator in gecko

Description:

Instead of having RIL SIM in gaia like Comment #19, put it in gecko. Not much to say otherwise.

Q&A

* How do we make the different approach work in desktop builds like Mulet?

Already discussed that

* Who would be able to work on it?

A combination of anyone that's worked on RIL, and anyone that'd written js inside of gecko. That's a much larger group than Comment #19 I think.

* When can we expect it to be done?

See timeline in comment #19

* How much of the code would be different from the code that runs on devices?

We'd be able to share the parcel parser, which is different than Comment #19. Other than that though, we're still rewriting the RIL. That's a big deal.

    
    

    
  
Idea: Use the Android Emulator

Description:

See Comment #4

Q&A, now mostly my opinions:

* How do we make the different approach work in desktop builds like Mulet?

It could work via the ideas in Comment #17, but it'd be painful. Bring up an emulator, detach rilproxy connection, bring up Mulet, have it attach via an adb forward. That's a lot of things to have running, especially for people having to already run in a VM for the emulator (which is how I understand that happens for some, but I may be wrong?)

* Who would be able to work on it?

Right now, anyone that can get the emulator. The reason I backed off of the Comment #19 approach was because I was told emulator builds would be publicly available soon. If that were the case, this might be slightly more feasible, but I'm not sure if we're distributing those yet. I heard it was mostly a legal issue.

In terms of /fixing and adding/ to the emulator side... I honestly have no idea who knows or works on that anymore, so I can't answer that. I'm guessing Vicamo knows.

* When can we expect it to be done?

It's already done. That's why there's push for it. It does just work. Slowly, and resource intensively, but it's there.

* How much of the code would be different from the code that runs on devices?

Same as Comment #19 or Comment #20. We're running all of the DOM code and what not, but the other side is still a simulator. Just written by the AOSP people, not us.

    
    

    
  
Working on a fake in-Gecko libril or on the emulator has the same drawbacks that we have to re-implement a whole RIL. Part of it is done in the emulator already, but there is a ton of work to complete and make it suitable for our needs. I think from this point of view there is no clearly differentiated solution.

I'm concerned for running the emulator: it's huge, potentially slow, and it will be painful to run inside a VM (nested virtulization in the best case).

    
    

    
  
(In reply to Alexandre LISSY :gerard-majax from comment #23)
> I'm concerned for running the emulator: it's huge, potentially slow, and it
> will be painful to run inside a VM (nested virtulization in the best case).

What about ripping out the modem emulation code from the emulator and make it runnable as a stand-alone daemon that would talk to the actual RIL code in b2g-desktop? I have worked briefly with it and it looks relatively self-contained. Having something like that would give us two advantages: we would re-use the existing RIL code and we would have the modem emulation commands available (to simulate SMS, calls, etc...).

IMHO the latter feature is quite important. I'm currently mentoring a GSoC which involves playing with the SMS app and w/o a device the only reasonable alternative for the contributor is to use the emulator and send fake SMS via the emulator telnet command console. We should have something similar for mulet too otherwise its usefulness will be limited.

    
    

    
  
(In reply to Jonas Sicking (:sicking) from comment #17)
> We should definitely move forward with this approach.
> 
> The default build environment that most of our gaia developers are using is
> going to be Mulet. It's important that in that environment that we can use
> the same APIs as are available on devices.

Mulet is a Firefox Desktop with b2g desktop bits.
So that there is no android emulator involved at all.

Also an important thing is that the mulet is going to replace b2g desktop,
and the special firefox nightly profile gaia build system craft with many hack..
... as well as the Firefox OS Simulator addon.

The simulator addon and the mulet would also benefit from mocking telephony API.
So if we can find ways to mock the API that doesn't depend on the android emulator,
nor any other external dependencies (nodejs), that would be really helpful!
Unless... we want to replace the Mulet and Simulator addon with an emulator package.

> But we should definitely only enable the fake rild in Mulet. Until we have
> Mulet running we can enable it on B2G desktop builds, but we should disable
> that as soon as Mulet ships. The fake rild should definitely not be included
> in stingray or flatfish builds.

There is absolutely no dependency on the Mulet.
We can start adding the helper to b2g desktop right away!

We don't have to wait for the mulet to improve tests running on b2g desktop (mochitests, gaia integration). Also it would allow us (the app team) to start working on ril scripts to make a fake working dialer and fake the mnc/mcc values paul mentioned right for the Firefox OS Simulator addon.
Blocks: 979861

    
    

    
  
Is there a way that we can simulate network conditions as part of this?  Eg. manually change the signal strength so that you can evaluate your app on a simulated poor network.

    
    

    
  
Could we change the number you get back that's calculated by the ril and sent over so we can display the correct number of bars at the top? Sure. 

But we're not actually shipping data, voice or otherwise, through this. This is /just/ for ICC/SIM status changes, events like calls, etc. So we can't degrade a stream, because we don't have them.

    
    

    
  
We definitely can't do only comment 18 since that solution doesn't work in automated testing. However we could possibly do comment 18 in addition to something else in order to enable testing against real hardware.


I'm not sure that I understand the proposal in comment 21 or comment 4.

Is the idea the same as in comment 18 except that rather than forwarding to a rild running on a hardware phone, we run an emulator and forward to a rild running on that emulator?

If so that might work as long as we make it really easy to get an emulator build that corresponds to the Mulet build.

Keep in mind that any time that we make changes to the ril interface it would break anyone that isn't using the right version of the emulator builds. So effectively we are doubling the size of the Mulet download since you'd have to download both the Mulet build and the emulator build at the same time to be sure that you have matching versions.

There's also the question how this solution affects people that work on the RIL. It'd be annoying if RIL developers would have to compile both a mulet build and an emulator build any time they wanted to test a change. Though I think RIL developers will generally only use emulator (or device) so this is probably not a problem?

We also would need to get the emulator to the state when scripts can automatically launch it. The fact that we are using emulator as part of "mocking" the telephony API needs to be transparent. So running tests locally should not need to require going through complex steps of manually launching the emulator. And during development, getting a working telephony API needs to be as simple as pushing a button or passing a command line argument when starting Mulet.

Is this possible?

    
    

    
  
(In reply to Jonas Sicking (:sicking) from comment #28)
> We definitely can't do only comment 18 since that solution doesn't work in
> automated testing. However we could possibly do comment 18 in addition to
> something else in order to enable testing against real hardware.

So there's really nothing saying "all of the above" couldn't be an option too, at least, technically. We get the emulator for free, basically. We already have the capabilities to run integration tests in the emulator (ask :aus or :gaye, that's still magic to me). So we have that if we want to try it anyways.

Reimplementing the network access shouldn't be that hard, though I could also be sticking my foot in my unknowledgable mouth there.

I think the pushback on having a RIL sim is the idea that if we have that we'll completely stop paying attention to the emulator, and a rift will form between RIL devs and frontend that causes way more regressions. I find it hard to say whether or not that's true.
 
> Is the idea the same as in comment 18 except that rather than forwarding to
> a rild running on a hardware phone, we run an emulator and forward to a rild
> running on that emulator?

Yes.

I still think the "access the emulator via mulet" option is FAR to heavyweight for gaia developers. That's why I pitched ril emulation in the first place.

One of the things I haven't seen here though is the fact that we need to be able to script ril emulation for tests. This isn't the same as mocking, just the ability to remove/insert sim, change networks, etc programmatically.

> If so that might work as long as we make it really easy to get an emulator
> build that corresponds to the Mulet build.
> 
> Keep in mind that any time that we make changes to the ril interface it
> would break anyone that isn't using the right version of the emulator
> builds. So effectively we are doubling the size of the Mulet download since
> you'd have to download both the Mulet build and the emulator build at the
> same time to be sure that you have matching versions.

As far as I'm aware, we'll never change anything that would break things in that way. The RIL in the emulator is AOSP managed, so it's going to work the same way anyone using android expects RIL to work.

> 
> There's also the question how this solution affects people that work on the
> RIL. It'd be annoying if RIL developers would have to compile both a mulet
> build and an emulator build any time they wanted to test a change. Though I
> think RIL developers will generally only use emulator (or device) so this is
> probably not a problem?

I have no idea.

> We also would need to get the emulator to the state when scripts can
> automatically launch it. The fact that we are using emulator as part of
> "mocking" the telephony API needs to be transparent. So running tests
> locally should not need to require going through complex steps of manually
> launching the emulator. And during development, getting a working telephony
> API needs to be as simple as pushing a button or passing a command line
> argument when starting Mulet.
> 
> Is this possible?

I /think/ we already have this with the integration tests running on the emulator, but I can't confirm it. I'm ni'ing the responsible people.
Flags: needinfo?(gaye)
Flags: needinfo?(aus)

    
    

    
  
(In reply to Kyle Machulis [:kmachulis] [:qdot] from comment #29)
> 
> One of the things I haven't seen here though is the fact that we need to be
> able to script ril emulation for tests. This isn't the same as mocking, just
> the ability to remove/insert sim, change networks, etc programmatically.
> 

Not sure it is obvious here, but we are in touch with the devtools people. And as soon we have something on the platform side they can use, they can help us to build an actor that would works for Gaia devs, the simulator and scripting for tests.

    
    

    
  
(In reply to Vivien Nicolas (:vingtetun) (:21) (NOT reading bugmails, needinfo? please) from comment #30)
> (In reply to Kyle Machulis [:kmachulis] [:qdot] from comment #29)
> > 
> > One of the things I haven't seen here though is the fact that we need to be
> > able to script ril emulation for tests. This isn't the same as mocking, just
> > the ability to remove/insert sim, change networks, etc programmatically.
> > 
> 
> Not sure it is obvious here, but we are in touch with the devtools people.
> And as soon we have something on the platform side they can use, they can
> help us to build an actor that would works for Gaia devs, the simulator and
> scripting for tests.

We are indeed eager to make the RIL emulation scriptable once it's available, in order to create new tools and to make RIL-tests possible in the simulator. I filed bug 1002463 to make this more obvious.
Blocks: 1002463

    
    

    
  
I've just pushed a rebased version of the hack to https://github.com/lissyx/mozilla-central/tree/libril-bug989926

This introduces two prefs:
 - one to enable/disable the fake rild/libriljs
 - one to define how many fake SIMs we have

This way I'm able to get DSDS on B2G Desktop.

    
    

    
  
From my understanding we already run some of the integration tests in the emulator on tbpl.
Flags: needinfo?(aus)

    
    

    
  
Kyle, reading the comments you put, my understanding is that most of the approach require to be able to compile a Gecko with the ril parcel code inside.

What Gecko connects too is a separate question. So does it makes sense to move forward and fix the build system so the gonk specific bits can be enabled in a desktop build ?
Flags: needinfo?(kyle)

    
    

    
  
Sure? Don't see what that'd hurt, since we can just pref off connecting to whatever we think rild is, versus our current method of ifdef'ing.
Flags: needinfo?(kyle)

    
    

    
  


  
Splitting patch: build system part.

        Attachment #8399317 -
        Attachment is obsolete: true

        Attachment #8402207 -
        Attachment is obsolete: true

    
    

    
  


  
Splitting patch: fake ril part.

    
    

    
  


  
Fixing undefined Cu.

        Attachment #8427679 -
        Attachment is obsolete: true

    
    

    
  
(In reply to Alexandre LISSY :gerard-majax from comment #36)
> Created attachment 8427678 [details] [diff] [review]
> 0001-Bug-989926-Make-B2G-Desktop-able-to-build-with-B2G_R.patch
> 
> Splitting patch: build system part.

I wonder if a new MOZ_B2G_GONK_RIL flag should not be used instead of the MOZ_B2G_RIL flag ?

    
    

    
  
Wesley Huang came to me this morning, told me this topic had been raised again last week.  I had a discussion on IRC with Vivien and Lissy once, but maybe I had not made myself clear.  Here is my two cents.

Idea: have a Mulet specific implementation of nsITelephonyProvider/nsISmsService/...

Description:

RIL developers have created internal API interfaces for different systems and platforms.  These APIs are minimum set to be called by DOM interfaces.  By implementing a new set of those internal interfaces [1], a new RIL supported platform is then achieved.

* How do we make the different approach work in desktop builds like Mulet?

The B2G RIL automation works because we have following stack:

  DOM API <--> internal API <--> RadioInterfaceLayer & RIL worker <--> rild <--> B2G emulator

B2G Emulator serves as a automation command listener and event generator.  Mulet has to implement the same role somewhere I don't know now.  It could be something similar to Marionette server.  Let's call it MuletInstrumentService here for further reference.  It talks to the nsITelephonyProvider implementing service, MuletTelephonyService for example, as well as other RIL internal interfaces implementing services.  MuletInstrumentService generates events by request, and those events are delivered to corresponding components.

So, the things to do for implementing Telephony API on Mulet are:

  1) create dom/telephony/mulet/TelephonyProvider.js, which implements nsITelephonyProvider,
  2) add ability to adopt Mulet TelephonyService in dom/telephony/TelephonyFactory.cpp,
  3) create dom/system/mulet/MuletInstrumentService.js and mechanism for automation instruments.

No emulator, parcel encoding/decoding, rild, etc., is involved.  All emulation bits are landed to Gecko repository because they're now a backend of all RIL functionalities.

* How much of the code would be different from the code that runs on devices?

Mostly.  Only DOM APIs, OOP functions are reused.

However, we can also alter the level of emulation a little bit lower to nsIRadioInterfaceLayer, which is also the interface we share with QCRIL.  In this way, we preserve most lines of current B2G RIL implementations but the RadioInterfaceLayer itself.  TODOs:

  1) create dom/system/mulet/MuletInstrumentService.js, which implements nsIRadioInterfaceLayer and mechanism for automation instruments,
  2) modify some 'moz.build' files to share some B2G RIL components.

    
    

    
  
(In reply to Vicamo Yang [:vicamo][:vyang] (PTO May 17 - Jun 1)) from comment #40)
> However, we can also alter the level of emulation a little bit lower to
> nsIRadioInterfaceLayer.

I'd like to avoid this as possible because the lower level of emulation is chosen, the more dependent to hardware/platform we get.  It may be a good thing for LOC reused, but the binding can be just too strong that we have to introduce yet another level of abstraction to resolve the problem.

For example, TelephonyService we have for B2G talks to AudioManager, which is certainly not available on desktop and we have a dummy implementation etched in source instead.  Its behavior may not match the one we want for desktop and may cause problems in the future.

So applies to RadioInterfaceLayer.  It's highly bound to NetworkManager, which may alter IP route settings on receiving some events.  NetworkManager depends on AOSP netd as well.  This follows it won't have correct behaviors without all related components being ported to Mulet.

    
    

    
  
Until we hack it in a more proper way as described by Vicamo in comment 40, here are updated patches that once again have been quite useful.

    
    

    
  
(In reply to Vicamo Yang [:vicamo][:vyang] from comment #40)
> Wesley Huang came to me this morning, told me this topic had been raised
> again last week.  I had a discussion on IRC with Vivien and Lissy once, but
> maybe I had not made myself clear.  Here is my two cents.
> 
> Idea: have a Mulet specific implementation of
> nsITelephonyProvider/nsISmsService/...
> 
> Description:
> 
> RIL developers have created internal API interfaces for different systems
> and platforms.  These APIs are minimum set to be called by DOM interfaces. 
> By implementing a new set of those internal interfaces [1], a new RIL
> supported platform is then achieved.
> 
> * How do we make the different approach work in desktop builds like Mulet?
> 
> The B2G RIL automation works because we have following stack:
> 
>   DOM API <--> internal API <--> RadioInterfaceLayer & RIL worker <--> rild
> <--> B2G emulator
> 
> B2G Emulator serves as a automation command listener and event generator. 
> Mulet has to implement the same role somewhere I don't know now.  It could
> be something similar to Marionette server.  Let's call it
> MuletInstrumentService here for further reference.  It talks to the
> nsITelephonyProvider implementing service, MuletTelephonyService for
> example, as well as other RIL internal interfaces implementing services. 
> MuletInstrumentService generates events by request, and those events are
> delivered to corresponding components.
> 
> So, the things to do for implementing Telephony API on Mulet are:
> 
>   1) create dom/telephony/mulet/TelephonyProvider.js, which implements
> nsITelephonyProvider,
>   2) add ability to adopt Mulet TelephonyService in
> dom/telephony/TelephonyFactory.cpp,
>   3) create dom/system/mulet/MuletInstrumentService.js and mechanism for
> automation instruments.
> 
> No emulator, parcel encoding/decoding, rild, etc., is involved.  All
> emulation bits are landed to Gecko repository because they're now a backend
> of all RIL functionalities.
> 
> * How much of the code would be different from the code that runs on devices?
> 
> Mostly.  Only DOM APIs, OOP functions are reused.
> 
> However, we can also alter the level of emulation a little bit lower to
> nsIRadioInterfaceLayer, which is also the interface we share with QCRIL.  In
> this way, we preserve most lines of current B2G RIL implementations but the
> RadioInterfaceLayer itself.  TODOs:
> 
>   1) create dom/system/mulet/MuletInstrumentService.js, which implements
> nsIRadioInterfaceLayer and mechanism for automation instruments,
>   2) modify some 'moz.build' files to share some B2G RIL components.

This all makes a lot of sense and obviously looks like the proper way to implement this from a top level feature point of view.

But I'm still sad that we won't be able to at least have a basic set of coverage for lower level stuff ; yet regarding the architecture I don't see a clean way to do this.

Meanwhile, the new attachment and my github branch provides uptodate hack, which works properly on B2G Desktop and Mulet. So until we have a clean way, it's still serving some purpose :)

    
    

    
  
(In reply to Alexandre LISSY :gerard-majax from comment #44)
> Until we hack it in a more proper way as described by Vicamo in comment 40,
> here are updated patches that once again have been quite useful.

https://github.com/vicamo/b2g_mozilla-central/tree/experimental/simulator

Here is a working branch for fxos simulator RIL support based on my comments. Nothing really works so far, but MOZ_B2G_RIL is enabled, DOM API attributes are available, Gaia boots to homescreen. To have a quick test on it:

  $ cd $B2G
  $ git clone https://git.mozilla.org/releases/gaia.git gaia
  $ ln -s b2g/config/mozconfigs/macosx64_gecko/nightly .mozconfig
  $ nice -n 20 ./mach build
  $ ./mach run

I'm still fighting with devtools Remote Debugging Protocol so that I may provide remote control to these emulated backend services.  I'll probably focus on only telephony and mobilemessage at beginning.

    
    

    
  
(In reply to Vicamo Yang [:vicamo][:vyang] from comment #46)
> I'm still fighting with devtools Remote Debugging Protocol so that I may
> provide remote control to these emulated backend services.  I'll probably
> focus on only telephony and mobilemessage at beginning.

You may now dial/accept/reject calls from either sides (simulator & app-manager). Conference and CDMA are not supported yet. DevTool RDP events still not working for me.

    
    

    
  


  
RDP event is working now. The main logic is done. When you dial/accept/hangup from either side, call states are automatically updated in the Voice Calls panel. Need some CSS polishes for sure.

With DevTools RDP, one may continue to work on automation for Mulet as well as user interface on App Manager and WebIDE. Yes, one can also add RDP support to fake rild method. Any comments?
Flags: needinfo?(lissyx+mozillians)
Flags: needinfo?(21)

    
    

    
  
Very very nice :). It provides me with tons of happiness :)
Flags: needinfo?(lissyx+mozillians)

    
    

    
  
(In reply to Alexandre LISSY :gerard-majax from comment #49)
> Very very nice :). It provides me with tons of happiness :)

Are we in December already? This feels like Christmas :)
See Also:  → 1038606

    
    

    
  
(In reply to Vicamo Yang [:vicamo][:vyang] from comment #48)
> With DevTools RDP, one may continue to work on automation for Mulet as well
> as user interface on App Manager and WebIDE. Yes, one can also add RDP
> support to fake rild method. Any comments?

If this needs to be integrated in WebIDE (I think it should), please file a bug under Firefox>DeveloperTools:WebIDE, and describe there what needs to be done. I'll need some help to understand what the UI should look like.

    
    

    
  
(In reply to Paul Rouget [:paul] (slow to respond. Ping me on IRC) from comment #51)
> (In reply to Vicamo Yang [:vicamo][:vyang] from comment #48)
> > With DevTools RDP, one may continue to work on automation for Mulet as well
> > as user interface on App Manager and WebIDE. Yes, one can also add RDP
> > support to fake rild method. Any comments?

I had filed bug 1038606 for implementing a special nsITelephonyService for FxOS simulator and will leave this bug open for further opinions.

> If this needs to be integrated in WebIDE (I think it should), please file a
> bug under Firefox>DeveloperTools:WebIDE, and describe there what needs to be
> done. I'll need some help to understand what the UI should look like.

Actually I'm still pretty confused about FxOS simulator(b2g-desktop) v.s. Mulet, and AppManager v.s. WebIDE.  I choose simulator because I don't know what to do after a Mulet build; and WebIDE doesn't look like to include a control panel like AppManager.  It seems WebIDE will only provide static informations for a running device/simulator. Since both WebIDE and AppManager utilize devtools RDP, I just choose an easy one in order to provide a basic prototype as soon as possible.

    
    

    
  
(In reply to Vicamo Yang [:vicamo][:vyang] from comment #52)
> > If this needs to be integrated in WebIDE (I think it should), please file a
> > bug under Firefox>DeveloperTools:WebIDE, and describe there what needs to be
> > done. I'll need some help to understand what the UI should look like.
> 
> Actually I'm still pretty confused about FxOS simulator(b2g-desktop) v.s.
> Mulet, and AppManager v.s. WebIDE.

AppManager will disappear very soon. WebIDE is AppManager V2.

> I choose simulator because I don't know
> what to do after a Mulet build; and WebIDE doesn't look like to include a
> control panel like AppManager.  It seems WebIDE will only provide static
> informations for a running device/simulator. Since both WebIDE and
> AppManager utilize devtools RDP, I just choose an easy one in order to
> provide a basic prototype as soon as possible.

WebIDE has all the features App Manager has. Only the UI is different. We can easily add a "Telephony" menu next to "Permissions Table" menu, where you could control the RIL.

    
    

    
  
(In reply to Paul Rouget [:paul] (slow to respond. Ping me on IRC) from comment #53)
> WebIDE has all the features App Manager has. Only the UI is different. We
> can easily add a "Telephony" menu next to "Permissions Table" menu, where
> you could control the RIL.

It's not that easy because in AppManager I see a MVC model, all I need is to modify a certain data store and UI will be updated automatically.  But I can't find similar bits in WebIDE.
Flags: needinfo?(21)
Whiteboard: [dependency: marketplace-partners]
Flags: needinfo?(gaye)

    
    

    
  
Dead end. See bug 1038606 for the future.
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → WONTFIX

          You need to log in
          before you can comment on or make changes to this bug.
        






  

    
  







  

    

      
Attachment

      

      
      
      
      
    

    

      

        

          

            
General

            

              Creator: 



            

            
Created: 

            
Updated: 

            
Size: