1163928 - [MCTS] Unexpected result for: navigator.mozTelephony.calls

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: Certification Suite, defect)

Description

[oo[oouyang]]

Attachment: firefox-os-certification_20150512093832.zip

Comment 1

[Josh Cheng [:josh]]

Hi Wesley,
This bug failed MCTS test per bug 1142426.
Could you help to find someone who can help to check why it fails?
Thanks!

Comment 2

[Wesley Huang [:wesley_huang] (EPM) (NI me)]

calling out to Telephony platform

Comment 3

[Hsin-Yi Tsai (she/her) [:hsinyi]]

There's too little context for us to start helping out. Can you please provide more context about the test first?

Comment 4

[oo[oouyang]]

In the test reported, it said that the cert:webapi in navigator.mozTelephony.calls get unexpected result.
In cert:webapi test, we compare the device with the template result one by one.
After the compare the result, we found that navigator.mozTelephony.calls is different from the template.

You can get the latest(master) mcts from https://github.com/mozilla-b2g/fxos-certsuite
Follow the instructions from http://fxos-certsuite.readthedocs.org/en/latest/
Create your own virtualenv and run the following command to reproduce this issue.

runcertsuite cert:webapi

We expected there should be no change after FC, so we created the template based on FC images.
We wonder why there is such difference in webapi.

Comment 5

[oo[oouyang]]

Attachment: diff.log

This is the diff result for your reference.

The FC build ID is 20150503162504
The latest build ID is 20150511162500

Comment 6

[Hsin-Yi Tsai (she/her) [:hsinyi]]

(In reply to oo[oouyang] from comment #5)
> Created attachment 8605088 [details]
> diff.log
> 
> This is the diff result for your reference.
> 
> The FC build ID is 20150503162504
> The latest build ID is 20150511162500

Hi Owen,
As the range is quite big and I couldn't think of any obvious changes made in RIL resulting those differences. Is it possible that you kindly do bisect to get a narrower range to see which commit causes this change? Thank you again! 

Hi Aknow,
Would you please help this, too?

Comment 7

[oo[oouyang]]

As in current stage and discussed with Josh, it waste time to have resource to find a issue that we know there is some changes that cause this the problem. 
I will mark this case as wontfix.

Comment 8

[Szu-Yu Chen [:aknow] (inactive)]

I ran the MCTS test on FC build (20150503162504) and get the exactly same error.

  Unexpected result for: navigator.mozTelephony.calls (expected object)

So, there is no difference between the result of these two build.

Also, if we'd like to dump all the properties of navigator.mozTelephony.calls, the new result seems more reasonable to me.  navigator.mozTelephony.calls is an Array. That means you should see the properties from Array, just as what you listed in the attachment 8605088 [details].

Anyway, I haven't find the root cause of this issue, but I suspect that the prepared expected_result is not correct at the beginning.