Closed Bug 970615 (b2g-cert-tests) Opened 7 years ago Closed 5 years ago

[meta] Create B2G Certification Tests

Categories

(Firefox OS Graveyard :: Certification Suite, defect)

defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: jgriffin, Assigned: wxiong)

References

Details

Attachments

(4 files)

This is a tracking bug for the creation of a set of tests that can be used by OEM's to test B2G certification requirements.
Depends on: 970620
Alias: b2g-cert-tests
Summary: (b2g-cert-tests) Create B2G Certification Tests → Create B2G Certification Tests
See Also: → 969142
Depends on: 972979
Patrick, our current README is at https://github.com/mozilla-b2g/fxos-certsuite; can you review?
Flags: needinfo?(phundal)
(In reply to Jonathan Griffin (:jgriffin) from comment #1)
> Patrick, our current README is at
> https://github.com/mozilla-b2g/fxos-certsuite; can you review?

Jonathan,

These look great... These look simple enough for me to follow. ;)

One question... Just so I understand, the end user turns on remote debugging on device, joins the same wifi network as the machine running the tests, and that's it?  You don't have to tell the test machine any information about the Firefox OS device?  (IP, device name, etc...)  Does it do some type of port scan over the wifi network?
Flags: needinfo?(phundal)
(In reply to Patrick Hundal from comment #2)
> (In reply to Jonathan Griffin (:jgriffin) from comment #1)
> > Patrick, our current README is at
> > https://github.com/mozilla-b2g/fxos-certsuite; can you review?
> 
> Jonathan,
> 
> These look great... These look simple enough for me to follow. ;)
> 
> One question... Just so I understand, the end user turns on remote debugging
> on device, joins the same wifi network as the machine running the tests, and
> that's it?  You don't have to tell the test machine any information about
> the Firefox OS device?  (IP, device name, etc...)  Does it do some type of
> port scan over the wifi network?

Once the tests start, they instruct the user to launch the browser app and navigate to a web page that's hosted by the test harness; e.g., something like http://192.168.1.1:8080/.
(In reply to Jonathan Griffin (:jgriffin) from comment #3)
> (In reply to Patrick Hundal from comment #2)
> > (In reply to Jonathan Griffin (:jgriffin) from comment #1)
> > > Patrick, our current README is at
> > > https://github.com/mozilla-b2g/fxos-certsuite; can you review?
> > 
> > Jonathan,
> > 
> > These look great... These look simple enough for me to follow. ;)
> > 
> > One question... Just so I understand, the end user turns on remote debugging
> > on device, joins the same wifi network as the machine running the tests, and
> > that's it?  You don't have to tell the test machine any information about
> > the Firefox OS device?  (IP, device name, etc...)  Does it do some type of
> > port scan over the wifi network?
> 
> Once the tests start, they instruct the user to launch the browser app and
> navigate to a web page that's hosted by the test harness; e.g., something
> like http://192.168.1.1:8080/.

Awesome. ;)
Depends on: 974694
Depends on: 975107
Depends on: 976354
Depends on: 976360
Depends on: 976706
Depends on: 981832
Attached file fxos-MCTS-1.3.0.2.zip
This is the Sprint 1 release of the fxos-certsuite tool.  It contains the same content as the MWC release, plus the following:

- omni_diff.py to produce diffs of omni_analyzer results
- a metaharness which is used to run the tests and which will be refined in future sprints
- a WebIDL verifier which verifies top-level WebIDL's in an unprivileged context; more will be added in future sprints
- a small set of WebAPI tests which are designed to be run manually, and invoked separately from the main harness; these will be hooked up to a harness in the next sprint
(In reply to Jonathan Griffin (:jgriffin) from comment #5)
> Created attachment 8391576 [details]
> fxos-MCTS-1.3.0.2.zip
> 
> This is the Sprint 1 release of the fxos-certsuite tool.  It contains the
> same content as the MWC release, plus the following:
> 
> - omni_diff.py to produce diffs of omni_analyzer results
> - a metaharness which is used to run the tests and which will be refined in
> future sprints
> - a WebIDL verifier which verifies top-level WebIDL's in an unprivileged
> context; more will be added in future sprints
> - a small set of WebAPI tests which are designed to be run manually, and
> invoked separately from the main harness; these will be hooked up to a
> harness in the next sprint

Bug 984547 created to promote Sprint 1 deliverable to live partner site.
No longer blocks: 984559
Depends on: 984559
Attached file fxos-MCTS-1.3.0.3.zip
This is the Sprint 2 release of the fxos-certsuite tool.  It contains the same content as the Sprint 1 release, plus the following:

* The data for the WebIDL comparisons is now generated by the in-tree WebIDL.py.
* The WebIDL comparisons now use a static reference to compare against; this eliminates false failures due to permissions issues or cases where our implementation of a WebIDL doesn't match the expectations set by testharness.js.
* The window and navigator name checkers are now recursive, and the navigator name checker checks for objects which are null on a reference version but not-null on the device, and vice-versa.  Failures of these null/not-null checks can be indicative of permissions problems on the device.
* The meta-harness now installs Marionette as an extension on the target device.
* A prototype of the new semi-automatic WebAPI test harness is included, along with a small set of tests; see RUNNING_WEBAPI_TESTS.rst in the archive for details.  This will be connected to the meta-harness in the next sprint.
* An automatic app generator/installer has been written and is used by the semi-automatic WebAPI harness, and will be used by the WebIDL tests in the next sprint.
* A document which describes how to interpret the results is included, see INTERPRETING_RESULTS.rst.
fxos-MCTS-1.3.0.3.zip posted on partner site.
Depends on: 995455
Depends on: 995466
Attached file fxos-MCTS-1.3.0.4.zip
This is the Sprint 3 release of the fxos-certsuite tool.  It contains the same content as the Sprint 2 release, plus the following:

* The WebIDL tests and window and navigator object tests are now run three times, once in each of the following contexts:  unprivileged, privileged, and certified.
* New semi-auto WebAPI tests have been added for the following API's:  telephony, vibration, proximity, orientation, tcp_socket.  The existing SMS tests have been improved.
* Integration points have been added to the meta-harness which will allow it to be connected to the semi-auto harness and to the web-platform-tests; the actual connections will likely be implemented in Sprint 4.
* The existing suite has been broken into a set of subtests, and the meta-harness can now list and execute these individually.
* Several bugs in the semi-auto harness have been fixed.
Version fxos-MCTS-1.3.0.4.zip posted on Mobile Partner Site.
Depends on: 996315
Depends on: 996317
Depends on: 996318
Depends on: 996320
Depends on: 996321
Depends on: 996325
Depends on: 996712
Depends on: 997241
Depends on: 997245
Depends on: 997248
Depends on: 999091
Depends on: 1000271
Depends on: 1000433
Depends on: 1001065
This is the 1.3.0.5 release of the fxos-certsuite tool.  It contains the same content as the 1.3.0.4 release, plus the following:

* New webapi tests have been added for fm_radio
* The semi-auto webapi tests now prevent the lockscreen from turning on and interrupting the tests
* A number of bug fixes have been made to the semi-auto test harness
* The meta-harness has been updated to better support multiple test suites

release url: http://people.mozilla.org/~jgriffin/fxos-MCTS-1.3.0.5.zip

(I'm no longer attaching the zip files to this bug, since they exceed the max size allowed by bugzilla.)
CTS V1.3.0.5 posted to the partner website.
Depends on: 1006025
Depends on: 1007772
Depends on: 1008198
Depends on: 1008970
No longer depends on: 997245, 1000271, 1000433, 1007772, 1008198
Created a tracking bug for the WebAPI test development (Bug 1008970)
Here's the 1.3.0.6. I didn't get it ready Friday because we hit a device-specific crash with the getUserMedia API and it took some time to narrow it down:
http://people.mozilla.org/~tmielczarek/fxos-MCTS-1.3.0.6.zip

New since the last release:
* We now run WebAPI tests in unprivileged, privileged, and certified apps to test app permissions
* Device state is backed-up before running tests and restored afterwards
(In reply to Ted Mielczarek [:ted.mielczarek] from comment #14)
> Here's the 1.3.0.6. I didn't get it ready Friday because we hit a
> device-specific crash with the getUserMedia API and it took some time to
> narrow it down:
> http://people.mozilla.org/~tmielczarek/fxos-MCTS-1.3.0.6.zip
> 
> New since the last release:
> * We now run WebAPI tests in unprivileged, privileged, and certified apps to
> test app permissions
> * Device state is backed-up before running tests and restored afterwards

V1.3.0.6 has been uploaded to the Mobile Partner Website.
Depends on: 1012857
Depends on: 1012859
Depends on: 1012862
Depends on: 1012867
Depends on: 1012872
Depends on: 1012878
Depends on: 1015307
This is the 1.3.0.7 release of the fxos-certsuite tool.  It contains the same content as the previous release, plus the following:

* WebAPI verifier tests are completely automated now, no manual interaction required
* New online docs at http://fxos-certsuite.readthedocs.org
* The suite of semi-automatic WebAPI tests can now be invoked using webapi_tests/setup_and_run.sh
* Telephony tests no longer fail on Tarako devices
* Result files are now timestamped

release url: http://people.mozilla.org/~jgriffin/fxos-MCTS-1.3.0.7.zip
Component: General → Certification Suite
Depends on: 1021756
(In reply to Jonathan Griffin (:jgriffin) from comment #16)
> This is the 1.3.0.7 release of the fxos-certsuite tool.  It contains the
> same content as the previous release, plus the following:
> 
> * WebAPI verifier tests are completely automated now, no manual interaction
> required
> * New online docs at http://fxos-certsuite.readthedocs.org
> * The suite of semi-automatic WebAPI tests can now be invoked using
> webapi_tests/setup_and_run.sh
> * Telephony tests no longer fail on Tarako devices
> * Result files are now timestamped
> 
> release url: http://people.mozilla.org/~jgriffin/fxos-MCTS-1.3.0.7.zip

MCTS-1.3.0.7 uploaded today to https://mobilepartners.mozilla.org/market/certify-your-ffos-device/
Depends on: 1022883
This is the 1.3.0.8 release of the fxos-certsuite tool.  In contains the same content
as the previous release, plus the following:

* The first set of the web-platform-tests
* A suite of permissions model tests
* New semi-automated WebAPI tests for geolocation and bluetooth

release url:  http://people.mozilla.org/~jgriffin/fxos-MCTS-1.3.0.8.zip
Request: Will you change the "Readme.md" file in the "Mozilla Certification Test Suite [MCTS]" to a .pdf or other format that is compatible with Mac's and non-PC platforms. (Reference Comment 5.3 in Bug 1010043)
Blocks: 1010043
Flags: needinfo?(jgriffin)
No longer blocks: 1010043
Blocks: 1010043
(In reply to Karen Ward [:kward] from comment #19)
> Request: Will you change the "Readme.md" file in the "Mozilla Certification
> Test Suite [MCTS]" to a .pdf or other format that is compatible with Mac's
> and non-PC platforms. (Reference Comment 5.3 in Bug 1010043)

I'll change this to a .txt file so it's point-and-click friendly.
(In reply to Jonathan Griffin (:jgriffin) from comment #20)
> (In reply to Karen Ward [:kward] from comment #19)
> > Request: Will you change the "Readme.md" file in the "Mozilla Certification
> > Test Suite [MCTS]" to a .pdf or other format that is compatible with Mac's
> > and non-PC platforms. (Reference Comment 5.3 in Bug 1010043)
> 
> I'll change this to a .txt file so it's point-and-click friendly.

I think probably having Sphinx generate the documentation in docs/ to PDF would be more appropriate.  IIRC it's something like `make latex` or `make pdf`.
I think the issue here is that the .md file (which is just a minimal pointer to the readthedocs site) doesn't open in a text editor by default if you're double-clicking on it from the Finder, etc.

But, since we're going to branch the certsuite repo, it means different versions of the docs will be needed, so it probably makes sense to include the docs themselves in PDF form, like you say.
Flags: needinfo?(jgriffin)
Depends on: 1024186
(In reply to Jonathan Griffin (:jgriffin) from comment #18)
> This is the 1.3.0.8 release of the fxos-certsuite tool.  In contains the
> same content
> as the previous release, plus the following:
> 
> * The first set of the web-platform-tests
> * A suite of permissions model tests
> * New semi-automated WebAPI tests for geolocation and bluetooth
> 
> release url:  http://people.mozilla.org/~jgriffin/fxos-MCTS-1.3.0.8.zip

Version 1.3.0.8 of the CTS posted to mobile partner site.
Hi.  Just got a request from Legal team (Geoff Piper) to also have the License file either in pdf or Notepad format in future versions. OK?
(In reply to Karen Ward [:kward] from comment #24)
> Hi.  Just got a request from Legal team (Geoff Piper) to also have the
> License file either in pdf or Notepad format in future versions. OK?

Sure thing!
Depends on: 1026816
Depends on: 1027699
Depends on: 1028938
Depends on: 1029093
Depends on: 1029272
Depends on: 1030238
This is the 1.3.0.9 release of the MCTS.  New features in this release:

* A new HTML report which contains the results of the web-platform-tests
* Documentation in PDF format is included within the suite
* The webapi tests are more tolerant of crashes and low-memory conditions
* The semi-auto tests are now run first, so that the tests can then finish unattended
* New semi-auto tests for several WebAPIs

Release url: http://people.mozilla.org/~jgriffin/fxos-MCTS-1.3.0.9.zip
Depends on: 1032136
Depends on: 1033070
Updated on the Mobile Partner site on July 2.  Do we anticipate any issues with downloading this file from the site given it's large size?
Flags: needinfo?(booboobenny+bugzilla)
Not from my perspective. Double checking with Josh.
Flags: needinfo?(booboobenny+bugzilla) → needinfo?(jmize)
Depends on: 1035556
Depends on: 1038418
Depends on: 1039042
Depends on: 1039060
The attached files are all well under the 100 MB limit currently set on the server.
Flags: needinfo?(jmize)
Depends on: 1039215
Depends on: 1039196
Depends on: 1039695
This is the 1.3.0.10 release of the MCTS.  New features in this release:

* An updated format for the HTML report that is more complete and usable
* We don't show web-platform-test failures if they're expected failures
* Better handling of wifi connection problems during rebooting while running web-platform-tests
* Many bug fixes

Release url: http://people.mozilla.org/~jgriffin/fxos-MCTS-1.3.0.10.zip
Depends on: 1040838
Depends on: 1044143
Depends on: 1036681
Depends on: 1044190
Depends on: 1044192
No longer depends on: 1028938
Version fxos-MCTS-1.3.0.10.zip posted to the Mobile Partners site.
Depends on: 1045728
Depends on: 1045818
Depends on: 1046127
Depends on: 1046266
Depends on: 1047004
Depends on: 1048597
This is the 1.3.0.11 release of the MCTS, for devices running Firefox OS v1.3.  New features in this release:

* An updated format for the HTML report
* Telephony tests disable themselves automatically if the harness can't disable the dialer agent
* Better timeout handling for permissions tests
* Several logging improvements, including full adb commands and outputs
* Python dependencies are now pinned to prevent future package releases from affecting the suite
* Android build properties with '=' are now handled correctly
* Many other bug fixes

Release url: http://people.mozilla.org/~jgriffin/fxos-MCTS-1.3.0.11.zip
Depends on: 1032119
(In reply to Jonathan Griffin (:jgriffin) from comment #32)
> This is the 1.3.0.11 release of the MCTS, for devices running Firefox OS
> v1.3.  New features in this release:
> 
> * An updated format for the HTML report
> * Telephony tests disable themselves automatically if the harness can't
> disable the dialer agent
> * Better timeout handling for permissions tests
> * Several logging improvements, including full adb commands and outputs
> * Python dependencies are now pinned to prevent future package releases from
> affecting the suite
> * Android build properties with '=' are now handled correctly
> * Many other bug fixes
> 
> Release url: http://people.mozilla.org/~jgriffin/fxos-MCTS-1.3.0.11.zip

KW: Version fxos-MCTS-1.3.0.1a.zip posted to the Mobile Partners site.
This is the 1.4.0.1 release of the MCTS, for devices running FirefoxOS v1.4.  Note that devices running FirefoxOS v1.3 should continue to use releases in the 1.3.0.x version line.

This is the initial MCTS release for v1.4; it omits web-platform-tests pending the resolution of bug 1036708; a future release will contain these tests.

Release url:  http://people.mozilla.org/~jgriffin/fxos-MCTS-1.4.0.1.zip
(In reply to Jonathan Griffin (:jgriffin) from comment #34)
> This is the 1.4.0.1 release of the MCTS, for devices running FirefoxOS v1.4.
> Note that devices running FirefoxOS v1.3 should continue to use releases in
> the 1.3.0.x version line.
> 
> This is the initial MCTS release for v1.4; it omits web-platform-tests
> pending the resolution of bug 1036708; a future release will contain these
> tests.
> 
> Release url:  http://people.mozilla.org/~jgriffin/fxos-MCTS-1.4.0.1.zip

KW: Version fxos-MCTS-1.4.0.1.zip posted to the Mobile Partners site.
This is the 1.3.0.12 release of the MCTS, for devices running Firefox v1.3.  It replaces version 1.3.0.11 of the suite.  Devices running FirefoxOS v1.4 should use a release on the 1.4.0.x version line.

This release fixes the way we identify missing WebAPI methods; see bug 1053174.

Release url:  http://people.mozilla.org/~jgriffin/fxos-MCTS-1.3.0.12.zip
This is the 1.4.0.2 release of the MCTS, for devices running FirefoxOS v1.4.  Note that devices running FirefoxOS v1.3 should continue to use releases in the 1.3.0.x version line.

New in this release:

* A fix in the way we identify missing WebAPI methods; see bug 1053174.
* Removed some checks for sender and receiver addresses in SMS tests, since they can be blank under certain conditions.

Release url: http://people.mozilla.org/~jgriffin/fxos-MCTS-1.4.0.2.zip
Depends on: 1058345
Depends on: 1058601
Depends on: 1059565
This is the 2.0.1 release of the MCTS, for devices running FirefoxOS v2.0.  Note that devices running earlier versions of FirefoxOS should use a release of the MCTS specifically targeted for that version.

New in this release:

* Initial release for FirefoxOS 2.0.

Release url:  http://people.mozilla.org/~jgriffin/fxos-MCTS-2.0.1.zip
Depends on: 1070140
Depends on: 1073049
KW: fxos-MCTS-2.0.1.zip, fxos-MCTS-1.4.0.2.zip and fxos-MCTS-1.3.0.12.zip all uploaded to Mobile Partner site.
Depends on: 1078385
Depends on: 1086171
Depends on: 1087047
Jonathan - Will there be a test suite for 1.3T or does 1.3 test suite cover both releases?
Flags: needinfo?(jgriffin)
Depends on: 1089554
(In reply to Karen Ward [:kward] from comment #40)
> Jonathan - Will there be a test suite for 1.3T or does 1.3 test suite cover
> both releases?

1.3 covers both releases.
Flags: needinfo?(jgriffin)
Depends on: 1090047
Depends on: 1090122
Depends on: 1092330
Depends on: 1092383
Depends on: 1095149
Depends on: 1095152
Depends on: 1098998
Depends on: 1102111
Depends on: 1102035
No longer depends on: 984559
Depends on: 1041755
Depends on: 1037536
Depends on: 1024404
Depends on: 1110545
Depends on: 1111085
When do you expect the 2.1 version to be ready?  Will it be attached here per our 'usual' process?
Flags: needinfo?(jgriffin)
I expect 2.1 to be ready sometime in early January, and yes it will be added to this bug in the usual way.
Flags: needinfo?(jgriffin)
Depends on: 1114432
Depends on: 1116401
Depends on: 1116418
Depends on: 1116870
Depends on: 1116878
Depends on: 1116879
This is version 2.1.0 of the MCTS, supporting Firefox OS version 2.1:  http://people.mozilla.org/~jgriffin/fxos-MCTS-2.1.0.zip

Note that devices running earlier versions of FirefoxOS should use a release of the MCTS specifically targeted for that version.

New in this release:

* Initial release for FirefoxOS 2.0.

Known issues:

* Bluetooth tests fail with "app = null" - bug 1116878
* Apps webapi tests are disabled because they time out - bug 1110545
* web-platform-tests are disabled - bug 1086171
(In reply to Jonathan Griffin (:jgriffin) from comment #44)
> This is version 2.1.0 of the MCTS, supporting Firefox OS version 2.1: 
> http://people.mozilla.org/~jgriffin/fxos-MCTS-2.1.0.zip
> 
> Note that devices running earlier versions of FirefoxOS should use a release
> of the MCTS specifically targeted for that version.
> 
> New in this release:
> 
> * Initial release for FirefoxOS 2.0.
> 
And of course that should be "Initial release for Firefox 2.1".
Depends on: 1121665
This is version 2.1.0 of the MCTS, supporting Firefox OS version 2.1:  http://people.mozilla.org/~jgriffin/fxos-MCTS-2.1.1.zip

Note that devices running earlier versions of FirefoxOS should use a release of the MCTS specifically targeted for that version.

New in this release:

* Fix 'apps' webapi tests to run correctly with Firefox OS 2.1
* Fix 'telephony' webapi tests to run correctly with Firefox OS 2.1
* Prevent errors when logs contain non-ascii characters

Known issues:

* Bluetooth tests fail with "app = null" - bug 1116878
* web-platform-tests are disabled - bug 1086171
For telephony, I still got 2 failed cases.
- webapi_tests.telephony.test_telephony_incoming_hold_outgoing.TestTelephonyIncomingHoldOutgoing.test_telephony_incoming_hold_outgoing

   AssertionError: Failed to hangup the second call or change the state of first call

- webapi_tests.telephony.test_telephony_incoming_multiple.TestTelephonyIncomingMultiple.test_telephony_incoming_multiple
   AssertionError: Failed to hangup the second call or change the state of first call

(In reply to Jonathan Griffin (:jgriffin) from comment #46)
> This is version 2.1.0 of the MCTS, supporting Firefox OS version 2.1: 
> http://people.mozilla.org/~jgriffin/fxos-MCTS-2.1.1.zip
> 
> Note that devices running earlier versions of FirefoxOS should use a release
> of the MCTS specifically targeted for that version.
> 
> New in this release:
> 
> * Fix 'apps' webapi tests to run correctly with Firefox OS 2.1
> * Fix 'telephony' webapi tests to run correctly with Firefox OS 2.1
> * Prevent errors when logs contain non-ascii characters
> 
> Known issues:
> 
> * Bluetooth tests fail with "app = null" - bug 1116878
> * web-platform-tests are disabled - bug 1086171

(In reply to Jonathan Griffin (:jgriffin) from comment #46)
> This is version 2.1.0 of the MCTS, supporting Firefox OS version 2.1: 
> http://people.mozilla.org/~jgriffin/fxos-MCTS-2.1.1.zip
> 
> Note that devices running earlier versions of FirefoxOS should use a release
> of the MCTS specifically targeted for that version.
> 
> New in this release:
> 
> * Fix 'apps' webapi tests to run correctly with Firefox OS 2.1
> * Fix 'telephony' webapi tests to run correctly with Firefox OS 2.1
> * Prevent errors when logs contain non-ascii characters
> 
> Known issues:
> 
> * Bluetooth tests fail with "app = null" - bug 1116878
> * web-platform-tests are disabled - bug 1086171
Flags: needinfo?(jgriffin)
Depends on: 1124899
Moving discussion of the 2.1 telephony issues to bug 1124899.
Flags: needinfo?(jgriffin)
Depends on: 1124986
(In reply to Jonathan Griffin (:jgriffin) from comment #46)
> This is version 2.1.0 of the MCTS, supporting Firefox OS version 2.1: 
> http://people.mozilla.org/~jgriffin/fxos-MCTS-2.1.1.zip
> 
> Note that devices running earlier versions of FirefoxOS should use a release
> of the MCTS specifically targeted for that version.
> 
> New in this release:
> 
> * Fix 'apps' webapi tests to run correctly with Firefox OS 2.1
> * Fix 'telephony' webapi tests to run correctly with Firefox OS 2.1
> * Prevent errors when logs contain non-ascii characters
> 
> Known issues:
> 
> * Bluetooth tests fail with "app = null" - bug 1116878
> * web-platform-tests are disabled - bug 1086171

Version 2.1.1 zip file posted to Mobile Partner site (https://mobilepartners.mozilla.org/market/certify-your-ffos-device/)
Summary: Create B2G Certification Tests → [meta] Create B2G Certification Tests
Depends on: 1133434
Depends on: 1135094
Depends on: 1137081
Hi Brian, when do you expect the MCTS for 2.2 to be ready?
Flags: needinfo?(brhuang)
(In reply to Karen Ward [:kward] from comment #50)
> Hi Brian, when do you expect the MCTS for 2.2 to be ready?

Hi Karen,

The first draft could be ready by the mid of the month (March). The official release will be the end of April.

Brian
Flags: needinfo?(brhuang)
Depends on: 1147771
Anything ready to post on the Mobile Partner site?
Flags: needinfo?(brhuang)
(In reply to Karen Ward [:kward] from comment #52)
> Anything ready to post on the Mobile Partner site?

We just finish the first draft and doing test. We can release soon. I include Al here to update when we are ready.
Flags: needinfo?(brhuang) → needinfo?(atsai)
(In reply to Karen Ward [:kward] from comment #52)
> Anything ready to post on the Mobile Partner site?

We just finish the first draft and doing test. We can release soon. I include Al here to update when we are ready.
Attached file fxos-mcts-2.0.2.zip
Hi Karen,

Would you mind to publish the v2.0 release to the partner website for us? The one on the website has some failures with adb connection and package version. fxos-mcts-2.0.2.zip should with the fixes.
Flags: needinfo?(kward)
(In reply to Brian Huang [:brianhuang][:brhuang] from comment #53)
> (In reply to Karen Ward [:kward] from comment #52)
> > Anything ready to post on the Mobile Partner site?
> 
> We just finish the first draft and doing test. We can release soon. I
> include Al here to update when we are ready.

For v2.2 release, since FC was postponed to April 29th. I'll prefer to announce MCTS v2.2 on April 29th as well. If there's any partners would like to have a pre-test, we can discuss this offline. Please feel free to let me know if there's any concerns.
Flags: needinfo?(atsai)
(In reply to oo[oouyang] from comment #55)
> Created attachment 8592662 [details]
> fxos-mcts-2.0.2.zip

The zip file only has 77 Bytes and is not downloading correctly.  Please try to upload again or give me a url where I can access the file so that I can update the Mobile Partner site.
Flags: needinfo?(kward)
There is file size limit with bugzilla.
So I gave a google doc share link as attachment.
Please check it again.

https://drive.google.com/file/d/0B3lbWXmgZ-UAV0E4U1gtdGpjY3c/view?usp=sharing
Flags: needinfo?(kward)
(In reply to oo[oouyang] from comment #59)
> There is file size limit with bugzilla.
> So I gave a google doc share link as attachment.
> Please check it again.
> 
> https://drive.google.com/file/d/0B3lbWXmgZ-UAV0E4U1gtdGpjY3c/view?usp=sharing

The English version of the site has been changed.  The Certify your Device page now has MCTS 2.0.2 test suite.
Flags: needinfo?(kward)
Wanting or Lufei - Please update the Chinese version of the "Certify Your Device" Page to link to Version 2.0.2 of the MCTS test suite (for Version 2.0).  Thanks!
Flags: needinfo?(wxiong)
Flags: needinfo?(ljia)
Depends on: 1159161
Any update?
updates done for Chinese version.
Status: NEW → RESOLVED
Closed: 5 years ago
Flags: needinfo?(wxiong)
Resolution: --- → FIXED
We keep this bug open for future versions of the MCTS
Status: RESOLVED → REOPENED
Flags: needinfo?(ljia)
Resolution: FIXED → ---
Updated mobile partner site with https://github.com/mozilla-b2g/fxos-certsuite/archive/2.1.3.zip 
Release Notes for this version:
* Bug 1116418 - Try harder to fix unicode decoding error
* Bug 1124899 - fix not hangup all phone when some cases failed
* Fix diff CLI args
* Bug 1000937 - semiauto web server should not cache static resources
* Bug 1116878 - add resume function for telephony behaviour change.
* bug 1116978 - return bt_adapter's attributes as an object
* add securitysuite into v2.1
* 2.1 ssl stingray
@Wanting, please update the Chinese version of the site too.
Flags: needinfo?(wxiong)
updates done.
Status: REOPENED → RESOLVED
Closed: 5 years ago5 years ago
Flags: needinfo?(wxiong)
Resolution: --- → FIXED
Hi Wanting, Please update the Chinese version of the mobile partner site to show the MCTS version for Release 2.2.  The 2.2 test suite is already loaded into the media library on the site. Thanks!
Assignee: nobody → wxiong
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Chinese site is updated accordingly.
Status: REOPENED → RESOLVED
Closed: 5 years ago5 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.