[META] 2.2 Test SPEC review

NEW
Unassigned

Status

Firefox OS
General
3 years ago
3 years ago

People

(Reporter: gchang, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(7 attachments, 1 obsolete attachment)

(Reporter)

Description

3 years ago
Each feature owner should attach the test spec and invite reviewers to sign off.
(Reporter)

Updated

3 years ago
Summary: [META] Test SPEC review → [META] 2.2 Test SPEC review

Comment 1

3 years ago
Created attachment 8557792 [details]
Productivity Copy/Paste test cases

Shing, 
Please help review my test cases. Thanks.
Flags: needinfo?(slyu)
Attachment #8557792 - Flags: review?(slyu)
Created attachment 8557842 [details]
App-grouping Test Cases
Attachment #8557842 - Flags: review?(echang)
Created attachment 8558285 [details]
Metrics Test Cases
Comment on attachment 8558285 [details]
Metrics Test Cases

Hi Gerry,

Please review my test cases, the only difference in 2.2 is that there will be more information in the metrics payload.
Attachment #8558285 - Attachment mime type: text/plain → text/html
Flags: needinfo?(slyu)
Attachment #8558285 - Flags: review?(gchang)
(Ignore Comment 5, I can't delete it)

@Teri: The test cases generally looks good to me. There are some minor issues though:

1. You can consider testing the non-selectable areas by long-tapping the high-risk areas that the RD may forget to mark as non-selectable.

2. Consider using the following Markdown syntax: `[Link Title](http://foo.com)` to create links in those test cases that links to the general cases.
Flags: needinfo?(twen)
Comment on attachment 8557842 [details]
App-grouping Test Cases

1. Case to remove apps and make sure remaining icon are moved to the left?
2. Case to collapse/Expand in edit mode
3. Delete/move from a group with 1 item, make sure the grouping is gone.
4. What's the max number of icons in a group?
Attachment #8557842 - Flags: review?(echang) → review-

Comment 8

3 years ago
Created attachment 8558902 [details]
NFC Test Cases
Attachment #8558902 - Flags: review?(hcheng)
Eric, thank you!

(In reply to Eric Chang [:ericcc] [:echang] from comment #7)
> Comment on attachment 8557842 [details]
> App-grouping Test Cases
> 
> 1. Case to remove apps and make sure remaining icon are moved to the left?
I assume it would be covered by Homescreen old test cases, and will find it and update the suite.
> 2. Case to collapse/Expand in edit mode
covered by 15219, 15220, 15495, 15496
> 3. Delete/move from a group with 1 item, make sure the grouping is gone.
covered by 15290, 15291
> 4. What's the max number of icons in a group?
No, it is not covered now. I assume it is unlimited and would be a potential bug for OOM in the future.......need to think how to cover it......

Comment 10

3 years ago
(In reply to Shing Lyu [:slyu] from comment #6)
Thanks Shing for the review. 

> (Ignore Comment 5, I can't delete it)
> 
> @Teri: The test cases generally looks good to me. There are some minor
> issues though:
> 
> 1. You can consider testing the non-selectable areas by long-tapping the
> high-risk areas that the RD may forget to mark as non-selectable.
> 
I have bugs opened, and will add test cases to moztrap.

> 2. Consider using the following Markdown syntax: `[Link
> Title](http://foo.com)` to create links in those test cases that links to
> the general cases.
Changed formatting to include test case title, see https://moztrap.mozilla.org/manage/case/15207/.
Flags: needinfo?(twen)
Comment on attachment 8557842 [details]
App-grouping Test Cases

leave boundary test not covered...
Attachment #8557842 - Flags: review- → review?(echang)
Created attachment 8558969 [details]
Bug 1001861 - short_name support.txt

Alison, since you have developed a webapp for FxOS, please help review this one about webapp manifest attribute. Thanks!
Attachment #8558969 - Flags: review?(ashiue)

Comment 13

3 years ago
Comment on attachment 8558969 [details]
Bug 1001861 - short_name support.txt

Hi Hermes,
I suggest add some test cases about the correlation between "name" and "short_name". But I am not sure if they have clear defined since I could not find the design conclusion from the bug discussion.
Attachment #8558969 - Flags: review?(ashiue) → review+
(In reply to Alison Shiue from comment #13)
> Comment on attachment 8558969 [details]
> Bug 1001861 - short_name support.txt
> 
> Hi Hermes,
> I suggest add some test cases about the correlation between "name" and
> "short_name". But I am not sure if they have clear defined since I could not
> find the design conclusion from the bug discussion.

Thanks! asking below questions to enhance coverage...

1. Will short_name be used in search app results?
2. Is there any correlation between "name" and "short_name"? Do we have any rule for it? Are we going to use "app-validator" or some methods to verify its correctness?
3. What is the maximum length of "short_name"? It seems not be defined in W3C spec.
Created attachment 8560324 [details]
Bug 911055 - [Dialer]DuT doesn't allow to dial a <<pause>>

Hi Alison, Please help to review test case for bug 911055, thank you
Attachment #8560324 - Flags: review?(ashiue)
Comment on attachment 8557842 [details]
App-grouping Test Cases

r+, thank you.
Attachment #8557842 - Flags: review?(echang) → review+
Created attachment 8560380 [details]
Rocketbar v2.2 - Search Provider (Bug 1098494) & part of Rocketbar refactor (Bug 1099157)

Gerry, help!
Attachment #8560380 - Flags: review?(gchang)
(Reporter)

Updated

3 years ago
Attachment #8558285 - Flags: review?(gchang) → review+
Comment on attachment 8558902 [details]
NFC Test Cases

Comments only for SE API:
4.1.1 without SIM card case?
4.3.* reconnect a logical/basic channel?
4.4.4 transmit with incorrect SECommand?
    a. right format but not exist command
    b. incorrect format

Test case to cover "SEIoError" and "SEGenericError"?
Flags: needinfo?(ashiue)

Comment 19

3 years ago
(In reply to Hermes Cheng[:hermescheng] from comment #18)
> Comment on attachment 8558902 [details]
> NFC Test Cases
> 
> Comments only for SE API:
> 4.1.1 without SIM card case?
https://moztrap.mozilla.org/manage/cases/?filter-id=15275 covered this case.

> 4.3.* reconnect a logical/basic channel?
I think if a channel has been closed, we just can open a new channel to the specific applet again, not reconnect.

> 4.4.4 transmit with incorrect SECommand?
>     a. right format but not exist comman
OK, I would try add a new test case such as "Open a channel to PPSE but send CRS APDU command".

>     b. incorrect format
I have ever created a test case "Command parameter shorter than 4 bytes" which already removed from test spec since developer suggest "SEChannel.transmit expects SECommand as an argument. SECommand is strongly typed and each of the 4 bytes is required in SECommand constructor. So it's not possible to transmit a shorter command." 
> 
> Test case to cover "SEIoError" and "SEGenericError"?
Until now, the error code still not defined well. (bug 1118102)
Flags: needinfo?(ashiue)

Comment 20

3 years ago
Comment on attachment 8560324 [details]
Bug 911055 - [Dialer]DuT doesn't allow to dial a <<pause>>

Looks good to me, r+, thank you.
Attachment #8560324 - Flags: review?(ashiue) → review+
Comment on attachment 8558902 [details]
NFC Test Cases

For Bug 1102019:
MozTrap ID - Comment
#15273  - similar with #15272, insert step 2: open a Contact app and then open NFC test app.
#15271  - do we need to add a similar case which use two phone, and send NDEF to each other?
#15270 - How do we verify the results written to tag?
#15269 - Suggest to tap the tag to show its content and then to format it. Make sure the tag contains correct info
Flags: needinfo?(ashiue)
Attachment #8558902 - Flags: review?(hcheng) → review+

Comment 22

3 years ago
(In reply to Hermes Cheng[:hermescheng] from comment #21)
> Comment on attachment 8558902 [details]
> NFC Test Cases
> 
> For Bug 1102019:
> MozTrap ID - Comment
> #15273  - similar with #15272, insert step 2: open a Contact app and then
> open NFC test app.
#15272 open Contact at step 4.
#15273 doesn't need to open Contact app since I want to test that System app would not dispatch event to suitable (Contact) app when current foreground app can handle it. The expected results at step 3 are: (1) NFC test app should show tag info (2) Contacts app should not import contact from tag

> #15271  - do we need to add a similar case which use two phone, and send
> NDEF to each other?
This case is exactly for testing send NDEF via phone to phone (peer to peer)

> #15270 - How do we verify the results written to tag?
We verify at Step 4: Go back to home screen and tap tag again - Browser would open URL which is written by NFC test app 

> #15269 - Suggest to tap the tag to show its content and then to format it.
> Make sure the tag contains correct info
OK, I've updated. (https://moztrap.mozilla.org/manage/cases/?filter-id=15269)

Thanks for your review!
Flags: needinfo?(ashiue)
0. Talk to EPM about the process.

1. A meta bug 
2. QA test review bug links to 1.
3. Also add UX link in 2.
4. Reviewer: UX, RD, EPM, QA.. suggest to have 1 of each.
Comment on attachment 8557792 [details]
Productivity Copy/Paste test cases

Sorry for the late r+
Attachment #8557792 - Flags: review?(slyu) → review+
Created attachment 8569627 [details]
Metrics Test Cases 20150226

Updated test cases for new features added to 2.2 scope
Attachment #8558285 - Attachment is obsolete: true
Attachment #8569627 - Flags: review?(gchang)
(Reporter)

Comment 26

3 years ago
Comment on attachment 8569627 [details]
Metrics Test Cases 20150226

I think one more category is to have background app running (ex: Music) and see if the background app will be calculated.
Attachment #8569627 - Flags: review?(gchang) → review+
(Reporter)

Updated

3 years ago
Attachment #8560380 - Flags: review?(gchang) → review+
You need to log in before you can comment on or make changes to this bug.