476236 - Mozmill test for check top site - Amazon

Status: VERIFIED FIXED

(Mozilla QA Graveyard :: Mozmill Tests, defect)

Description

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Attachment: Patch v1

The following test checks the Amazon website for several elements. That's the only thing we can do in MozMill. With the existing reftests for the top sites,
layout issues will be already covered.

Comment 1

[cmtalbert]

This looks really good, Henrik.  I have a couple of comments though before deciding how to review it:
* Shouldn't the delayed* functions go into a shared module?  They seem like they would be just the right thing for a shared module.

* While I do think this test is good, I'd love to see the test expanded to do the following:
1. Click the first thing that you can buy on the site and add it to the cart
2. Then in the last step, open the cart and ensure that the thing is there.

This would test that the basic functionality of the web application of amazon works, which is enough for us, I believe.  The problem here is whether or not their page is crafted such that there is a general, repeatable way to grab that first element and click "add to cart".  Does that exist?

Once we get these issues sorted, I'll know how to review this test.

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to comment #1)
> * Shouldn't the delayed* functions go into a shared module?  They seem like
> they would be just the right thing for a shared module.

The question is should this behavior not better integrated into MozMill itself? All tests will run into this problem when handling lazily loaded content. This doesn't only apply for content but also chrome elements - see the bookmarkEditPanel. Mikeal and Adam, what do you both think?

> * While I do think this test is good, I'd love to see the test expanded to do
> the following:
> 1. Click the first thing that you can buy on the site and add it to the cart
> 2. Then in the last step, open the cart and ensure that the thing is there.
> 
> This would test that the basic functionality of the web application of amazon
> works, which is enough for us, I believe.  The problem here is whether or not
> their page is crafted such that there is a general, repeatable way to grab that
> first element and click "add to cart".  Does that exist?

I already had those steps in my test but decided to remove them again because elements varies between different categories. I did some further investigations and it looks like we can use it when we definitely get results from the same category each time. I've modified the second search to "The Police CD" which will return a list of audio cd's. That should work and doesn't fail on my side.

I'll attach an updated version of the patch.

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Attachment: Patch v2 (with cart check)

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Attachment: Patch v2.1 (checked-in)

Sorry, missed a qrefresh.

Comment 5

[cmtalbert]

(In reply to comment #2)
> (In reply to comment #1)
> > * Shouldn't the delayed* functions go into a shared module?  They seem like
> > they would be just the right thing for a shared module.
> 
> The question is should this behavior not better integrated into MozMill itself?
> All tests will run into this problem when handling lazily loaded content. This
> doesn't only apply for content but also chrome elements - see the
> bookmarkEditPanel. Mikeal and Adam, what do you both think?
> 
All right, let's open another bug to settle this issue, and leave them in the test as is. 

> > * While I do think this test is good, I'd love to see the test expanded to do
> > the following:
> > 1. Click the first thing that you can buy on the site and add it to the cart
> > 2. Then in the last step, open the cart and ensure that the thing is there.
> > 
> > This would test that the basic functionality of the web application of amazon
> > works, which is enough for us, I believe.  The problem here is whether or not
> > their page is crafted such that there is a general, repeatable way to grab that
> > first element and click "add to cart".  Does that exist?
> 
> I already had those steps in my test but decided to remove them again because
> elements varies between different categories. I did some further investigations
> and it looks like we can use it when we definitely get results from the same
> category each time. I've modified the second search to "The Police CD" which
> will return a list of audio cd's. That should work and doesn't fail on my side.
> 
Sweet, looks good. r=ctalbert
Thanks!

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to comment #5)
> > The question is should this behavior not better integrated into MozMill itself?
> > All tests will run into this problem when handling lazily loaded content. This
> > doesn't only apply for content but also chrome elements - see the
> > bookmarkEditPanel. Mikeal and Adam, what do you both think?
> > 
> All right, let's open another bug to settle this issue, and leave them in the
> test as is. 

Filed as bug 477135.

Changeset: http://hg.mozilla.org/qa/mozmill-tests/rev/ff329b4c0661

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+2]]

I had to update the test. Let's reopen this bug for now. I think its better to track changes in the same bug. Patch is following...

Comment 8

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Attachment: Follow-up patch

What does the patch update:
* Remove all cookies when test has been finished. The currently checked-in version will fail after the second run.
* Now we can select a category from the select field
* The search field at the bottom has been removed by Amazon

Now it looks much cleaner and the only bug I have to wait is for the delayed action.

Comment 9

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Attachment: Follow-up patch v2

Ok, some more cleaning up for this test:

* Removed delayedClick function in favor of delayedAsserNode and our future work in bug 477135

Comment 10

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Dependencies were introduced and broken elements fixed. We can call it fixed with:
http://hg.mozilla.org/qa/mozmill-tests/rev/9554a9d2373f

Comment 11

[Henrik Skupin [:whimboo][⌚️UTC+2]]

No failures anymore with bug 516729 fixed.

Comment 12

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Mass move of Mozmill Test related project bugs to newly created components. You can filter out those emails by using "Mozmill-Tests-to-MozillaQA" as criteria.