518369 - [mozmill] Opening the Downloads dialog

Status: RESOLVED FIXED

(Mozilla QA Graveyard :: Mozmill Tests, defect)

Description

[Henrik Skupin [:whimboo][⌚️UTC+1]]

This Mozmill test will cover the following Litmus tests:

https://litmus.mozilla.org/show_test.cgi?id=5936
https://litmus.mozilla.org/show_test.cgi?id=7979

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Attachment: Patch v1

Test with a new shared module called DownloadsAPI which we can extend in the near future.

Comment 2

[Shawn Wilsher :sdwilsh]

Er, why?  I think all of this is covered by automated tests run by tinderbox.  Additionally, it looks like your test doesn't listen for windows properly (you can register for a window open topic).

# Ctrl+J on Windows, Cmd+J on Mac or Ctrl+Shift+Y on Linux.
http://mxr.mozilla.org/mozilla-central/source/toolkit/components/downloads/test/browser/browser_bug414214.js#92

For that matter, bug 436552 has a patch that is essentially completed that automates the rest of the test bits.  Your efforts would be better spent finishing the work in that bug and disabling the litmus test.

In general, there a number of "automate litmus test X" in the downloads component that have mostly finished patches.  Please look at them first.

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Shawn, for now we will create Mozmill tests even they are already automated or not. We haven't got to the point where we decide if the automated test is enough. Until those tests are not run on localized builds we will further see a litmus test. The Mozmill suite is new and we need as much as possible tests. We already had a couple of tests which have been caused a side-effect which wasn't covered by any automated test.

(In reply to comment #2)
> Additionally, it looks like your test doesn't listen for windows properly (you
> can register for a window open topic).

That is something we will do next quarter.

Comment 4

[Shawn Wilsher :sdwilsh]

Hey, if you want to duplicate efforts, it's your resources, not mine...

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+1]]

(In reply to comment #1)
> Created an attachment (id=402376) [details]
> Patch v1
> 
> Test with a new shared module called DownloadsAPI which we can extend in the
> near future.

Aakash, before you ask at the moment we cannot get the localized commandKeys because they are located inside a DTD. I will have to update the code once bug 504635 is fixed.

Comment 6

[Aakash Desai [:aakashd]]

Comment on attachment 402376 [details] [diff] [review]
Patch v1

This works for me and passes on osx and xp. I'd like to mention that we should start looking into writing documentation about the various functions within the shared-modules somehow. This is nothing that needs to be implemented now, but something that we should think about in the same lines as what Chris Blizzard is doing with dev-docs.

Comment 7

[Mike Shaver (:shaver -- probably not reading bugmail closely)]

How do we decide to write mozmill tests, if not "we need additional automated coverage for this code path or user activity"?  I can understand missing the existing tests sometimes when scanning them before embarking on a mozmilling project, but to continue duplicating the tests -- thereby doubling the maintenance effort for them when the DLM changes -- once the duplication has been pointed out seems odd.

Needing mozmill tests is fine, I suppose, but we should be writing ones we don't have, not rewriting ones we have just to add to the mozmill corpus, no?  Apologies if I'm missing something here, though!

Comment 8

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Landed as:
http://hg.mozilla.org/qa/mozmill-tests/rev/8c711c72cc4a
http://hg.mozilla.org/qa/mozmill-tests/rev/b66a6bd04479

(In reply to comment #6)
> (From update of attachment 402376 [details] [diff] [review])
> This works for me and passes on osx and xp. I'd like to mention that we should
> start looking into writing documentation about the various functions within the
> shared-modules somehow. This is nothing that needs to be implemented now, but

As said lately its on my list for the next month.

Comment 9

[Henrik Skupin [:whimboo][⌚️UTC+1]]

(In reply to comment #7)
> How do we decide to write mozmill tests, if not "we need additional automated
> coverage for this code path or user activity"?  I can understand missing the
> existing tests sometimes when scanning them before embarking on a mozmilling
> project, but to continue duplicating the tests -- thereby doubling the
> maintenance effort for them when the DLM changes -- once the duplication has
> been pointed out seems odd.

Such a discussion should be better of in the newsgroups. But to give a short answer in-front... We haven't checked yet, in which areas automated tests are already existent. But that's not the problem. Having users tested those highly used code paths would be even better. Such a test is short and simple to do.

But the main point why we need those is that it help us drastically for localized versions. We don't run any automated test for localized builds. And what I have heard it will not be changed in the near future. Given that localizers would have to run Litmus tests to verify that their version is running. Removing those Litmus tests would be a step backwards because we could miss major points in testing. A really good example was the XML parsing error for the software update dialog for mk and sr for Firefox 3.5. If we would have a test which covers this dialog we would have found it.

I will have a talk at MozCamp next week and will promote Mozmill as testing tool. This framework can be set up by nearly everyone in some simple steps. Running those tests is easy too. So we will hopefully will get much more attention and people running those tests on localized nightly builds mainly days and/or weeks before a major release.

> Needing mozmill tests is fine, I suppose, but we should be writing ones we
> don't have, not rewriting ones we have just to add to the mozmill corpus, no? 
> Apologies if I'm missing something here, though!

I hope my above answer is more concrete this time. I will have a blog post once I have been returned from the conference. Seems like we should more clarify our target.

Comment 11

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Move of Mozmill Test related project bugs to newly created components. You can
filter out those emails by using "Mozmill-Tests-to-MozillaQA" as criteria.