929608 - Upgrade to Mozmill 1.5.24 for Thunderbird

Status: RESOLVED WONTFIX

(Thunderbird :: Testing Infrastructure, defect)

Description

[Jens Müller (:tessarakt)]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:26.0) Gecko/20100101 Firefox/26.0 (Beta/Release)
Build ID: 20131019004007

Steps to reproduce:

The Mozmill used for automated GUI tests of Thunderbird should be upgraded to the latest 1.5.x, i.e., 1.5.24.

Comment 1

[Mark Banner (:standard8)]

The general procedure for upgrade is:

1) Import latest version of MozMill into the comm-central locations.
2) Run the tests
3) Fix anything that breaks
4) Confirm tests work on try server

If they still turn out green, then we should be good to take the upgrade. The upgrade doesn't need review for MozMill itself (only the fact that it's being upgraded & the tests still pass).

Comment 2

[Jens Müller (:tessarakt)]

I tested on Linux 64bit only to check the config. This worked fine: https://tbpl.mozilla.org/?tree=Thunderbird-Try&rev=d112962b2a26

Now  I am running Mozmill tests again on all platforms: https://tbpl.mozilla.org/?tree=Thunderbird-Try&rev=d112962b2a26

I hope there won't be any problems as well.

Comment 3

[Jens Müller (:tessarakt)]

Attachment: 929608-1-upgrade-mozmill-to-1_5_24-v1.patch

Mozmill 1.5.24, released on October 15th, 2013, was downloaded from https://pypi.python.org/pypi/mozmill/1.5.24.

The unpacked and renamed directory replaces the existing directory mail/test/resources/.

My main question is whether certain new files/directories are needed (or rather, wanted), e.g. mozmill.egg-info/ and PKG-INFO.

Comment 4

[Jens Müller (:tessarakt)]

(In reply to Jens Müller (:tessarakt) from comment #2)
> Now  I am running Mozmill tests again on all platforms:
> https://tbpl.mozilla.org/?tree=Thunderbird-Try&rev=d112962b2a26
> 

https://tbpl.mozilla.org/?tree=Thunderbird-Try&rev=277a45a10868 - I guess we want both debug and opt builds.

Comment 5

[Jens Müller (:tessarakt)]

All oranges are already known: bug 904152, bug 757394, bug 908352

Comment 6

[:aceman]

Thanks for looking into this Jens. Looks like we were a bit behind (1.5.16 -> 1.5.24). In the patch there is a focus change that may help on Linux theoretically (however, in my tests in the past I could not confirm this claim).

Would you want to take on the upgrade to Mozmill 2.0 in the future? Of course, we may want to wait for some maintenance releases of it first.

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+1]]

(In reply to :aceman from comment #6)
> Would you want to take on the upgrade to Mozmill 2.0 in the future? Of
> course, we may want to wait for some maintenance releases of it first.

Yes you should wait with the upgrade for comm-central, but I would highly be interested in getting feedback how the Thunderbird tests are working with Mozmill 2.0! We haven't gotten any feedback from you guys yet, so if problems exists we can fix those. Thanks.

Comment 8

[ISHIKAWA, Chiaki]

(In reply to Henrik Skupin (:whimboo) from comment #7)
> (In reply to :aceman from comment #6)
> > Would you want to take on the upgrade to Mozmill 2.0 in the future? Of
> > course, we may want to wait for some maintenance releases of it first.
> 
> Yes you should wait with the upgrade for comm-central, but I would highly be
> interested in getting feedback how the Thunderbird tests are working with
> Mozmill 2.0! We haven't gotten any feedback from you guys yet, so if
> problems exists we can fix those. Thanks.

Can you kindly describe the steps for someone like me to test this
locally, say, by downloading mozmill 2.o(?) into local source tree?

Or it is possible only in the TryServer setting.

If I can contribute, I will be happy to, especially I see some focus-related patch(?) in
the diff.

TIA

Comment 9

[Jens Müller (:tessarakt)]

I created bug 930732 to track an eventual upgrade to Mozmill 2.0. Let's continue the discussion there.

Comment 10

[:aceman]

(In reply to ISHIKAWA, Chiaki from comment #8)
> Can you kindly describe the steps for someone like me to test this
> locally, say, by downloading mozmill 2.o(?) into local source tree?
> If I can contribute, I will be happy to, especially I see some focus-related
> patch(?) in
> the diff.
The diff is for 1.5.24 and Jens is taking care of that. My question about 2.0 was also for Jens. But you can of course help him in bug 930732. I just wanted to CC you here due to the focus change in 1.5.24.

Comment 11

[Mark Banner (:standard8)]

Comment on attachment 820612 [details] [diff] [review]
929608-1-upgrade-mozmill-to-1_5_24-v1.patch

Review of attachment 820612 [details] [diff] [review]:
-----------------------------------------------------------------

I've re-run the tests a few times and the fact the orange Z on Linux 32 debug seems to be permanent has me concerned about landing this without first fixing that.

Comment 12

[Jens Müller (:tessarakt)]

As bug 904152 has been fixed apparently, I'll give this another try (pun intended): https://tbpl.mozilla.org/?tree=Thunderbird-Try&rev=a07400c85078

Comment 13

[Jens Müller (:tessarakt)]

(In reply to Jens Müller (:tessarakt) from comment #12)
> As bug 904152 has been fixed apparently, I'll give this another try (pun
> intended): https://tbpl.mozilla.org/?tree=Thunderbird-Try&rev=a07400c85078

Bug 904152 (orange on Linux 32 debug) no longer occurs ...

Instead now bug 910293 appears ...

Comment 14

[:aceman]

Can we try to push this again?

Comment 15

[:aceman]

The tree is relatively green again.
Aryx, can you push this to try server to see if it still applies?

Comment 16

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Many failing hunks, please provide an updated patch.

Comment 17

[:aceman]

I tried to use the hotfix-1.5 branch of mozmill here, but it seems to me the code is very outdated and it would not even run today. Using this branch would introduce back a ton of trailing whitespace and regress the "for each" removals and also the let/var/const fixes that are needed today. I tried to remove those regressions from the patch but it will need to reduce a 400KB patch down to ~150KB.

So it depends if it is worth pursuing this, or whether upgrading to mozmill 2 would be easier. Depends if that one has these let/var/const fixes already in.

Comment 18

[Magnus Melin [:mkmelin]]

mozmill is gone