1460239 - [wdspec] Use longer timeout for all user prompt tests

Status: RESOLVED INCOMPLETE

(Testing :: geckodriver, enhancement, P1)

Description

[Henrik Skupin [:whimboo][⌚️UTC+1]]

To be finally able to continue on bug 1434618 with an earlier shutdown of workers I would propose we increase the timeout for user prompt tests. This would unblock us from doing this major change in Firefox.

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Attachment: Bug 1460239 - [wdspec] Use longer timeout for all user prompt tests.

Most of the user prompt tests require a restart of the browser.
As such the default timeout is too short and needs to be extended.

Review commit: https://reviewboard.mozilla.org/r/242694/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/242694/

Comment 2

[Andreas Tolfsen ❲:ato❳]

(In reply to Henrik Skupin (:whimboo) from comment #0)

> To be finally able to continue on bug 1434618 with an earlier
> shutdown of workers I would propose we increase the timeout for
> user prompt tests. This would unblock us from doing this major
> change in Firefox.

Why would extending the timeout unblock us?  Are these currently
timing out?

I thought the plan here was to _disable_ the user prompt handler
tests until we have implemented it.

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+1]]

I can disable but that doesn't help us to identify any remaining shutdown hangs. Increasing the timeout might be counter-productive but at least we could identify shutdown hangs by letting the background hang monitor to kick in.

But if you think that we should disable them all, let me know and we can do that instead.

Comment 4

[Andreas Tolfsen ❲:ato❳]

(In reply to Henrik Skupin (:whimboo) from comment #3)

> I can disable but that doesn't help us to identify any remaining
> shutdown hangs. Increasing the timeout might be counter-productive
> but at least we could identify shutdown hangs by letting the
> background hang monitor to kick in.
> 
> But if you think that we should disable them all, let me know and
> we can do that instead.

I think I would personally prefer not to run tests for features we
know we don’t support and which we know are causing problems due
to significant shutdown hangs, but if you think the tests are useful
for identifying the shutdown hang then landing them with an extended
timeout would make some sense.

Comment 5

[Andreas Tolfsen ❲:ato❳]

Comment on attachment 8974351 [details]
Bug 1460239 - [wdspec] Use longer timeout for all user prompt tests.

https://reviewboard.mozilla.org/r/242694/#review248582

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Andrea, can you please run your patch with the one here applied too? Do the wd failures stop? If not we will have to completely disable the tests.

Comment 7

[Andrea Marchesini [:baku]]

Here the push: https://treeherder.mozilla.org/#/jobs?repo=try&revision=b20ebbb2bc64c73e1d7f606db8e99ada7a367775

Comment 8

[Henrik Skupin [:whimboo][⌚️UTC+1]]

So that looks good. I only forgot to add the longer timeout to the new session tests which have the same problem. I will update the patch in a moment.

Comment 9

[Henrik Skupin [:whimboo][⌚️UTC+1]]

I checked and both the new session tests have already a longer timeout. So there is nothing wrong on our side. Given some extra findings I made and which I will report on bug 1434618 I'm a bit vague if we should really push that patch on this bug. I think it will be wise to wait if those user prompts tests generally fail on integration branches.

Comment 10

[Henrik Skupin [:whimboo][⌚️UTC+1]]

As it looks like most of the unexpected failures and timeouts we currently see are from webrender enabled builds. As such I'm going to disable the tests for that particular build on bug 1461253.

Comment 11

[Henrik Skupin [:whimboo][⌚️UTC+1]]

I would close this bug for now and concentrate more on bug 1460193.