1183717 - Default individual reftest timeout is too short on Android 4.3 Debug

Status: RESOLVED FIXED

(Testing :: Reftest, defect)

Description

[Geoff Brown [:gbrown]]

runreftest.py uses a default of 5*60 seconds for options.timeout, but on Android 4.3 Debug, some tests run longer than 300 seconds (and fail due to timeout). For instance, in jsreftest-2, parse-array-gc.js often takes nearly 400 seconds.

http://ftp.mozilla.org/pub/mozilla.org/mobile/try-builds/[e-mail]-543606d7de04/try-android-api-11-debug/try_ubuntu64_vm_armv7_large-debug_test-jsreftest-2-bm68-tests1-linux64-build17.txt.gz

21:40:57     INFO -  REFTEST FINISHED: Slowest test took 395323ms (http://10.0.2.2:8854/jsreftest/tests/jsreftest.html?test=ecma_5/JSON/parse-array-gc.js)


It looks like b2g mochitests encountered a similar issue: http://hg.mozilla.org/mozilla-central/annotate/e786406bc683/testing/mochitest/runtestsb2g.py#l135

It seems reasonable to increase the default timeout for Android Debug.

Comment 1

[Geoff Brown [:gbrown]]

Attachment: increase the default timeout for remote reftests run against Debug build

I might prefer to call remotereftest.py with "--timeout 600" but I don't think there is a convenient place to do that: we do not have separate debug/opt mozharness configs and I would prefer not to change the opt timeout.

Comment 2

[Joel Maher ( :jmaher ) (UTC -8)]

Comment on attachment 8633639 [details] [diff] [review]
increase the default timeout for remote reftests run against Debug build

Review of attachment 8633639 [details] [diff] [review]:
-----------------------------------------------------------------

::: layout/tools/reftest/remotereftest.py
@@ +516,5 @@
>      reftest = RemoteReftest(automation, dm, options, SCRIPT_DIRECTORY)
>      options = parser.verifyCommonOptions(options, reftest)
>  
> +    if mozinfo.info['debug']:
> +        print "using 600 second timeout for remote debug reftests"

maybe say:
changing timeout for remote debug reftests from %s to 600 seconds" % options.timeout

is there any need to make this a structured log?

Comment 3

[Geoff Brown [:gbrown]]

(In reply to Joel Maher (:jmaher) from comment #2)
> > +    if mozinfo.info['debug']:
> > +        print "using 600 second timeout for remote debug reftests"
> 
> maybe say:
> changing timeout for remote debug reftests from %s to 600 seconds" %
> options.timeout

Sounds good.

> is there any need to make this a structured log?

All the other diagnostics in this file use print.

Comment 4

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/c66c5db7e21a

Comment 5

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/c66c5db7e21a