696356 - Split Android crashtests since they take a long time to run

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Marco Bonardo [:mak]]

I'm not sure if this is feasible or already planned, but Android Crashtests are taking more than 1 hour and regression finding on these results is really time consuming.
May be useful to have C1 C2, like it's done for jsreftests and reftests, to reduce time to about 35-40 mins.

Comment 1

[Marco Bonardo [:mak]]

hm, looks like the long time was due to an orange causing them to timeout... now they completed in 30 minutes...

So my question is about why they take 40 minutes more than usual if one of them timeouts (due to a crash).

Comment 2

[Phil Ringnalda (:philor)]

The failure message tells you that, "buildbot.slave.commands.TimeoutError: command timed out: 2400 seconds without output, killing pid 38940" - 30 minutes or so to get to the hang, then 40 minutes waiting to see if it's really hung.

Comment 3

[John Ford [:jhford] CET/CEST Berlin Time]

Bear, think you'd have a chance to look at this?

Comment 4

[Mike Taylor [:bear]]

bouncing to Joel to see if ATeam needs to do something before RelEng

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

We can split crashtests into two chunks with no consequences in the harness or tests.  Thanks for checking!

Comment 6

[Marco Bonardo [:mak]]

actually, the title is misleading, the issue is related with too long timeouts, without timeouts crashtests there take about 30 minutes, that is in line with other harness.
but if a crashtest timeouts it takes about 70 minutes to be reported.

Comment 7

[Joel Maher ( :jmaher ) (UTC -8)]

so splitting the crashtests into 2 runs will make the total runtime ~15 minutes.  We need to solve the 40 minute timeout problem, but on android there are test cases where we can wait for 10+ minutes without seeing any output.  

Maybe we put the limit at 15 minutes with not output and kill the test?  This would make a maximum test run time on crashtest 30 minutes after splitting into 2 runs.

Comment 8

[Mike Taylor [:bear]]

fixed as part of bug 697470