847720 - Need out-of-disk tests

Status: RESOLVED INCOMPLETE

(Firefox OS Graveyard :: General, defect)

Description

[Jonathan Griffin (:jgriffin)]

From Andreas:

"We should make sure that the phone always
successfully restarts even when the disk is completely full, and
we should make sure we handle out of disk gracefully. We probably
want to start warning when the disk is almost full, and then
refuse to write to it when it gets really fully. We might also
have to purge data on startup. We need a plan here. I am
concerned we could brick phones right now."

We need to determine if we can use emulators for this, and if not, we need to stand up another unagi/slave for potentially destructive testing like this.

Comment 1

[Jonathan Griffin (:jgriffin)]

The problem with emulators is that 'adb reboot' hangs, and restarting the emulator process entirely causes it to be launched with a clean system image..this is a design limitation of the emulator.  Therefore, I'm not sure it would adequately reflect what would happen on a real device.

From http://developer.android.com/tools/devices/emulator.html:

"The emulator creates two writeable images at startup that it deletes at device power-off. The images are:

    A writable copy of the Android system image
    The /cache partition image

The emulator does not permit renaming the temporary system image or persisting it at device power-off. "

Comment 2

[Marshall Culpepper [:marshall_law]]

Yeah, it sounds like a bank of unagis is our only real solution -- we will realistically need this long term for the update tests too, where we also expect to eventually run into bricking a test device..

FWIW, the update test harness current has the ability to fail over to a working device if any of the connected / configured devices become bricked. I think these kind of failures will always require some manual intervention to recover from, though..

Comment 3

[Geo Mealer [:geo] -- This account is inactive after 2015-07-07]

OK, so we can't boot off a custom image? That kills my other idea of perhaps having the test occur in two separate phases.

Comment 4

[Marshall Culpepper [:marshall_law]]

(In reply to Geo Mealer [:geo] from comment #3)
> OK, so we can't boot off a custom image? That kills my other idea of perhaps
> having the test occur in two separate phases.

We can boot from a custom image -- I think the test phases idea is actually a good one, this is the way the update tests work :)

The main problem is that even with isolated test phases, you have the possibility that a brick occurs, and you can never get to the device again to flash it without a physical reset. We might be able to solve this problem with an automated power relay, but I think this would need some prototyping..

Jonathan, do we currently use power relays in any of our testing automation?

Comment 5

[Jonathan Griffin (:jgriffin)]

(In reply to Marshall Culpepper [:marshall_law] from comment #4)
> 
> Jonathan, do we currently use power relays in any of our testing automation?

We don't right now, but ted is doing some experiments with them.  It's more complicated than just a power relay, since after a phone goes off for whatever reason, you need to physically press a button to turn it back on...just connecting power won't do it.

Comment 6

[Geo Mealer [:geo] -- This account is inactive after 2015-07-07]

(In reply to Marshall Culpepper [:marshall_law] from comment #4)

> The main problem is that even with isolated test phases, you have the
> possibility that a brick occurs, and you can never get to the device again
> to flash it without a physical reset. We might be able to solve this problem
> with an automated power relay, but I think this would need some prototyping..

Well, I meant isolated test phases by loading a full image against the emulator, where bricking isn't an issue for the test harness.

I hadn't really thought about it, but I guess there probably is value in the phased approach even on devices, too. Ideally the OS won't actually let you fill the filesystem as long as you don't go around it, but the second phase could be an "if everything goes wacky" negative test starting from full.

Comment 7

[Stephen Donner [:stephend] Not actively reading bugmail]

Just some points from my end:

1) I'd be happiest if we used an emulator, and it did what we want, because I, too, am slightly worried about bricking phones (but I'm told more are coming), and we are down to a few free left.  Still, I'll do what we think is best, for coverage.

2) The devices are near me (either 3rd floor or in Haxxor, 2nd floor), so -- although not ideal -- I (and members of my team) are available to manually re-flash a device, should that be necessary.

Let me know if there's anything else I can do!