(a-team) Panda boards image work

RESOLVED FIXED

Status

Testing
General
RESOLVED FIXED
6 years ago
5 years ago

People

(Reporter: armenzg, Unassigned)

Tracking

Trunk
ARM
Android
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

(Reporter)

Description

6 years ago
Not sure if the binaries had to be confidential in any way and that is why I set the flag. If you don't think so please remove.

With this bug we would like to check that the build works as expected and can be reproduced by others beside ted.

Once we do we can ask Jake to get his boards imaged and cabled on the rack.

I believe blassey, jmaher and myself had signed up to test this.

########################################
You can just use "dd" to copy the image, like:
bzip2 -dc linaro-android-mac-fix.img.bz2 | dd of=/dev/<whatever> bs=32M

where <whatever> is the device name of your SD card.

-Ted

----- Original Message -----
> Do you have instructions on how to set the panda up?
> Do I just unzip the contents unto the SD card?
>
> Ted Mielczarek <tmielczarek@mozilla.com> wrote:
>
>> I had linked this on IRC yesterday, and in the wiki notes, but I
>> figured I'd send out an email as well just in case anyone missed
>> it:
>> http://people.mozilla.com/~tmielczarek/linaro-android-mac-fix.img.bz2
>>
>> I also documented the steps I took to create this image on the wiki:
>> https://wiki.mozilla.org/Auto-tools/Projects/Pandaboard_Setup/Using_Linaro_Prebuilt_Image
>>
>> -Ted
>
(Reporter)

Updated

6 years ago
Blocks: 769429
(Reporter)

Updated

6 years ago
Duplicate of this bug: 725844
Group: mozilla-corporation-confidential
Can be put on Panda: YES
Can run tests: YES
Can run tests as we normally do: NO

Here are the issues I have found:
* unable to connect to sutagent
** adb shell in and it is running
** adb shell in and sometimes it is not running
* adb reboot doesn't always bring the device back up
** requires a power toggle
* sutagent takes an extra 30 seconds to come online
* I have tried 10 times to run tests (after a fresh reboot) and only 1 instance ran

I suspect the majority of the issues here can be resolved debugging watcher/sutagent.
(Reporter)

Comment 3

6 years ago
Adjusting the summary to match what we're trying to address now.
Please move to a correct component if you can think of a more appropriate one.
Summary: Test that ted's build can be put unto other panda boards and can run tests → Provide staging usable image for panda boards
(Reporter)

Comment 4

6 years ago
Can we say that we have an image that we can test on the panda boards that Jake has?
And for releng to test that foopy/CP/buildbot work with them?

This is the impression I got from our emails and the mobile testing meeting.
We can test, but don't expect things to work.

1) SUTAgent dies quite early on
2) adb dies (I have tried rebooting, etc.. on my host with no luck)
3) An idle machine loses the sutagent within an hour, usually much faster
4) sutagent can take up to 2 minutes to get launched

As long as we have the power cycle available we should be in a good position.
(Reporter)

Updated

6 years ago
Blocks: 725544
No longer blocks: 769429
Component: Release Engineering → General
Product: mozilla.org → Testing
Version: other → Trunk
(Reporter)

Updated

6 years ago
Blocks: 725845
I found that sutagent was dying out about 15-20 minutes after a reboot.  This was even without tests running.  This was due to the suspend feature in the android OS.  We can disable this with:
adb shell svc power stayon true 

This needs to be done everytime after a reboot.  We can write some code to do this inside the agent by either issuing the above command, or using the api:
http://grepcode.com/file/repository.grepcode.com/java/ext/com.google.android/android/2.2_r1.1/com/android/commands/svc/PowerCommand.java#PowerCommand.%3Cinit%3E%28%29

Unfortunately after doing this tests still cause fennec to crash and the panda to hang (no adb or sut access).
(Reporter)

Comment 7

6 years ago
(In reply to Joel Maher (:jmaher) from comment #6)
> I found that sutagent was dying out about 15-20 minutes after a reboot. 
> This was even without tests running.  This was due to the suspend feature in
> the android OS.  We can disable this with:
> adb shell svc power stayon true 
> 
> This needs to be done everytime after a reboot.  We can write some code to
> do this inside the agent by either issuing the above command, or using the
> api:
> http://grepcode.com/file/repository.grepcode.com/java/ext/com.google.android/
> android/2.2_r1.1/com/android/commands/svc/PowerCommand.java#PowerCommand.
> %3Cinit%3E%28%29
> 
> Unfortunately after doing this tests still cause fennec to crash and the
> panda to hang (no adb or sut access).

Is this something that can be fixed on the SUT agent? or through the harness?
It is, but we need to make sure it works.  I have a bunch of log files that show it doesn't work all the time.  Once we figure out the appropriate commands to execute on the device we can hardcode those into the agent.
(Reporter)

Comment 9

6 years ago
This bug should not hard-block the work on bug 725845.
This bug is important for a staging setup to run consistently.
No longer blocks: 725845
(Reporter)

Updated

6 years ago
Summary: Provide staging usable image for panda boards → (a-team) Panda boards image work
(Reporter)

Updated

6 years ago
Depends on: 763497
(Reporter)

Updated

6 years ago
Depends on: 772967
(Reporter)

Comment 10

6 years ago
Perhaps bug 777500 is *general* rather than *panda* specific but unless we were to have foopy-less setup I would assume that we could have similar reboot issues until the root cause is figured out.
I'm mainly adding it to keep track of it.
Depends on: 777500
Blocks: 778733
No longer blocks: 725544
Depends on: 779853
Created attachment 648345 [details]
adb logcat showing initial startup of fennec with a 4 minutes siesta
Blocks: 780233
Depends on: 779949
I have an updated panda image on my people account:
people.mozilla.org/~jmaher/panda_rc1.img.bz2

Please note this is a 16GB image (I don't have an 8GB card).

I have verified with this image that all mochitests, reftests, crashtests and jsreftests can be run.  

I call this RC1 as I would like to test with the full talos suite as well as robocop.  Also I would like to ensure we have a burnin script ready and tested with this image.  Lastly a full cleanup pass on watcher and sutagent would be nice.
(In reply to Joel Maher (:jmaher) from comment #12)
> I have an updated panda image on my people account:
> people.mozilla.org/~jmaher/panda_rc1.img.bz2
> 
> Please note this is a 16GB image (I don't have an 8GB card).
> 
> I have verified with this image that all mochitests, reftests, crashtests
> and jsreftests can be run.  
> 
> I call this RC1 as I would like to test with the full talos suite as well as
> robocop.  Also I would like to ensure we have a burnin script ready and
> tested with this image.  Lastly a full cleanup pass on watcher and sutagent
> would be nice.

Quick status on whats left to do before we give IT the formal "ok to start imaging lots of machines"?
[on behalf of joel, who can't get BMO to load right now]

We just need 
* "The burn-in script" which is being tested
* "Robocop"
Depends on: 789516
Here is what I am calling the final panda image:
people.mozilla.org/~jmaher/panda/panda_final_2gb.img.bz2

This has the latest watcher and sutagent with final changes and tested a couple times locally.  This version of watcher/sutagent are also on most of the pandas in the rack that I tested, so it has been tested quite a bit.

The smoketest was used to validate these boards and is tracked here:
https://bugzilla.mozilla.org/show_bug.cgi?id=781341

This smoketest includes 4 patches to sut_tools, 3 of which are being deployed today to the tegras and the official repository.  That leaves 1 patch for the pdu management which has some more definitions to resolve which Callek is driving to completion.  The final patch has some sut_tools adjustments to run on the pandas (not adjusting the resolution, etc...) as well as the smoketest.py script itself.
Status: NEW → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.