538517 - talos and unit tests on N900s

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Aki Sasaki (not active)]

* asset tag
* static ips
* image
* graphs sql
* connect to buildbot
* vet in staging
* move to production

John Ford and I will both be working on this, but I'm making John point.

Comment 1

[Aki Sasaki (not active)]

We can put these into staging before bug 528431 is done (maemo5 builds) if needed, since we can test on Maemo4 builds.

Comment 2

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

We expect, from previous timings on n810s, and predicted load, that we will need 100-150 n900s to keep up with the load. However, from talking with Stuart and Justin, the plan is to bring this initial batch of 20 online, and then have Stuart decide if the gaps in coverage merit buying more phones. Thats being tracked in bug#538516. If more phones are asked for, we will file new separate bugs.

Morphing this bug to be clearly just about taking the (20) n900s we currently have into production, and reporting results on tinderbox server+graph server. Once these 20 are up, reporting results for all the relevent branches, we can close this.

Comment 5

[John Ford [:jhford] CET/CEST Berlin Time]

Status Update:
* asset tag - DONE - bug 546836
* static ips - DONE - bug 546836
* image - DONE - bug 538510
* graphs sql - DONE - bug 548150 bug 545754
* connect to buildbot - DONE - staging-mobile-master.b.m.o/buildslaves

* vet in staging
I need to create a new talos + pageloader tarball to test this, or else modify the factories to use CVS.  I am going to use the tarballs temporarily
* move to production

Comment 7

[John Ford [:jhford] CET/CEST Berlin Time]

I have tests running and reporting to:
http://tinderbox.mozilla.org/showbuilds.cgi?tree=MobileTest

I currently have only Mozilla Central enabled while I test.

Comment 8

[Aki Sasaki (not active)]

This is looking good to me -- feel free to submit your patches for review when you're ready.

Comment 9

[John Ford [:jhford] CET/CEST Berlin Time]

Attachment: [not fully tested] configs changes

this is the code that I am currently trying to test.  I have hit a bunch of road blocks in my testing, otherwise this would be r?

Comment 10

[John Ford [:jhford] CET/CEST Berlin Time]

Attachment: [not fully tested] custom changes

the corresponding buildbot-custom patch.

Comment 11

[John Ford [:jhford] CET/CEST Berlin Time]

Comment on attachment 430450 [details] [diff] [review]
[not fully tested] configs changes 

mobile-rw-staging is a temporary directory as we want to maintain two seperate copies until we have an n810 image that is able to work with the new automation

Comment 12

[John Ford [:jhford] CET/CEST Berlin Time]

Attachment: [not fully tested] configs changes

cut the patch incorrectly

Comment 13

[John Ford [:jhford] CET/CEST Berlin Time]

Attachment: [not fully tested] custom changes

cut the patch incorrectly

Comment 14

[Aki Sasaki (not active)]

Comment on attachment 430454 [details] [diff] [review]
[not fully tested] configs changes

Cool, I'll dig into this tomorrow.

Comment 15

[John Ford [:jhford] CET/CEST Berlin Time]

there might (will) be some testing garbage in these files, but the general structure looks pretty much what I am aiming for in the final result.

Not reflected in my patch is that when i submit the final patch, i am going to put the factories in a new python module as i realised that we need to keep the old factories around for a little while.

I currently derive the unit test factory from the talos one.  I don't have strong opinions that we should keep that inheritance.  We could have a common base class that has the minimum amount of overlap possible with no-op functions for the setup and run steps but I don't know if the extra testing is worthwhile as the only redundancy is a few strings that aren't used.

Comment 16

[Aki Sasaki (not active)]

Comment on attachment 430454 [details] [diff] [review]
[not fully tested] configs changes

For the most part, looks good.

A non-important question: rw? read-write?

My main concerns really have nothing to do with this patch and much more to do with incoming patches.

I'm soon going to have major changes ready to land for the winmo factories/configs; we also will need to be able to enable/disable nightly- and per-checkin- level polling on various branches.

I don't think any of that will be especially difficult, but we will be bitrotting each others' patches, so I'm flagging that now.

Comment 17

[John Ford [:jhford] CET/CEST Berlin Time]

what is the inheritance chain for the new winmo factories?

The configs shouldn't be too difficult to adapt if they are structured similarly to the n810 configs.

rw=reworking

Comment 19

[Aki Sasaki (not active)]

Comment on attachment 430455 [details] [diff] [review]
[not fully tested] custom changes

Ok.  Not going over this with a fine tooth comb; just touching on what jumps out at me.  The proof is in the working factories.

Pet peeve: tons of trailing whitespace. Please remove.

These lambdas are grokkable, though they'll definitely slow me down if I need to change things.

I don't like the hardcoded IP there but assume you have your reasons.

I pretty much disagree with your indentation, since this doesn't make sense at all to me:

self.addStep(Step(
        # why am i starting things here
        # and closing the parens here
        ))

Since I seem to be nitpicking relatively small details, I'm going to say, overall great work!

Comment 20

[Aki Sasaki (not active)]

(In reply to comment #17)
> what is the inheritance chain for the new winmo factories?

WinmoTalosFactory inherits MobileTalosFactory.  When do you think you'll be able to submit working n900 patches for r?  I think those should land first, assuming you're ready; I can un-bitrot my winmo stuff afterwards.

> The configs shouldn't be too difficult to adapt if they are structured
> similarly to the n810 configs.

Ok.
Pretty sure you said you were going to have a different module or something, which will allow us to have beta n900/winmo stuff running in parallel with existing n810 stuff until we have the chance to merge all of it.

Comment 21

[John Ford [:jhford] CET/CEST Berlin Time]

(In reply to comment #20)
> WinmoTalosFactory inherits MobileTalosFactory.  When do you think you'll be
> able to submit working n900 patches for r?  I think those should land first,
> assuming you're ready; I can un-bitrot my winmo stuff afterwards.
I am going to tidy up the patch on the train ride home tonight.  I have one issue left to deal with.

> Ok.
> Pretty sure you said you were going to have a different module or something,
> which will allow us to have beta n900/winmo stuff running in parallel with
> existing n810 stuff until we have the chance to merge all of it.
Yes, I am going to have a separate file for both the buildbotcustom and buildbot-config additions.  The current configs for the n810 will not be touched at all.

Comment 22

[John Ford [:jhford] CET/CEST Berlin Time]

Attachment: new factories

here are the new buildbot factories.  Unit test factories aren't working 100% but I will fix that in a follow up patch.  Leaving broken step commented out of factory.

Comment 23

[John Ford [:jhford] CET/CEST Berlin Time]

Attachment: n900 config patch

unit tests are disabled until i figure out what is up with them.  that will be a follow up patch

Comment 24

[John Ford [:jhford] CET/CEST Berlin Time]

Attachment: n900 factories

this is the same patch(as earlier today), other than a minor typo in the mobileunittest factory

Comment 25

[Aki Sasaki (not active)]

Comment on attachment 431393 [details] [diff] [review]
n900 config patch

r=me with a ton of whitespace nits... checked in:

http://hg.mozilla.org/build/buildbot-configs/rev/ac35bd61092b

There may be more changes needed for winmo+n810 -- we may want to master_main + master[1-3].cfg it since there really isn't any cross-polling or anything that's shared.

This is a good first step though, and awesome to see the n900 work going in.

Comment 26

[Aki Sasaki (not active)]

Comment on attachment 431394 [details] [diff] [review]
n900 factories

Again r=me with a ton of whitespace nits.

http://hg.mozilla.org/build/buildbotcustom/rev/87efd4252d9d

I'm a bit worried about whether a hardcoded value of 7 will break the n810s, but since we're temporarily forked that's not a blocker. We can test the n810s on this and tweak as necessary.

Comment 27

[John Ford [:jhford] CET/CEST Berlin Time]

Attachment: changes to match what is in staging

this is what is working in staging

Comment 28

[John Ford [:jhford] CET/CEST Berlin Time]

Attachment: n900 imaging tools update (for real)

Comment 29

[Aki Sasaki (not active)]

Comment on attachment 431513 [details] [diff] [review]
n900 imaging tools update (for real)

*stamp*

Comment 30

[John Ford [:jhford] CET/CEST Berlin Time]

Comment on attachment 431513 [details] [diff] [review]
n900 imaging tools update (for real)

http://hg.mozilla.org/build/tools/rev/7dee7e34f2e6

Comment 31

[John Ford [:jhford] CET/CEST Berlin Time]

* asset tag
* static ips
* image
* graphs sql
* connect to buildbot
* vet in staging
all done

* move to production 
being tracked in bug 551504

Comment 32

[Aki Sasaki (not active)]

Attachment: smm->pmm

Comment 33

[John Ford [:jhford] CET/CEST Berlin Time]

Comment on attachment 433018 [details] [diff] [review]
smm->pmm

this was landed, with an irc r+ a while ago http://hg.mozilla.org/build/buildbot-configs/rev/f0db7a70f0eb

Comment 34

[Aki Sasaki (not active)]

Attachment: add n900-21 through 50 to graphs

Comment 35

[alice nodelman [:alice] [:anode]]

Comment on attachment 433651 [details] [diff] [review]
add n900-21 through 50 to graphs

stamp

Comment 36

[Aki Sasaki (not active)]

Attachment: fix up n900s (cpu -> NULL, isactive->1)

Comment 37

[Aki Sasaki (not active)]

Attachment: same thing, but cpu speed back to 0.5ghz

Comment 38

[alice nodelman [:alice] [:anode]]

Comment on attachment 433655 [details] [diff] [review]
same thing, but cpu speed back to 0.5ghz

stamp stamp