Closed Bug 798125 Opened 12 years ago Closed 12 years ago

Buy 400 Panda ES boards

Categories

(Infrastructure & Operations :: RelOps: General, task)

x86
macOS
task
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: joduinn, Assigned: melissa)

References

Details

Please buy 400 of:

panda ES boards:
http://www.digikey.com/product-highlights/us/en/texas-instruments-pandaboard/686#tabs-2

(Note: there is no need to buy separate power supplies, as these will be provided as part of the chassis.)

SanDisk Extreme Pro 16gb SDSDXPA-016G-X46 SD cards X46:
http://www.amazon.com/SanDisk-Extreme-UHS-1-Memory-SDSDXPA-016G-X46/dp/B005LFT3MA/ref=sr_1_1?ie=UTF8&qid=1337007826&sr=8-1


For reference the last orders we did were: bug#754362, bug#769732, bug#725846 and bug#731694.



Note: per yesterday's planning meeting, these should not be delivered to SF or MV offices; these should be delivered to either SCL1 or SCL3 - :dividehex (cc'd) working to confirm delivery address which is also where assembly should happen.

Note: These boards are for testing nativeFennec on Android4.0 with these boards, hence marked as blocking bug#725544. Separate work continues for testing B2G with these boards, and a separate order will be placed once that eval is completed.
Assignee: desktop-support → moconnor
Component: Server Operations: Desktop Issues → Server Operations: RelEng
QA Contact: tfairfield → arich
We may just have 400 2GB SD cards in stock that we're trying to RMA, let me know if you guys can use them here.
urgh: just heard that we might want to get different SD cards after all. For now, please do not buy 400 SDcards. I'll meet ctalbert and sort this out. Stay tuned.
(In reply to Henry Langi [:hlangi] from comment #1)
> We may just have 400 2GB SD cards in stock that we're trying to RMA, let me
> know if you guys can use them here.

:hlangi, thanks for the offer, but 2GB is too low.... we're borderline between 8GB and 16GB.

:ctalbert just order one industrial board for eval. http://www.amazon.com/gp/product/B007DIS6TY/ref=ox_ya_os_product. Stay tuned.
Summary: Buy 400 Panda ES boards and 400 SD cards → Buy 400 Panda ES boards
The PO will need VP & CFO approval. 

What I will need from all of you to help this process is data and a plan, including a short executive summary (the narrative).  

The types of questions I have to answer are:
* Business Unit / Purpose
* Why do we need this?
* What are we purchasing?
* What is the risk of not purchasing it?
* What is the alternative?  How did we decide to go with this recommendation?

I've also learned that any capacity planning and forecasting helps (graphs).
I was only looking at the cost of the boards.  The whole system cost is over $140,000 and will also require CEO approval.
Additional info.

More info:
* Business Unit / Purpose: Release Engineering

* Why do we need this? 
We do a lot of builds and test for every checkin. More details at http://oduinn.com/blog/2012/08/21/137-hours-compute-hours-every-6-minutes/ and http://oduinn.com/blog/2012/09/04/infrastructure-load-for-august-2012/.  Until now, we've not been able to test Fennec on Android4.0. Now that we have a scalable process that works reliably , we need these boards to start providing Android4.0 coverage. 

* What are we purchasing?
400 panda boards, sdcards. Additionally, we need chassis, racks, to host these boards
Based on initial timings, we find these panda boards are comparable to existing tegra boards. We therefore estimate that 400 should be enough to handle current capacity, based on results from our ~290 tegra boards (which provide Android 2.x test coverage). Its hard to predict future checkin load, so we'll have a better idea of whether this is enough after we have these Panda running in production.

* What is the risk of not purchasing it? No Android 4.0 coverage.

* What is the alternative?  How did we decide to go with this recommendation?
Alternatives unknown. Earlier experiments w/beagle, tegras were unsuccessful. We've not yet found a board that can be used to test android4.0 at scale. Over the last months, this has been discussed at length with devs (damons, mfinkle, blassey) ateam (ctalbert), releasecoordinators (akeybl). This is the best option we know of.
Your Salesorder Number is: 34256094
Your Purchase Order Number is: mrz-Fennec-798125
Your Web ID is: 42574010
Your Access ID is: 75032
joduinn any updates on the SD card situation?
These folks move fast.  All 400 ship to Vantage tonight.  Finance has the credit app.  I'll wait until that is wrapped up before pushing for the other 400.

John owes an update on the SD cards.  Should have that today.
----- Shipment from Digi-Key on  9-OCT-2012  -----
Ship Method: UPS 3 day                      Tracking #: 1Z5674321207642131         
Track Package
http://wwwapps.ups.com/tracking/tracking.cgi?tracknum=1Z5674321207642131           
Ship Method: UPS 3 day                      Tracking #: 1Z5674321207642140         
Track Package
http://wwwapps.ups.com/tracking/tracking.cgi?tracknum=1Z5674321207642140           
Ship Method: UPS 3 day                      Tracking #: 1Z5674321207642168         
Track Package
http://wwwapps.ups.com/tracking/tracking.cgi?tracknum=1Z5674321207642168           
Ship Method: UPS 3 day                      Tracking #: 1Z5674321207642177         
Track Package
http://wwwapps.ups.com/tracking/tracking.cgi?tracknum=1Z5674321207642177           
Ship Method: UPS 3 day                      Tracking #: 1Z5674321207642186         
Track Package
http://wwwapps.ups.com/tracking/tracking.cgi?tracknum=1Z5674321207642186           
Ship Method: UPS 3 day                      Tracking #: 1Z5674321207642202         
Track Package
http://wwwapps.ups.com/tracking/tracking.cgi?tracknum=1Z5674321207642202
(In reply to matthew zeier [:mrz] from comment #9)
> These folks move fast.  All 400 ship to Vantage tonight.  Finance has the
> credit app.  I'll wait until that is wrapped up before pushing for the other
> 400.
> 
> John owes an update on the SD cards.  Should have that today.

(In reply to Melissa O'Connor [:melissa] from comment #8)
> joduinn any updates on the SD card situation?

ctalbert did get delivery of one "industrial SD card" today, but mid-afternoon today was still wrestling with weird boot/image problems. If he cant get this resolved soon (today or early tmrw), we'll abort and go with the sd cards we know *do* work, just to unblock things for now, and revisit these "industrial" SD cards later.
Blocks: 799922
(In reply to John O'Duinn [:joduinn] from comment #11)
> (In reply to matthew zeier [:mrz] from comment #9)
> > These folks move fast.  All 400 ship to Vantage tonight.  Finance has the
> > credit app.  I'll wait until that is wrapped up before pushing for the other
> > 400.
> > 
> > John owes an update on the SD cards.  Should have that today.
> 
> (In reply to Melissa O'Connor [:melissa] from comment #8)
> > joduinn any updates on the SD card situation?
> 
> ctalbert did get delivery of one "industrial SD card" today, but
> mid-afternoon today was still wrestling with weird boot/image problems. If
> he cant get this resolved soon (today or early tmrw), we'll abort and go
> with the sd cards we know *do* work, just to unblock things for now, and
> revisit these "industrial" SD cards later.

Followup: met w/ctalbert, moconnor just now. 

1) The industrial card is still not usable, so lets not block on this. Investigations will continue in bug#799303, and if we get these to work, we can switch over later.

2) For now, lets go ahead with buying 850 SDcards specified in comment#0, which we know work. We'll need at least 800 (for the two batches of 400 pandas), plus should probably get some spares. We've no fail rate data for the cards, so I'll swag 50 spares for now.
No longer blocks: 799922
Blocks: 799922
400 boards have been delivered to the assembly house.
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → FIXED
Component: Server Operations: RelEng → RelOps
Product: mozilla.org → Infrastructure & Operations
You need to log in before you can comment on or make changes to this bug.