Closed Bug 555892 Opened 10 years ago Closed 10 years ago

don't reclone all repositories every n900 test run

Categories

(Release Engineering :: General, defect, P3)

ARM
Maemo
defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: aki, Assigned: aki)

References

Details

(Whiteboard: [mobile][automation])

Attachments

(4 files, 3 obsolete files)

Our MV<->MPT pipe is at a premium currently.

We switched from downloading a tarball every run from MV->MV (n810s), to cloning tools,talos&maemkit every run from hg/cvs on the n900s.

I think my preferred solution is to have a clean checkout somewhere on device that we can update & export to a working area (so we only grab diffs per run); a second possible solution is to have a clean checkout on the master that we update&tar up nightly.
Summary: don't reclone all repositories every talos run → don't reclone all repositories every n900 talos run
Summary: don't reclone all repositories every n900 talos run → don't reclone all repositories every n900 test run
Blocks: mobile-pool
I don't know that these are optimizations we need to be making.

I am not sure how the cvs -z parameter works, but I am going to assume that it does something similar to a per-file gziping.  Given that the tarball only beats per-file gzip -9 by 2.3MB, I don't know if that is worth the time to implement.  I also don't know if cvs does some sort of tar operation before.

For unittests, doing the maemkit checkout has 112KB overhead vs a total of 32MB of browser+tests.  I really don't think that we should be trying to optimize that.

In the patch I am about to attach, I will remove the dependency on the tools repository for our talos runs as we only need one file from the repository.



jhford-wifi:~/mozilla/new-n810-image $ du -sc talos/ talos-no-CVS-dirs/ talos.tar.gz talos.tar.bz2
16956   talos/
16460   talos-no-CVS-dirs/
4628    talos.tar.gz
3896    talos.tar.bz2

jhford-wifi:~/mozilla/new-n810-image/talos $ find -type f -exec gzip -9 {} \;
jhford-wifi:~/mozilla/new-n810-image/talos $ du -hsc
6228    .
6228    total

jhford-wifi:~/mozilla/new-n810-image/maemkit $ du -sc
188     .
188     total
jhford-wifi:~/mozilla/new-n810-image/maemkit $ rm -rf .hg
jhford-wifi:~/mozilla/new-n810-image/maemkit $ du -sc
76      .
76      total
jhford-wifi:~/mozilla/new-n810-image/maemkit $
Attached patch buildbotcustom-fixes (obsolete) — Splinter Review
use the dns name and only download the one file we need.
Assignee: nobody → jhford
Status: NEW → ASSIGNED
Attachment #442175 - Flags: review?(aki)
(In reply to comment #1)
> I don't know that these are optimizations we need to be making.

Gah, i forgot that if we have it as a tarball on a machine in MV, we don't need to do any transfer.  Maybe the solution for talos is to have a CVS mirror in MV for *all* talos machines, not just mobile
should've included the -z9 flag as a freebie bandwidth usage reduction.
Attachment #442175 - Attachment is obsolete: true
Attachment #442178 - Flags: review?(aki)
Attachment #442175 - Flags: review?(aki)
sorry about the bugmail spam :S
Attachment #442178 - Attachment is obsolete: true
Attachment #442185 - Flags: review?(aki)
Attachment #442178 - Flags: review?(aki)
(In reply to comment #1)
> I don't know that these are optimizations we need to be making.

Have you *seen* all the redness from broken checkouts?
I think it is an optimization we desperately need to make.
(In reply to comment #6)
> (In reply to comment #1)
> > I don't know that these are optimizations we need to be making.
> 
> Have you *seen* all the redness from broken checkouts?
> I think it is an optimization we desperately need to make.

yes, i have.  those are influenced by at least:
1. using the MPT dns server on the n900s, will be fixed in next n900 image by pointing them to the MV mirror of the build dns server
2. using hardcoded ip for talos checkouts which is fixed in this patch
3. tools repository checkouts failing which is removed by this patch.

I'd like to fix these three things before we make a call on this.
Comment on attachment 442185 [details] [diff] [review]
buildbotcustom-fixes

Ok, worth a shot as a first step towards green.
Attachment #442185 - Flags: review?(aki) → review+
Whiteboard: [mobile][automation]
15 test runs died this morning because of this.
36 failures since comment 9 because of this. I have the feeling things have gotten worse in the past couple days.
44 failures since comment 10.
Lets investigate what is going wrong in bug 576335
Depends on: 576335
66 failures since comment 11.
151 failures over the long weekend.
77 since Wednesday.
59 since Friday, but I think pmm was wedged over the weekend. Kicked it earlier today.

I'm probably going to create tarballs to fix this.
41 since comment 16.
I really want this bug resolved; taking.
Assignee: jhford → aki
33 failures since the last update =\
Attachment #457131 - Attachment is obsolete: true
27 failures since the last comment.
Attachment #457172 - Flags: review?(jhford)
Names the pageloader/talos methods correctly, and adds maemkit tarball functionality for unit tests.

After this lands, our main vulnerability (besides the devices' overall fragility) will be downloading builds/tests from ftp.m.o and bc-proxy01. We'll be in the exact same boat as desktop tests there, though.

Downloaded&extracted the tarballs on smm2 just fine.
Attachment #457179 - Flags: review?(jhford)
Comment on attachment 457172 [details] [diff] [review]
start using tarballs -- configs

looks good
Attachment #457172 - Flags: review?(jhford) → review+
Comment on attachment 457179 [details] [diff] [review]
start using tarballs -- custom

looks good
Attachment #457179 - Flags: review?(jhford) → review+
Comment on attachment 457150 [details]
include maemkit in the tarball list

it would be good if the script could do the initial checkouts, but that's definitely a p5 enhancement for the future!
Attachment #457150 - Flags: review+
25 failures since comment 20 =\
There may still be some queued up either in mail or currently running.
However, I expect to see these green up (or at least go orange/red after the cloning steps would have happened) real soon.
Status: ASSIGNED → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
There were 2 failures last night (for tarball downloads) due to dns lookups.
Definitely looks better.
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.