Closed Bug 966412 Opened 8 years ago Closed 8 years ago

limit pvtbuilds uploads

Categories

(Release Engineering :: General, defect)

x86
macOS
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: aki, Assigned: aki)

References

Details

Attachments

(3 files, 2 obsolete files)

I think we can turn off all non-eng non-emulator dep uploads to pvtbuilds.
This is the #2 source of traffic from AWS through our VPC, and we *can't* send this over the internet, even via https, so limiting what we upload will help reduce our load.
Tony, Jonathan: Is comment 0 accurate?  Or do we have use cases for specific dep uploads?
Flags: needinfo?(tchung)
Flags: needinfo?(jgriffin)
Both this bug and bug 914111 will probably need us to spin up more nightlies.
Hi Aki, are you referring to stop uploading the full device production builds on pvtbuilds like:

 [ ]	hamachi.zip	31-Jan-2014 10:27 	86M	 
	leo.zip	        31-Jan-2014 13:57 	93M	 

etc.?   trying to understand what the question here.
Flags: needinfo?(tchung)
There won't be any impact on automation for turning off the non-eng uploads (we only use eng builds).
Flags: needinfo?(jgriffin)
(In reply to Tony Chung [:tchung] from comment #3)
> Hi Aki, are you referring to stop uploading the full device production
> builds on pvtbuilds like:
> 
>  [ ]	hamachi.zip	31-Jan-2014 10:27 	86M	 
> 	leo.zip	        31-Jan-2014 13:57 	93M	 
> 
> etc.?   trying to understand what the question here.

Essentially, anything in http://pvtbuilds.pvt.build.mozilla.org/pvt/mozilla.org/b2gotoro/tinderbox-builds/ that isn't -eng will not be uploaded on push anymore.

They'll be uploaded to http://pvtbuilds.pvt.build.mozilla.org/pvt/mozilla.org/b2gotoro/nightly/ if they have associated nightly builds.
(In reply to Aki Sasaki [:aki] from comment #5)
> (In reply to Tony Chung [:tchung] from comment #3)
> > Hi Aki, are you referring to stop uploading the full device production
> > builds on pvtbuilds like:
> > 
> >  [ ]	hamachi.zip	31-Jan-2014 10:27 	86M	 
> > 	leo.zip	        31-Jan-2014 13:57 	93M	 
> > 
> > etc.?   trying to understand what the question here.
> 
> Essentially, anything in
> http://pvtbuilds.pvt.build.mozilla.org/pvt/mozilla.org/b2gotoro/tinderbox-
> builds/ that isn't -eng will not be uploaded on push anymore.
> 
> They'll be uploaded to
> http://pvtbuilds.pvt.build.mozilla.org/pvt/mozilla.org/b2gotoro/nightly/ if
> they have associated nightly builds.

in that case, kill it.   Thanks.
This would be extremely helpful for reducing disk usage on ftp.m.o.
(In reply to Nick Thomas [:nthomas] from comment #7)
> This would be extremely helpful for reducing disk usage on ftp.m.o.

The latest patch in bug 914111 turns off public dep uploads anyway, so this specific bug should have no effect on ftp.m.o (but would reduce disk usage on pvtbuilds).
Attached patch limit-private-mozharness (obsolete) — Splinter Review
* updates the upload_dep_target_exclusions for in-use config files (excluding -eng and try)
* removes unused config files
Attachment #8369266 - Flags: review?(catlee)
er, with wasabi
Attachment #8369266 - Attachment is obsolete: true
Attachment #8369266 - Flags: review?(catlee)
Attachment #8369267 - Flags: review?(catlee)
Enable wasabi nightlies, since we won't have dep uploads.
Attachment #8369268 - Flags: review?(catlee)
Comment on attachment 8369267 [details] [diff] [review]
limit-private-mozharness

Review of attachment 8369267 [details] [diff] [review]:
-----------------------------------------------------------------

lesscode++
Attachment #8369267 - Flags: review?(catlee) → review+
Attachment #8369268 - Flags: review?(catlee) → review+
Landed https://hg.mozilla.org/webtools/tbpl/rev/6f2354446159 for wasabi nightlies on tbpl.
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Depends on: 968433
(In reply to Aki Sasaki [:aki] from comment #15)
> Landed https://hg.mozilla.org/webtools/tbpl/rev/6f2354446159 for wasabi
> nightlies on tbpl.

(In reply to Ryan VanderMeulen [:RyanVM UTC-5][Limited connectivity due to mass power outage] from comment #16)
> Follow-up typo fix: https://hg.mozilla.org/webtools/tbpl/rev/2e096737fbc1

In production :)
Looks like I need to s,nexus-4,mako, in mozharness.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Attached patch s,nexus-4,mako (obsolete) — Splinter Review
Just noticed that tinderbox-builds/b2g-inbound-mako/ etc were still being populated.
Attachment #8373574 - Flags: review?(catlee)
Attachment #8373574 - Flags: review?(catlee) → review+
Same, but enable hamachi depend uploads per bug 969542 comment 11.
Attachment #8373632 - Flags: review?(catlee)
Attachment #8373632 - Flags: review?(catlee) → review+
Transplanted to production.
Attachment #8373574 - Attachment is obsolete: true
Ok, I'm seeing hamachi depend builds uploading. and I'm seeing nexus-4 depend builds not uploading.
Status: REOPENED → RESOLVED
Closed: 8 years ago8 years ago
Resolution: --- → FIXED
Mozharness change live in production.
Component: General Automation → General
You need to log in before you can comment on or make changes to this bug.