1122589 - Don't use the puppetagain yum repository in building docker images

Status: RESOLVED FIXED

(Taskcluster :: General, defect)

Description

[Dustin J. Mitchell [:dustin] (he/him)]

http://puppetagain.pub.build.mozilla.org isn't even intended as a repository -- it's a place for external users of PuppetAgain to rsync /data from.  It's single-hosted and can go down without notice or care.  That made it a *horrible* choice for use *within* tasks.  For building docker images, it's just not a good idea.

http://mockbuild-repos.pub.build.mozilla.org/ is the replacement.

I should be able to take care of this fix.

Comment 1

[Dustin J. Mitchell [:dustin] (he/him)]

Attachment: no-puppetagain.patch

This is a start.  mockbuild-repos only has yum packages in it.

Honestly, these packages should probably all be moved to an s3 bucket of some sort.  If you're comfortable with installing them directly from URLs (rather than as repositories), then that bucket could really be pretty simple.  It's probably also worth including comments as to where the definition of the packages lives (in mozilla/build-puppet, for the most part).

I'd like to get some feedback here, and then I can go and do a fuller job of this for yum and apt.

Comment 2

[Jonas Finnemann Jensen (:jonasfj)]

Comment on attachment 8550373 [details] [diff] [review]
no-puppetagain.patch

> That made it a *horrible* choice for use *within* tasks. 
> For building docker images, it's just not a good idea
Just to clarify, unless the packages installed somehow references the source from which they were installed (probably not the case), then PuppetAgain isn't used from *within* tasks.

Images are AFAIK built manually... That said, I don't see a reason not to use a more stable mirror - assuming it's the same packages. As you say, using the repo is "just not a good idea" :)

Note, I think :wander did the discovery on how to mirror relengs build environment, and, hence, why we're installing from there. But if it's the same packages I don't see any issues.

Comment 3

[Greg Arndt [:garndt]]

There was some discussion in #releng back when the research for this started and there were two URL's given (puppetagain and mockbuild-repos) as sources for the packages.  Appears that puppetagain was just the first one picked to use and I don't believe we understood that it was a mirror not meant to be relied upon.

Currently the images are manually built and deployed so this would only be an issue then, but still an issue nonetheless.  Especially so once we do smarter things for automatically building/deploying images.

Would be awesome for this to move to a more stable source.  Thanks for the help :dustin!

Comment 4

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8550373 [details] [diff] [review]
no-puppetagain.patch

On further thought, I'd like to land this as-is for the moment, because I don't want to try to build out a system for hosting yum and apt repos in s3 right now.

What's the process for landing on alder?  Just push?

Comment 5

[Jonas Finnemann Jensen (:jonasfj)]

> What's the process for landing on alder?  Just push?
Good question, maybe jlal can answer?

Comment 6

[James Lal [:lightsofapollo]]

yup just push

Comment 7

[Dustin J. Mitchell [:dustin] (he/him)]

   https://hg.mozilla.org/projects/alder/rev/253403427bb4

Comment 8

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/c3a90afa2dee

Comment 9

[Dustin J. Mitchell [:dustin] (he/him)]

Comment on attachment 8550373 [details] [diff] [review]
no-puppetagain.patch

..since we were talking about this yesterday

Comment 10

[James Lal [:lightsofapollo]]

Comment on attachment 8550373 [details] [diff] [review]
no-puppetagain.patch

Ah not sure if this was supposed to be a review or heads up but we are taking this (actually your improved approach with yum configs) in b2g-builder now.

Comment 11

[Pete Moore [:pmoore][:pete]]

Resetting Version and Target Milestone that accidentally got changed...