843568 - Please do not use http://puppetagain.pub.build.mozilla.org for high-reliability Python installs

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Dustin J. Mitchell [:dustin] (he/him)]

This vhost is external, and single-hosted.  It's intended as an means to expose puppetagain' data to external users via rsync and http.  If it goes down, meh, external users can't see the data for a while.  No great emergency, no trees closed.

In PuppetAgain in general, resiliency is client-side, meaning that clients using data on puppet masters should be prepared to search for a working puppet master.  It's fine to use this host for stuff that can break for a few hours, but not for things that will close the tree.

On PuppetAgain-managed systems, ~/.pip/pip.conf is configured to point to all of the mirrors of this data on all of the puppetagain masters.  This is the correct way to install Python packages reliably.

For the duration of our not having everything managed by PuppetAgain, it'd be best to replicate this config some other way (old-puppet? mozharness?) and use it for the Python installs.

http://mxr.mozilla.org/build/search?string=puppetagain.pub.build.mozilla.org
suggests a bunch of places to fix this, but they all seem mozharness-related, so maybe that's not too bad.

Comment 1

[Armen [:armenzg]]

We now run unit tests officially through mozharness.

For instance, what should I run instead of this?
/builds/panda-0131/test/build/venv/bin/pip install --find-links http://puppetagain.pub.build.mozilla.org/data/python/packages mozpoolclient

Comment 3

[Dustin J. Mitchell [:dustin] (he/him)]

pip install mozpoolclient

should do the trick, assuming ~/.pip/pip.conf is in place.

http://hg.mozilla.org/build/puppet/file/63a5b3005adb/modules/python/templates/user-pip-conf.erb

Comment 4

[Armen [:armenzg]]

Hrmm... we should look at deploying that to the Windows slaves and verify that the foopies, linux and mac machines have it deployed.

Comment 5

[Aki Sasaki (not active)]

Bug 852473 comment 5 through 15 have some discussion about this bug.

Comment 6

[Aki Sasaki (not active)]

Attachment: puppetagain/data/ -> repos/

Comment 7

[Aki Sasaki (not active)]

Comment on attachment 727015 [details] [diff] [review]
puppetagain/data/ -> repos/

http://hg.mozilla.org/build/mozharness/rev/9d66f86b5b69

I'll push to Cedar and wait for some green before merging.

Comment 8

[Aki Sasaki (not active)]

Seeing green on Cedar.
Merged to production.

Comment 9

[Dustin J. Mitchell [:dustin] (he/him)]

This is better, but not resilient yet.

http://repos just points to the nearest puppet server, so it's no more fault-tolerant than pointing at that server itself, although it is a shorter path than puppetagain.pub.b.m.o.  

As I indicated in comment 3, the correct fix here is to use a pip.conf that lists multiple mirrors:
  http://hg.mozilla.org/build/puppet/file/tip/modules/python/templates/user-pip-conf.erb
and then pip install specific versions of packages (bug 851305).

Comment 10

[Aki Sasaki (not active)]

Ok.  I'm not working on comment 9.

Comment 11

[Dustin J. Mitchell [:dustin] (he/him)]

I think mozharness is consistently using pypi.pub/pvt.build.mozilla.org now, right?

Comment 12

[Aki Sasaki (not active)]

Afaict yes. http://mxr.mozilla.org/build/search?string=puppetagain&find=&findi=&filter=^[^\0]*%24&hitlimit=&tree=build