Closed Bug 1509943 Opened 4 years ago Closed 4 years ago

Use proxy in WSGI environment

Categories

(Developer Services :: Mercurial: hg.mozilla.org, enhancement)

enhancement
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: gps, Assigned: gps)

References

(Blocks 2 open bugs)

Details

Attachments

(6 files)

We need to use the HTTP proxy in the WSGI environment running hgweb so connections to S3 go through the proxy, since the hosts have no direct outbound connectivity.

We could set the HTTPS_PROXY environment variable, preferably by sourcing /etc/environment. We could also set things manually in Python code. Environment should be easier though.

One thing to watch out for is WSGI uses HTTP_* environment variables to communicate HTTP headers. So if we set the HTTP_PROXY environment variable, WSGI will think the client sent a "Proxy:" request header. I don't think we perform any non-https connections from the WSGI code. So we could forego setting it.
Assignee: nobody → gps
Status: NEW → ASSIGNED
We could have potentially done this with SetEnv in httpd.conf or by
sourcing /etc/environment in the httpd systemd unit. However, because
/etc/environment defines extra variables that are not relevant and
one of those - HTTP_PROXY - could confuse WSGI, we want to only
apply what we need.

Reading the Internets, apparently some WSGI servers strip environment
variables coming from the parent process. So setting a value in SetEnv
may not work.

In addition, each server may have its own proxy server. And teaching
Ansible to parse the /etc/environment file or to define per-server
proxies feels like a bit of work. Minimal Python code in the shared
bootstrap file gets the job done.
Blocks: 1118506
httpd.conf only references the various hgweb.wsgi files. The
mercurial.hgweb.hgweb() function creates a regular hgweb or hgwebdir
application depending on the active config.

This commit deletes the various hgwebdir.wsgi files because I'm almost
certain they are dead code.
These files import mercurial.* modules and thus need to be
GPL licensed.
The hgweb.wsgi files are all the same boilerplate.

This commit teaches all of those files to execute a shared
bootstrap.py file. This will allow us to aggregate common code
into every file without having to update every file. The
added bootstrap.py is empty: this commit simply introduces the
ability to execute common code without doing anything meaningful.
There's no need to do this separately in every hgweb.wsgi file.
Previously, every hgweb.wsgi file had boilerplate for creating the
hgweb wsgi application. This commit moves that boilerplate into
a function in bootstrap.py.

A nice benefit of the change is that we no longer need to hardcode
absolute paths to hgweb.config files into each hgweb.wsgi file.

Every hgweb.wsgi file now contains almost the exact lints of
boilerplate. The only difference is how many parent paths there are
to bootstrap.py.
Pushed by gszorc@mozilla.com:
https://hg.mozilla.org/hgcustom/version-control-tools/rev/521a127d57fd
hgwsgi: remove hgwebdir.wsgi files ; r=sheehan
https://hg.mozilla.org/hgcustom/version-control-tools/rev/9df0a1309c39
hgwsgi: add license header to WSGI files ; r=sheehan
https://hg.mozilla.org/hgcustom/version-control-tools/rev/6677153d26d5
hgwsgi: execute bootstrap file from every .wsgi file ; r=sheehan
https://hg.mozilla.org/hgcustom/version-control-tools/rev/cb49c7290eee
hgwsgi: set HGENCODING from bootstrap.py ; r=sheehan
https://hg.mozilla.org/hgcustom/version-control-tools/rev/202e07254d44
hgwsgi: use function for creating wsgi application ; r=sheehan
https://hg.mozilla.org/hgcustom/version-control-tools/rev/bb40022dc281
hgwsgi: set HTTPS_PROXY from /etc/environment ; r=sheehan
Status: ASSIGNED → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
Blocks: 1507221
Blocks: 1511241
You need to log in before you can comment on or make changes to this bug.