Reusing connections to improve performance

UNCONFIRMED
Unassigned

Status

Testing
mozregression
P5
minor
UNCONFIRMED
2 years ago
2 years ago

People

(Reporter: YF (Yang), Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

2 years ago
I see repeated connections open and closed for fetch info from queue.taskcluster.net:443 and index.taskcluster.net:443 (and maybe more) by an proxy software, they transfer a small amount of information (every ~5KB), but open a TLS connection requires a complete synchronization delay for it.

At least, I see the packs download connections is persistent.
(In reply to YF (Yang) from comment #0)
> I see repeated connections open and closed for fetch info from
> queue.taskcluster.net:443 and index.taskcluster.net:443 (and maybe more) by
> an proxy software, they transfer a small amount of information (every ~5KB),
> but open a TLS connection requires a complete synchronization delay for it.
> 
> At least, I see the packs download connections is persistent.


Those requests are done by the taskcluster-client.py module, that we use - other requests in mozregression are using a single http session (through requests.Session) and I believe they don't suffer this problem.

I don't see in taskcluster-client any way to specify a custom session, and as far as I can see there is no session concept in taskcluster-client, and there is no way for an user of the library to change that:

https://github.com/parkouss/taskcluster-client.py/blob/master/taskcluster/utils.py#L187

Jonas, I believe this can be fixed in taskcluster-client.py ? We would need to be able to allow to pass a custom requests.Session to use in Queue and Index, or at least it should be able to create a persistent connection (i.e., requests.Session again) for those - so multiple calls can benefits from it.

Thoughts ?
Flags: needinfo?(jopsen)
> Thoughts ?

Agree, we definitely should do this.. I defer to jhford who owns the taskcluster-client.py
This is a pretty serious bug, as one quickly runs out of available ports...
Flags: needinfo?(jopsen) → needinfo?(jhford)
Depends on: 1247728
https://github.com/taskcluster/taskcluster-client.py/pull/40
Flags: needinfo?(jhford)
(Reporter)

Comment 4

2 years ago
I still see some short-lived connections to index.taskcluster.net:443, queue.taskcluster.net:443, archive.mozilla.org:443 and others host in mozregression-gui 0.9.1 mozregression 2.3.1. I guess they should be persisted.
You need to log in before you can comment on or make changes to this bug.