1990607 - Explicitly close old DB connections in worker threads before making DB calls

Status: RESOLVED FIXED

(Tecken :: Upload, task, P2)

Description

[Bianca Danforth [:bdanforth]]

As a follow-up to Bug 1990061.

Django’s CONN_MAX_AGE and CONN_HEALTH_CHECKS only apply to DB connections in the request thread. Since we’re using separate worker threads to upload files in parallel for each upload request, and since each thread maintains their own DB connection, we need to explicitly guard the connection at task boundaries by calling close_old_connections() before doing DB work in the thread.

from django.db import close_old_connections

close_old_connections()

Comment 1

[Sven Marnach :sven]

I had a quick look at the Django source code. I think enabling health checks is all we need. Every change made to the database requests a cursor from database connection object, and enabling health checks makes sure the connection is checked before returning the cursor. This happens regardless of whether the connection is held by the main thread or any other thread.

Comment 2

[Bianca Danforth [:bdanforth]]

Attachment: [mozilla-services/tecken] bug-1990607: enable CONN_HEALTH_CHECKS for Django DB (#3235)

Comment 3

[Bianca Danforth [:bdanforth]]

biancadanforth merged PR [mozilla-services/tecken]: bug-1990607: enable CONN_HEALTH_CHECKS for Django DB (#3235) in f51dd75.

Comment 4

[Bianca Danforth [:bdanforth]]

I've verified this in stage. Given that it's Friday afternoon in my timezone, and I'm out on Monday, I'll plan to deploy this to production when I return on Tuesday, 2025-09-30. I'll add a comment here once it's deployed and (hopefully) verified at that time.

Comment 5

[Bianca Danforth [:bdanforth]]

This was deployed in v2025.09.30. I've verified it in prod, so I'm going to close this out.