Closed Bug 1020554 Opened 7 years ago Closed 5 years ago
Mapper should yield content when returning full mapfiles
See https://bugzilla.mozilla.org/show_bug.cgi?id=847640#c30 for context. Basically, mapper aggregates the full mapfile in memory, retrieving content from the database, and only when it has created the full response content in-memory, it starts to stream it back to the requester. Instead, it should write back to the http response as it gets data from the database, so there is not a significant delay between making a request, and starting to receive content. This is important, since after 30s of inactivity, the load balancer will cut the connection and return an HTTP 500 response.
We should verify, but I think that the load balancers give up after 30 seconds of inactivity, so as long as data is flowing, things are fine. Flask does make streaming pretty easy, thankfully: http://flask.pocoo.org/docs/patterns/streaming/
However, the mapfile data is sorted, so streaming may not reduce time-to-first-byte.
(In reply to Hal Wine [:hwine] (use needinfo) from comment #2) > However, the mapfile data is sorted, so streaming may not reduce > time-to-first-byte. There is a delay from querying the DB until starting to receive rows back via the database connection, and there is a second delay from receiving the first data back from the database connection until the full mapfile is built up in memory. I think we can impact this second delay with streaming, but you are quite right Hal that the first part also incurs a delay. I'm not sure how much is the first part, and how much is the second part, at the moment.
Whiteboard: [kanban:engops:https://mozilla.kanbanize.com/ctrl_board/6/2736] → [kanban:engops:https://mozilla.kanbanize.com/ctrl_board/6/2743]
no one to work on this at the moment If anyone is seeing this in the real world, could they add details to help with prioritization.
Assignee: hwine → nobody
No need to touch this. No problems in over a year of use, and not a system we need to heavily invest in, especially with deprioritisation of B2G which was main user.
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.