Closed Bug 1277275 Opened 8 years ago Closed 8 years ago

Diagnose how long cache IO thread queues can be

Categories

(Core :: Networking: Cache, defect)

Product:
Core
Component:
Networking: Cache
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla50
Tracking Flags:
Tracking Status
firefox49 --- affected
firefox50 --- fixed

People

(Reporter: mayhemer, Assigned: mayhemer)

References

Details

(Whiteboard: [necko-active])

Attachments

(1 file, 1 obsolete file)

v1.1
9.02 KB, patch
mayhemer
: review+
Details | Diff | Splinter Review 
We have a small rate crash in mozilla::net::CacheIOThread::LoopOneLevel:

https://crash-stats.mozilla.com/search/?signature=~mozilla%3A%3Anet%3A%3ACacheIOThread%3A%3ALoopOneLevel&date=%3C%3D2016-06-01&_facets=signature&_columns=date&_columns=signature&_columns=product&_columns=version&_columns=build_id&_columns=platform#facet-signature

Which sometimes crashes on OOM.  Might be worth checking how long the queue can get.

I think a DIAGNOSTIC_ASSERT for 1000 operations could do as a start.  Or should it be an unpliftable telemetry?
Assignee: nobody → honzab.moz
Whiteboard: [necko-active]
Summary: Diagnose how long cache IO thread queues can be long → Diagnose how long cache IO thread queues can be
Status: NEW → ASSIGNED
Attached patch v1 (obsolete) — — Splinter Review 
- we accumulate at the moment we dequeue a runnable on the IO thread, we do so separately for each level
- there are 10 buckets for each IO thread level with 30 as a granularity
- when there are e.g. 65 pending runnables at the moment we dequeue, we accumulate bucket #2 (60) and remember to only report (accumulate) when the queue length is more then 90 (60 + 30) next time
- everything over 300 is reported in the last bucket
- some types clean up
Attachment #8761654 - Flags: review?(michal.novotny)
Attachment #8761654 - Flags: review?(michal.novotny) → review+
Keywords: checkin-needed
Pushed by cbook@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/5038112b5f78
Cache I/O queue length telemetry, r=michal
Keywords: checkin-needed
sorry had to back this out for bustage like https://treeherder.mozilla.org/logviewer.html#?job_id=30066432&repo=mozilla-inbound
Flags: needinfo?(honzab.moz)
Backout by cbook@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/977d6bed6e0a
Backed out changeset 5038112b5f78 for bustage
Attached patch v1.1 — — Splinter Review 
https://treeherder.mozilla.org/#/jobs?repo=try&revision=cee75e4f673b14f8a714dfa368e8a71a795bbefb

The two errors fixed, let's see.
Attachment #8761654 - Attachment is obsolete: true
Flags: needinfo?(honzab.moz)
Attachment #8765441 - Flags: review+
Green, let's do it!
Keywords: checkin-needed
Pushed by cbook@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/4070b1ce5332
Cache I/O queue length telemetry, r=michal
Keywords: checkin-needed
https://hg.mozilla.org/mozilla-central/rev/4070b1ce5332
Status: ASSIGNED → RESOLVED
Closed: 8 years ago
status-firefox50: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla50
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: