996836 - HTTP cache v2: Consider merging WRITE, CLOSE, EVICT IO thread levels to only a single one

Status: RESOLVED FIXED

(Core :: Networking: Cache, defect)

Description

[Honza Bambas (:mayhemer)]

I am not sure why I originally have split WRITE and CLOSE.  If we have a lot of writes scheduled, this may block closing the files and our handles for a long time.  And waste resources (sys handles/buffers).

Then, to make the time we overgrow the cache disk consumption over the limit as short as possible, we should not delay evicting by lowering its priority to the absolute bottom but rather beside the writes.  Also may lower out-of-disk-space error rate on small partitions.

I suggest instead of

    OPEN_PRIORITY,
    READ_PRIORITY,
    OPEN,
    READ,
    MANAGEMENT,
    WRITE,
    CLOSE,
    INDEX,
    EVICT,

the following list:

    OPEN_PRIORITY,
    READ_PRIORITY,
    OPEN,
    READ,
    MANAGEMENT,
    WRITE,
    INDEX,

and post CLOSE and EVICT on WRITE.

Comment 1

[Honza Bambas (:mayhemer)]

Attachment: v1

Let's leave the CLOSE distinction for now.  We can cleanup later when this proves itself.

Will push to try later.

Comment 2

[Honza Bambas (:mayhemer)]

Note: leaving the EVICT level for now.  I don't think we should remove/merge it.

Comment 3

[Honza Bambas (:mayhemer)]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=727cd1b5e149

Comment 4

[Michal Novotny [:michal]]

Comment on attachment 8704296 [details] [diff] [review]
v1

Review of attachment 8704296 [details] [diff] [review]:
-----------------------------------------------------------------

r+ but I don't think this will have any effect. CLOSE priority is used only during shutdown and when releasing NSPR handle. ShutdownEvent is posted after CacheStorageService is shut down so it's unlikely that any other event will be posted to the IO thread. Delaying ReleaseNSPRHandleEvent isn't a problem since the handle would be released anyway when the total number of handles would exceed the limit while opening another file.

Comment 5

[Honza Bambas (:mayhemer)]

I still think it may have some positive effect.  Let's see.

Comment 6

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/1725b75a6604

Comment 7

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/1725b75a6604