835099 - Avoid fragmenting cache files on MacOSX

Status: RESOLVED WONTFIX

(Core :: Networking: Cache, defect)

Description

[Michal Novotny [:michal]]

Bug #816642 didn't fix the issue on MacOSX because of Tp5 regression.

Comment 1

[Michal Novotny [:michal]]

Attachment: touch each block of preallocated space instead of calling fcntl(..., F_PREALLOCATE, ...)

I've compared tpn results on MacOSX for

1) no fallocate - https://tbpl.mozilla.org/?tree=Try&rev=efda50ae0f0e
2) mozilla::fallocate as defined for XP_MACOSX - https://tbpl.mozilla.org/?tree=Try&rev=c46e99c72f29
3) mozilla::fallocate as defined for XP_UNIX - https://tbpl.mozilla.org/?tree=Try&rev=f3eea364832e

Average tp5n_shutdown_paint of (1) and (3) are the same. There is 65-90% increase in case of (2).


I've also tried to skip contiguous allocation in mozilla::fallocate:

1) no fallocate - https://tbpl.mozilla.org/?tree=Try&rev=1b67d6ad7de5
2) unchanged mozilla::fallocate - https://tbpl.mozilla.org/?tree=Try&rev=3c553e2e2e89
3) only non-continuous allocation in mozilla::fallocate - https://tbpl.mozilla.org/?tree=Try&rev=62f231b66ff5

The increase of tp5n_shutdown_paint is smaller by 10-15% in case of (3) when compared to (2) but it is still a huge regression.

Comment 2

[(dormant account)]

I think the problem may be that we are allocating space twice. 
Can you do another try run with only
-  int fd = PR_FileDesc2NativeHandle(aFD);
-  fstore_t store = {F_ALLOCATECONTIG, F_PEOFPOSMODE, 0, aLength};
-  // Try to get a continous chunk of disk space
-  int ret = fcntl(fd, F_PREALLOCATE, &store);



I don't trust the XP_UNIX branch on osx. It's still likely to allocate poorly, so i'd rather not do fallocate if that's our only option on osx.

Comment 3

[(dormant account)]

Comment on attachment 710615 [details] [diff] [review]
touch each block of preallocated space instead of calling fcntl(..., F_PREALLOCATE, ...)

removing review until another try run gives us more data

Comment 4

[Michal Novotny [:michal]]

no fallocate:     https://tbpl.mozilla.org/?tree=Try&rev=49bea70adbb2
F_ALLOCATEALL:    https://tbpl.mozilla.org/?tree=Try&rev=65a4c4dce228
F_ALLOCATECONTIG: https://tbpl.mozilla.org/?tree=Try&rev=07ff1210bd6c


Average tp5n_shutdown_paint values are:

             OSX 10.6  10.7  10.8
no fallocate      418   345   241
F_ALLOCATEALL     728   647   420
F_ALLOCATECONTIG  752   701   400

Comment 5

[(dormant account)]

Thanks, looks like we can't use osx fallocate stuff as long as we block on it. This is too bad since osx is worst at this.

Comment 6

[(dormant account)]

(In reply to Taras Glek (:taras) from comment #5)
> Thanks, looks like we can't use osx fallocate stuff as long as we block on
> it. This is too bad since osx is worst at this.

Michal, I feel that unless we do a telemetry field trial via telemetry to see if cache performance is better with XP_UNIX fallback, we should keep not using fallocate of any sort on mac. When I did my testing the F_ALLOCATE* stuff was much more likely to lay stuff out usefully on disk.

I think time would be better spent on getting rid of blockfiles altogether. So lets WONTFIX this bug.