1000718 - Add local storage to sccache

Status: RESOLVED FIXED

(Firefox Build System :: General, defect)

Comment 1

[Mike Hommey [:glandium]]

Attachment: Add basic local storage

This applied directly on top of https://github.com/glandium/sccache/

The one thing this doesn't do is do some cleanup when the cache is using a lot of diskspace. I'll leave that to a followup.

Comment 2

[Mike Hommey [:glandium]]

Attachment: Avoid requiring boto when not using S3 storage

Comment 3

[Michael Shal [:mshal]]

Comment on attachment 8411582 [details] [diff] [review]
Add basic local storage

>+        directory = os.environ.get('SCCACHE_DIR')

SCCACHE_DIR should also be described in a comment in from_environment()

In addition to the lack of cleanup from disk space, there may be performance concerns by putting all the objects into one directory. A simple one or two-level directory hierarchy like what ccache or .git/objects use would probably suffice.

Comment 4

[Mike Hommey [:glandium]]

(In reply to Michael Shal [:mshal] from comment #3)
> In addition to the lack of cleanup from disk space, there may be performance
> concerns by putting all the objects into one directory. A simple one or
> two-level directory hierarchy like what ccache or .git/objects use would
> probably suffice.

The key comes with a three-level hierarchy already. Ideally, the hierarchy thing would be handled on the storage end.

Comment 5

[Mike Hommey [:glandium]]

Attachment: Create hierarchy at the storage level instead of when creating the key

Comment 6

[Mike Hommey [:glandium]]

https://github.com/glandium/sccache/commit/986c863e9cc627c06e5738748c1255079ba5ab4e
https://github.com/glandium/sccache/commit/e401cb6a9564f841b1036a78950ea8504410c510
https://github.com/glandium/sccache/commit/27424e20e0358143660ee4c1331486d9dfa21033