1324541 - Use memory instead of disk for SQLite temporary files

Status: RESOLVED DUPLICATE of bug 1313021

(Toolkit :: Storage, enhancement)

Description

[meta-coder]

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:50.0) Gecko/20100101 Firefox/50.0
Build ID: 20161209093955

Steps to reproduce:

I am trying to minimize disk I/O of Firefox in order to save disk drive from excessive wear and for better performance/responsiveness.
I keep my profile and user cache directory on tmpfs. 

On further observation of FF's disk activity I noticed that built-in SQLite database uses /var/tmp for its temporary files.

Start Firefox,
Observe disk activity
$ sudo strace -p <pid> -f  -e trace=file
...
You'll see temporary files being created and unlinked in /var/tmp/etilqs_*

On reading https://www.sqlite.org/tempfiles.html, it would seem that Firefox doesn't provide any preference for location of SQLite's temp files, and so SQLite uses /var/tmp. On my distro /var/tmp is on disk, thus causing disk spin up.

It would be very efficient if a memory backed directory is used for SQLite temporary files.

On Ubuntu /run/shm is tmpfs, so exporting SQLITE_TMPDIR=/run/shm env var works for me.

Comment 1

[Andrew Sutherland [:asuth] (he/him)]

Great minds think alike! (note, I'm referring to :mak, not me ;)

We conditionally use in-memory tables in cases where we believe we have the address-space to spare (64-bit builds), or simply must (android, no /tmp) as of bug 1313021 which landed for Firefox 52 (currently the aurora branch).

I think it probably makes sense to dupe this bug to that one, since:
- the fix would end up being 32-bit linux-specific, which is a very small set of users to support given that...
- our builds are multi-distro and picking the right temp dir seems non-trivial
- distro builds can override SQLITE_TMPDIR if they're so inclined

Comment 2

[meta-coder]

I wouldn't be surprised if many 32-bit Linux systems have more than 4GB of usable RAM, thanks to Linux's ability to use Physical Address Extension.

Other choices for memory backed temp dir are /run, /run/user/<uid>/. At least one of these must be in Linux Standard Base or some other standard.

Since you are aware about this enhancement, you may proceed as you see fit.

Comment 3

[Marco Bonardo [:mak] (Away Apr 25 - May 5)]

I also think we should just dupe, I completely agree with Andrew about the small population a fix here would hit, so the cost for us over weights the benefit.
If you care about this, you may use a 64 build once FF52 is out and you'll get a memory TEMP store.