385470 - Thousands of "etilqs_******" files created in the TEMP folder

Status: RESOLVED FIXED

(Core :: SQLite and Embedded Database Bindings, defect)

Description

[Volkmar Kostka]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9a6pre) Gecko/20070621 Minefield/3.0a6pre
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9a6pre) Gecko/20070621 Minefield/3.0a6pre

Since updating the sqlite database thousands of "etilqs_******" files are created in the TEMP folder. All of them are empty or have small sizes.

Reproducible: Always

Steps to Reproduce:
Just look in the TEMP folder and browse a bit.
Actual Results:  
Empty and small files are created.

Expected Results:  
No files should be created or at least removed after use.

http://forums.mozillazine.org/viewtopic.php?t=560452

Comment 1

[:Gavin Sharp [email: [e-mail]]]

Sounds like this might be fallout from the sqlite upgrade?

Comment 2

[gabe]

seems to be 3.3.8 was version of sqlite that changed the files name of temp files to make it backwords 

anyway all of those files are 0 bytes we should not make those files

Comment 3

[Shawn Wilsher :sdwilsh]

we don't - sqlite does.  Suggesting RESOLVED->WONTFIX

Comment 4

[Volkmar Kostka]

WONTFIX is not an option. If sqlite does it either backout the new version or forward it to the producer to fix it. On one of my systems i removed over 8000 files only for one day. If this goes into the release some people will find millions of those files after a while.

Comment 5

[gabe]

http://www.sqlite.org/pragma.html

talks about temp files

Comment 6

[Marco Bonardo [:mak]]

probably even if they are not deleted (but they should be) it could be better to rename them to avoid confusion with other apps sqlite temp files 

they can be renamed with
-DTEMP_FILE_PREFIX=something

Probably could be bettet to use something like
-DTEMP_FILE_PREFIX=mozStorage_

mcafee has the same "problem" but their files go away on closing app, they are also renaming them using TEMP_FILE_PREFIX

Comment 7

[gabe]

seems to me this may be the most usefull thing ive found on the site above so far

PRAGMA temp_store;
PRAGMA temp_store = DEFAULT; (0)
PRAGMA temp_store = FILE; (1)
PRAGMA temp_store = MEMORY; (2)

Query or change the setting of the "temp_store" parameter. When temp_store is DEFAULT (0), the compile-time C preprocessor macro TEMP_STORE is used to determine where temporary tables and indices are stored. When temp_store is MEMORY (2) temporary tables and indices are kept in memory. When temp_store is FILE (1) temporary tables and indices are stored in a file. The temp_store_directory pragma can be used to specify the directory containing this file. FILE is specified. When the temp_store setting is changed, all existing temporary tables, indices, triggers, and views are immediately deleted.

It is possible for the library compile-time C preprocessor symbol TEMP_STORE to override this pragma setting. The following table summarizes the interaction of the TEMP_STORE preprocessor macro and the temp_store pragma:

    TEMP_STORE 	PRAGMA
    temp_store 	Storage used for
    TEMP tables and indices
    0 	any 	file
    1 	0 	file
    1 	1 	file
    1 	2 	memory
    2 	0 	memory
    2 	1 	file
    2 	2 	memory
    3 	any 	memory

Comment 8

[Noah (NoahSUMO/SolidSnake) [:Noah]]

sdwilsh mentioned in a Sqlite thread that new performance slowdowns w/ Places like in the Bookmarks Manager are most likely due to bug 385066 being turned off.

Now this could just be fallout from the upgrade like gavin was saying. And gabe looks like he's definitely on the money w/ what he found. But in case that doesn't fix the infinite temp file problem, could this be a side effect of bug 385066 being turned off? Since it was disabled to complete the upgrade to the new Sqlite version.

Maybe someone could test w/ it re-enabled? Unless placing that code in the old area is not an option anymore due to the upgrade changing things.

Comment 9

[Shawn Wilsher :sdwilsh]

Not a side effect of Bug 385066.  This may be fixed by my checking of Bug 385396.  Please check a build after that cycles and let me know.

Comment 10

[Shawn Wilsher :sdwilsh]

As per Bug 385396 Comment 5.  Should be all better now :)  Sorry about that!

Comment 11

[Marco Bonardo [:mak]]

What about changing the names of Tempo files to mozStorage_#######  as per comment #6?

So temp files are not confused between different apps that use sqlite and could be easy to find problems about them

Comment 12

[Volkmar Kostka]

Yep. No more files. Thanks. :-)

Comment 13

[Shawn Wilsher :sdwilsh]

(In reply to comment #11)
> What about changing the names of Tempo files to mozStorage_#######  as per
> comment #6?
Because NSS will be using this same copy of sqlite, it doesn't make sense.

Comment 14

[Marco Bonardo [:mak]]

But if this (mozStorage) is a toolkit part every toolikit app will use it, and it could makes sense to distinguish mozStorage temp files from other (mcaffee and other vendors)... 
it's not a Firefox only component, so mozStorage_xxxx is enough generic name, isn't it?

Comment 15

[Shawn Wilsher :sdwilsh]

NSS isn't using mozStroage, they will be linking to the sqlite library.