The default bug view has changed. See this FAQ.

[PATCH] Use sqlite FTS for faster text queries

RESOLVED FIXED

Status

Webtools
DXR
RESOLVED FIXED
5 years ago
5 years ago

People

(Reporter: Carlos Garnacho, Assigned: Carlos Garnacho)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments)

(Assignee)

Description

5 years ago
Created attachment 593125 [details] [diff] [review]
0001-dxr-index-Use-FTS4-sqlite-table-to-store-searchable-.patch

User Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.4+ (KHTML, like Gecko) Version/5.0 Safari/535.4+ Epiphany/3.2.1

Steps to reproduce:

I am attaching 2 patches to make DXR use sqlite's FTS4 for searches within the code, this table contains all plain text content from files, which replaces both files_index.txt and index_index.txt. 

FTS4 MATCH queries result in a much lower constant time per-query for text search, with overall times typically quite below files_index.txt searching. It also results in a lower disk usage on large projects as the tokenization payload is incidently lower than storing line numbers in ascii
(Assignee)

Comment 1

5 years ago
Created attachment 593126 [details] [diff] [review]
second patch, make search.cgi use FTS4
(Assignee)

Updated

5 years ago
Attachment #593126 - Flags: review?(ehsan)
(Assignee)

Updated

5 years ago
Attachment #593125 - Flags: review?(ehsan)
Comment on attachment 593125 [details] [diff] [review]
0001-dxr-index-Use-FTS4-sqlite-table-to-store-searchable-.patch

Nit: please remove the spaces before open parenthesis.  r=me with that!
Attachment #593125 - Attachment is patch: true
Attachment #593125 - Flags: review?(ehsan) → review+
Comment on attachment 593126 [details] [diff] [review]
second patch, make search.cgi use FTS4

Looks great!

Updated

5 years ago
Attachment #593126 - Flags: review?(ehsan) → review+

Comment 4

5 years ago
merged
Assignee: nobody → carlos
Status: UNCONFIRMED → RESOLVED
Last Resolved: 5 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.