Closed Bug 1252937 Opened 4 years ago Closed 4 years ago

Define SQLITE_ENABLE_FTS3_TOKENIZER for Thunderbird/SeaMonkey only for the gloda global database that uses a custom FTS3 tokenizer

Categories

(Toolkit :: Storage, defect, major)

defect
Not set
major

Tracking

()

RESOLVED FIXED
mozilla47
Tracking Status
firefox47 --- fixed

People

(Reporter: Paenglab, Assigned: aceman)

References

Details

(Keywords: dogfood, regression)

Attachments

(1 file, 1 obsolete file)

The errors I get:

Wed Mar 02 2016 19:08:28
Error: 2016-03-02 19:08:28	gloda.datastore	ERROR	Unexpected error when trying to open the database: [Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIFts3Tokenizer.registerTokenizer]"  nsresult: "0x80004005 (NS_ERROR_FAILURE)"  location: "JS frame :: resource:///modules/gloda/datastore.js :: gloda_ds_init :: line 1062"  data: no]

Source file: resource://app/modules/gloda/log4moz.js
Line: 691
 ----------
2016-03-02 19:08:28	gloda.datastore	ERROR	Unexpected error when trying to open the database: [Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIFts3Tokenizer.registerTokenizer]"  nsresult: "0x80004005 (NS_ERROR_FAILURE)"  location: "JS frame :: resource:///modules/gloda/datastore.js :: gloda_ds_init :: line 1062"  data: no]

 ----------
Wed Mar 02 2016 19:08:28
Error: 2016-03-02 19:08:28	gloda.everybody	ERROR	!!! error loading resource:///modules/gloda/fundattr.js

Source file: resource://app/modules/gloda/log4moz.js
Line: 691
 ----------
2016-03-02 19:08:28	gloda.everybody	ERROR	!!! error loading resource:///modules/gloda/fundattr.js

 ----------
Wed Mar 02 2016 19:08:28
Error: 2016-03-02 19:08:28	gloda.explattr	ERROR	Error in init: TypeError: this.asyncConnection is null

Source file: resource://app/modules/gloda/log4moz.js
Line: 691
 ----------
2016-03-02 19:08:28	gloda.explattr	ERROR	Error in init: TypeError: this.asyncConnection is null

 ----------
Wed Mar 02 2016 19:08:28
Error: 2016-03-02 19:08:28	gloda.everybody	ERROR	!!! error initializing resource:///modules/gloda/explattr.js

Source file: resource://app/modules/gloda/log4moz.js
Line: 691
 ----------
2016-03-02 19:08:28	gloda.everybody	ERROR	!!! error initializing resource:///modules/gloda/explattr.js

 ----------
Wed Mar 02 2016 19:08:28
Error: 2016-03-02 19:08:28	gloda.everybody	ERROR	!!! error loading resource:///modules/gloda/noun_mimetype.js

Source file: resource://app/modules/gloda/log4moz.js
Line: 691
 ----------
2016-03-02 19:08:28	gloda.everybody	ERROR	!!! error loading resource:///modules/gloda/noun_mimetype.js

 ----------
Wed Mar 02 2016 19:08:28
Error: 2016-03-02 19:08:28	gloda.everybody	ERROR	(resource:///modules/gloda/datastore.js:1418) TypeError: this.asyncConnection is null

Source file: resource://app/modules/gloda/log4moz.js
Line: 691
 ----------
2016-03-02 19:08:28	gloda.everybody	ERROR	(resource:///modules/gloda/datastore.js:1418) TypeError: this.asyncConnection is null

 ----------
Wed Mar 02 2016 19:08:28
Error: 2016-03-02 19:08:28	gloda.abattrs	ERROR	Error in init: TypeError: this.asyncConnection is null

Source file: resource://app/modules/gloda/log4moz.js
Line: 691
 ----------
2016-03-02 19:08:28	gloda.abattrs	ERROR	Error in init: TypeError: this.asyncConnection is null

 ----------
Wed Mar 02 2016 19:08:28
Error: 2016-03-02 19:08:28	gloda.everybody	ERROR	!!! error initializing resource:///modules/gloda/index_ab.js

Source file: resource://app/modules/gloda/log4moz.js
Line: 691
 ----------
2016-03-02 19:08:28	gloda.everybody	ERROR	!!! error initializing resource:///modules/gloda/index_ab.js

 ----------
Wed Mar 02 2016 19:08:28
Error: 2016-03-02 19:08:28	gloda.everybody	ERROR	(resource:///modules/gloda/datastore.js:1550) TypeError: this.asyncConnection is null

Source file: resource://app/modules/gloda/log4moz.js
Line: 691
 ----------
2016-03-02 19:08:28	gloda.everybody	ERROR	(resource:///modules/gloda/datastore.js:1550) TypeError: this.asyncConnection is null

 ----------
Wed Mar 02 2016 19:08:29
Error: nounDef is undefined
Source file: resource://app/modules/gloda/gloda.js
Line: 1901
 ----------
Wed Mar 02 2016 19:08:30
Error: TypeError: this.asyncConnection is null
Source file: resource://app/modules/gloda/datastore.js
Line: 1418
 ----------
Wed Mar 02 2016 19:08:39
Error: TypeError: this.asyncConnection is null
Source file: resource://app/modules/gloda/datastore.js
Line: 1550


I don't know from which change in m-c this comes.
Duplicate of this bug: 1252953
Severity: normal → major
Version: unspecified → Trunk
Last good push on C-C:
Mon Feb 29, 23:19:03
Nightly Requested: Tue Mar 1, 13:45:56 is bad.
So your window is pretty small.

Must be one of these two:
https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=8ef94be995a4 - Tue Mar 1, 11:59:06
https://hg.mozilla.org/mozilla-central/pushloghtml?changeset=8ef94be995a4

https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=5cafa6f3019b - Tue Mar 1, 12:02:11
https://hg.mozilla.org/mozilla-central/pushloghtml?changeset=5cafa6f3019b
Summary: Lot of errors in gloda since 20160301 → Lot of errors in gloda since 2016-03-01
Typical error seems to be:
TypeError: msgNounDef is undefined at resources/glodaTestHelper.js:208
Source:
206  let msgNounDef = GlodaMessage.prototype.NOUN_DEF; <===
207  let orig_insertMessage = GlodaDatastore.insertMessage;
208  GlodaDatastore.insertMessage = msgNounDef.objInsert = function() {

So something has changed in JS.
agree we know the window, but let's let the keyword stand please until we have identified the checkin at fault.
Sqlite got updated in Bug 1248472, and the changelog says:
"Because of continuing security concerns, the two-argument version of of the seldom-used and little-known fts3_tokenizer() function is disabled unless SQLite is compiled with the SQLITE_ENABLE_FTS3_TOKENIZER."
https://www.sqlite.org/releaselog/3_11_0.html
Attached patch patch (obsolete) — Splinter Review
This patch according to bug 1248472 comment 9 works for me locally.
Assignee: nobody → acelists
Status: NEW → ASSIGNED
Attachment #8726307 - Flags: review?(mak77)
Attachment #8726307 - Flags: feedback?(drh)
Should this bug go to Toolkit / Storage like bug 1248472?
Attached patch patch v2Splinter Review
In case you do not like to use that option for toolkit and Firefox, this enables it only when building Thunderbird and Seamonkey.
Attachment #8726307 - Attachment is obsolete: true
Attachment #8726307 - Flags: review?(mak77)
Attachment #8726307 - Flags: feedback?(drh)
Attachment #8726327 - Flags: review?(mak77)
Attachment #8726327 - Flags: feedback?(drh)
Component: Search → Storage
Product: Thunderbird → Toolkit
Comment on attachment 8726327 [details] [diff] [review]
patch v2

Since the tokenizer needs to be native and thread-safe and Firefox does not support binary extensions anymore, it does seem appropriate to only enable this for Thunderbird and SeaMonkey.

I'm a storage peer so I can r+ the semantic change, but I think https://wiki.mozilla.org/Build_config (which seems slightly out of date, but I think the intent still stands) strongly suggests that a build peer review this change too.  I got to that page via searching for "mozilla-central build changes peer"; it's possible there is a more up-to-date appropriate page that that page should redirect to or something.
Attachment #8726327 - Flags: review?(mak77)
Attachment #8726327 - Flags: review?(gps)
Attachment #8726327 - Flags: review+
Attachment #8726327 - Flags: feedback?(drh)
Summary: Lot of errors in gloda since 2016-03-01 → Define SQLITE_ENABLE_FTS3_TOKENIZER for Thunderbird/SeaMonkey only for the gloda global database that uses a custom FTS3 tokenizer
Comment on attachment 8726327 [details] [diff] [review]
patch v2

Review of attachment 8726327 [details] [diff] [review]:
-----------------------------------------------------------------

You don't need a build peer review on this since this has very little to do with the build system and more about sqlite3.
Attachment #8726327 - Flags: review?(gps)
https://hg.mozilla.org/mozilla-central/rev/8c7227510049
Status: ASSIGNED → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla47
Duplicate of this bug: 1510309
You need to log in before you can comment on or make changes to this bug.