1152939 - (SQLite3.8.9) Upgrade to SQLite 3.8.9

Status: RESOLVED FIXED

(Toolkit :: Storage, defect)

Description

[Arthur K. (he/him)]

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:36.0) Gecko/20100101 Firefox/36.0 SeaMonkey/2.33.1
Build ID: 20150321194901

Steps to reproduce:

http://www.sqlite.org/news.html

http://www.sqlite.org/releaselog/3_8_9.html

"The 3.8.8.3 patch release fixes an obscure problem in the SQLite code generator that can cause incorrect results when the qualifying expression of a partial index is used inside the ON clause of a LEFT JOIN. This problem has been in the code since support for partial indexes was first added in version 3.8.0. However, it is difficult to imagine a valid reason to every put the qualifying constraint inside the ON clause of a LEFT JOIN, and so this issue has never come up before."

Comment 1

[Arthur K. (he/him)]

These are the correct relnotes:

"SQLite version 3.8.9 is a regularly scheduled maintenance release. New features in this release include the PRAGMA index_xinfo command, the sqlite3_status64() interface, and the ".dbinfo" command of the command-line shell. See the release notes for additional information. "

Comment 2

[Ryan VanderMeulen [:RyanVM]]

http://lcamtuf.blogspot.com/2015/04/finding-bugs-in-sqlite-easy-way.html

Yikes.

Comment 3

[Ben Turner (not reading bugmail, use the needinfo flag!)]

I'm glad we didn't do WebSQL!

Comment 4

[Arthur K. (he/him)]

(In reply to Ryan VanderMeulen [:RyanVM UTC-4] from comment #2)
> http://lcamtuf.blogspot.com/2015/04/finding-bugs-in-sqlite-easy-way.html
> 
> Yikes.

Someone with lots more time (and creativity) to fuzz will likely yield a 3.8.9.1. Wash, rinse, repeat.

Comment 5

[Marco Bonardo [:mak]]

One of the good things about using a so much known and used library, is we get a lot of third party testing. This is all wins for us :)

Comment 6

[Daniel Veditz [:dveditz]]

(In reply to Arthur K. from comment #4)
> Someone with lots more time (and creativity) to fuzz 

You would be hard pressed to find more creativity than Zalewski's

>  will likely yield a 3.8.9.1. Wash, rinse, repeat.

That's the software treadmill for you. If you're not prepared to stay on top of a library's upgrades you shouldn't be using that library.

Since we don't operate on raw SQL from the web hopefully we're insulated from the security bugs, but better safe than sorry.

[Tracking Requested - why for this release]:
I assume we wouldn't actually upgrade ESR-31 (EOL this summer) unless we know a specific security bug could be triggered from web content but I wanted to raise the possibility. It would be good to get this on ESR-38 though so we don't have nagging worries for the next year. Not urgent enough at this point to take on Beta 38, so 39 and ESR 38.1 are probably good enough.

Comment 7

[Ryan VanderMeulen [:RyanVM]]

If we decide to take this for esr38, I'd like to get it on b2g37 for v2.2 as well. I could argue for b2g34 (v2.1) as well.

Comment 8

[Reed Loden [:reed]]

Attachment: patch - v1

Comment 9

[Ryan VanderMeulen [:RyanVM]]

Not sure if you were planning to run this through Try or not, but please do. We've caught regressions in the past by doing so.

Comment 10

[Reed Loden [:reed]]

(In reply to Ryan VanderMeulen [:RyanVM UTC-4] from comment #9)
> Not sure if you were planning to run this through Try or not, but please do.
> We've caught regressions in the past by doing so.

I'm waiting for my mercurial access to be re-enabled (bug 1154543) so I can do that...

Comment 11

[Reed Loden [:reed]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=a219d371c3dc

Comment 12

[Sylvestre Ledru [:Sylvestre]]

Tracking as we want to update it.
It would be great if it could land today (ie have an uplift request) to be part of 38 beta 5 (gtb tomorrow)

Comment 13

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/40957d7011f2

Comment 14

[Reed Loden [:reed]]

Comment on attachment 8592562 [details] [diff] [review]
patch - v1

Approval Request Comment
[Feature/regressing bug #]: Pre-existing SQLite bugs (some of which have security impact)
[User impact if declined]: Leave users vulnerable to known (public) security-impact bugs
[Describe test coverage new/current, TreeHerder]: Just landed on inbound, green Try runs, and passes SQLite's test suites.
[Risks and why]: Low risk
[String/UUID change made/needed]: None.

Comment 15

[Reed Loden [:reed]]

Comment on attachment 8592562 [details] [diff] [review]
patch - v1

Forgive me if I'm doing this wrong... Wasn't sure if you wanted approval requests for both aurora and beta or just one or the other... Please fix as needed. :)

Comment 16

[Sylvestre Ledru [:Sylvestre]]

It is the same for us :)
Thanks for the reactivity, we will approve it once it is in m-c.

Comment 17

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/40957d7011f2

Comment 18

[Arthur K. (he/him)]

(In reply to Daniel Veditz [:dveditz] from comment #6)
> (In reply to Arthur K. from comment #4)
> > Someone with lots more time (and creativity) to fuzz 
> 
> You would be hard pressed to find more creativity than Zalewski's

Indeed. But those guys at Pwn2Own get *really* deep and creative as well.

Comment 19

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/fc14167b2b76

Comment 20

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/01e0d4e09b6d

Comment 21

[Al Billings [:abillings - ex-MoCo]]

Can we categorically state that we don't want to take this in ESR31?

Comment 22

[Ryan VanderMeulen [:RyanVM]]

Try pushes for b2g37 and b2g34 look good. I'm going to let this bake for a week or so before nominating for uplift, though.

Comment 23

[Daniel Veditz [:dveditz]]

(In reply to Al Billings [:abillings] from comment #21)
> Can we categorically state that we don't want to take this in ESR31?

Given the choice I'd love to see this in ESR-31 so I'm not going to "categorically" say don't take it, but it doesn't meet our usual bar of a known high/critical security bug or a topcrash type stability issue. 

ESR-31 is currently using 3.8.4.2

Comment 24

[Lawrence Mandel [:lmandel] (use needinfo)]

(In reply to Daniel Veditz [:dveditz] from comment #23)
> (In reply to Al Billings [:abillings] from comment #21)
> > Can we categorically state that we don't want to take this in ESR31?
> 
> Given the choice I'd love to see this in ESR-31 so I'm not going to
> "categorically" say don't take it, but it doesn't meet our usual bar of a
> known high/critical security bug or a topcrash type stability issue. 
> 
> ESR-31 is currently using 3.8.4.2

This to me suggests that we should not take this in ESR31. More to the point, I think we need a pretty good argument to take this change on ESR31.

Comment 25

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 8592562 [details] [diff] [review]
patch - v1

This has baked for a couple weeks now across various releases without issue. Let's get this on b2g34/b2g37. See comment 14 for answers to previous uplift request questions.

Comment 26

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-b2g37_v2_2/rev/fb6996086ceb
https://hg.mozilla.org/releases/mozilla-b2g34_v2_1/rev/b1a28164d222

Comment 27

[Ryan VanderMeulen [:RyanVM]]

(In reply to Ryan VanderMeulen [:RyanVM UTC-4] from comment #26)
> https://hg.mozilla.org/releases/mozilla-b2g37_v2_2/rev/fb6996086ceb

Correction:
https://hg.mozilla.org/releases/mozilla-b2g37_v2_2/rev/1063eb65195a

Comment 28

[Vincent Liu[:vliu]]

https://hg.mozilla.org/releases/mozilla-b2g34_v2_1s/rev/b1a28164d222

Comment 29

[Arthur K. (he/him)]

Someone want to create the new bug?

SQLite 3.8.10 has been released: https://www.sqlite.org/releaselog/3_8_10.html


    Added the sqldiff.exe utility program for computing the differences between two SQLite database files.
    Added the y format string to the matchinfo() function of FTS3.
    Performance improvements for ORDER BY, VACUUM, CREATE INDEX, PRAGMA integrity_check, and PRAGMA quick_check.
    Fix many obscure problems discovered while SQL fuzzing.
    Identify all methods for important objects in the interface documentation. (example)
    Made the American Fuzzy Lop fuzzer a standard part of SQLite's testing strategy.
    Add the ".binary" and ".limits" commands to the command-line shell.
    Make the "dbstat" virtual table part of standard builds when compiled with the SQLITE_ENABLE_DBSTAT_VTAB option.
    SQLITE_SOURCE_ID: "2015-05-07 11:53:08 cf975957b9ae671f34bb65f049acf351e650d437"
    SHA1 for sqlite3.c: 0b34f0de356a3f21b9dfc761f3b7821b6353c570