1212601 - Table creation fails when parsing large digest256 list

Status: RESOLVED WORKSFORME

(Toolkit :: Safe Browsing, defect)

Description

[Matt Wobensmith [:mwobensmith][:matt:]]

1. Unzip sample files.
2. Launch pyserver13000.py
3. Launch pyserver13001.py
4. Change Fx42 prefs for shavar list URL to "localhost:13000"
5. Delete on-disk tracking protection tables. 
6. Relaunch Fx42 via command line, using environment variables from here for debugging: https://wiki.mozilla.org/Phishing_Protection#QA
7. Watch terminal for debug output.

Results:
Indicates that 88mb list was accepted and parsed, but tables were not created.

Expected:
Create the tables or indicate reason for failure.

Comment 1

[Matt Wobensmith [:mwobensmith][:matt:]]

Attachment: stdout.txt

Comment 2

[Matt Wobensmith [:mwobensmith][:matt:]]

Test files too big to attach here. 

Download ZIP instead:
http://people.mozilla.org/~mwobensmith/test/1212601.zip

Comment 3

[François Marier [:francois]]

We should figure out what it is that makes the parsing fail first and then put a reasonable file size limit on this (bug 1212600).

Comment 4

[Dimi Lee [:dimi]]

Hi François,
If you don't mind, i would like to work on this bug

Comment 5

[Dimi Lee [:dimi]]

(In reply to Matt Wobensmith [:mwobensmith][:matt] from comment #2)
> Test files too big to attach here. 
> 
> Download ZIP instead:
> http://people.mozilla.org/~mwobensmith/test/1212601.zip

Hi Matt,
I can't download the file now, could you help provide it again ? thanks!

Comment 6

[Matt Wobensmith [:mwobensmith][:matt:]]

Hi Dimi, the file has been uploaded. Give it a try.

Comment 7

[Dimi Lee [:dimi]]

(In reply to Matt Wobensmith [:mwobensmith][:matt] from comment #6)
> Hi Dimi, the file has been uploaded. Give it a try.
Thanks!

The reason that makes parsing fail is because of maximum chunk size allowed[1].
The file indicates that chunk size is 88317856 (a:1444243726:32:88317856), which over the current threshold (1024*1024)[2]

[1] https://dxr.mozilla.org/mozilla-central/source/toolkit/components/url-classifier/ProtocolParser.cpp#216
[2] https://dxr.mozilla.org/mozilla-central/source/toolkit/components/url-classifier/ProtocolParser.cpp#23

Comment 8

[François Marier [:francois]]

(In reply to Dimi Lee[:dimi][:dlee] from comment #7)
> The reason that makes parsing fail is because of maximum chunk size
> allowed[1].
> The file indicates that chunk size is 88317856 (a:1444243726:32:88317856),
> which over the current threshold (1024*1024)[2]

That looks like it's working as intended: throwing an error and aborting the parsing.