A tool for stress testing objects and PIC's

RESOLVED FIXED

Status

()

Core
JavaScript Engine
RESOLVED FIXED
7 years ago
2 years ago

People

(Reporter: jandem, Assigned: jandem)

Tracking

({meta, sec-other})

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [sg:nse meta])

Attachments

(3 attachments, 2 obsolete attachments)

(Assignee)

Description

7 years ago
Created attachment 509206 [details]
Script

A few months ago I wrote a simple script for stress-testing operations on objects. It's by no means a fuzzer like the ones from Jesse and decoder. It performs only these steps:

1) generate random objects
2) generate functions (using a pre-defined list of statements)
3) generate calls (pass a random object to a random function)

Since I wrote it in a few days and it's a very simple script (< 400 loc), I will just post it here. Maybe other people are interested in this or know about other operations to add. And I only tested this on my MBP, maybe others can run it on more powerful machines and test on Windows, ARM etc.

Marking security sensitive because there are still (possible) security-sensitive bugs open and I haven't tested this against FF 3.6. After these issues are fixed I will just move it to github or somewhere else.
(Assignee)

Updated

7 years ago
Depends on: 630865
(Assignee)

Updated

7 years ago
Depends on: 625718
(Assignee)

Updated

7 years ago
Depends on: 627984
(Assignee)

Updated

7 years ago
Depends on: 628334
(Assignee)

Updated

7 years ago
Depends on: 630366
Keywords: meta
Whiteboard: [sg:nse meta]
(Assignee)

Updated

7 years ago
Depends on: 631219
(Assignee)

Updated

7 years ago
Depends on: 632778
(Assignee)

Updated

7 years ago
Depends on: 632901
(Assignee)

Updated

7 years ago
Depends on: 632924
(Assignee)

Updated

7 years ago
Depends on: 633525
(Assignee)

Updated

7 years ago
Depends on: 633741
(Assignee)

Updated

7 years ago
Depends on: 633752
(Assignee)

Updated

7 years ago
Depends on: 634210
(Assignee)

Updated

7 years ago
Depends on: 635200
(Assignee)

Updated

7 years ago
Depends on: 635531
Depends on: 636394
jandem should own his fuzzer bug. :)
Assignee: general → jandemooij
Depends on: 636421
Created attachment 514769 [details]
script with rudimentary additions to simple objects

Here's a rudimentary hack to the fuzzer. I taught some e4x objects (porting over hackishly from jsfunfuzz) among others to this fuzzer, and it at least found bug 636421 as well.
Attachment #514769 - Attachment mime type: application/octet-stream → text/x-python-script
Depends on: 636428
(Assignee)

Updated

7 years ago
Depends on: 636487
(Assignee)

Comment 3

7 years ago
(In reply to comment #2)
> I taught some e4x objects (porting
> over hackishly from jsfunfuzz) among others to this fuzzer,

That's a nice list, thanks again. (we talked on IRC, I have some local changes as well, we may try to create a repository somewhere)
Depends on: 636697
Created attachment 515154 [details]
script with comprehensive additions to simple objects

I wrote a objectTester.js script which uses the js shell to test if certain words and their combinations (e.g. "Array" and ".prototype") are objects. After removing duplicates, I added them to test.py.

This is a way more comprehensive list of simple objects, btw.
Attachment #514769 - Attachment is obsolete: true
Depends on: 636818
Created attachment 515159 [details]
objectTester.js

This script bridges the gap between jsfunfuzz and this fuzzer. It uses words taught to jsfunfuzz (e.g. e4x and gang), puts them in combination with each other (e.g. Iterator and .prototype) as described in the previous comment, and tests if they are objects, printing them if they are.

To remove dupes, I piped the stdout through `sort` and then `uniq`.

Thanks to jorendorff for helping me out - his advice was invaluable. This has found bug 636818 very quickly.
(Assignee)

Comment 6

7 years ago
It might be interesting to use the output of this script as input to decoder's adaptive fuzzer. This can be done easily by saving every file instead of files which cause problems.
(In reply to comment #6)
> It might be interesting to use the output of this script as input to decoder's
> adaptive fuzzer. This can be done easily by saving every file instead of files
> which cause problems.

I was thinking exactly the same thing for jsfunfuzz just yesterday before I plonked down on bed.

Or we could think about combining all 3 js fuzzers in some way, this one, jsfunfuzz and decoder's one. What language is decoder's fuzzer in?
(Assignee)

Comment 8

7 years ago
We talked about this on IRC, would love to hear what it finds :)

About the license, MPL is probably easiest if jsfunfuzz uses it too. So I'll stick with MPL for now, but let me know if another license would make things easier.
Both ideas (using jsfunfuzz and jandem's script as LangFuzz input) will surely work and sound good. As soon as you can write the source to a file instead of running it, you can either tell LangFuzz to learn script fragments from it (which will be put into existing regression tests), or we would mutate the tests that those tools generate themselves.

Both should work out of the box, we only have to think about how to get the source into single files for langfuzz.

Note though that E4X is unsupported as the grammar I am using does not support it.
(In reply to comment #9)
> Both ideas (using jsfunfuzz and jandem's script as LangFuzz input) will surely
> work and sound good. As soon as you can write the source to a file instead of
> running it, you can either tell LangFuzz to learn script fragments from it
> (which will be put into existing regression tests), or we would mutate the
> tests that those tools generate themselves.
> 
> Both should work out of the box, we only have to think about how to get the
> source into single files for langfuzz.
> 

decoder, I have it working (splicing jandem's generated files into jsfunfuzz scripts prior to running the latter) in a private Mozilla repository, would you like access? (jandem's getting access soon - this will make things much easier)

I'm not sure how langfuzz will fit in but here's how it's done:

jandem's fuzzer generates files which then get appended into jsfunfuzz. The script is then run by a js shell, where jsfunfuzz begins its random generation of fuzz data. If a crash is found by jandem's fuzzer, the edited jsfunfuzz script will display the error before it even generates its random data.

Sidenote: we could even splice in existing regression tests from the TM source code repository, but it's not done yet.

> Note though that E4X is unsupported as the grammar I am using does not support
> it.

Could it be trained for E4X? jandem's fuzzer didn't support E4X too till it got trained.
(In reply to comment #8)
> About the license, MPL is probably easiest if jsfunfuzz uses it too. So I'll
> stick with MPL for now, but let me know if another license would make things
> easier.

jsfunfuzz is on tri-license (MPL/ LGPL/ GPL) so I'm guessing that's the one you want for your fuzzer.

ref http://www.mozilla.org/MPL/boilerplate-1.1/mpl-tri-license-c
Created attachment 515330 [details]
4test.py

This was my modification to test.py to generate a script (current.js) at a time (I merely commented out some file generation code near the bottom).

Through startjsfunfuzz.py, jsfunfuzz will then take care of the rest of the file generation. The fuzzing harness will then run it continuously. It has a ton of assumptions, including the fuzzing repo must lie in ~/fuzzing/ and tracemonkey is assumed to lie in ~/tracemonkey/

Go to fuzzing/jsfunfuzz and run `python startjsfunfuzz.py 32 dbg tm" to get a about-to-be-fuzzed 32-bit js debug shell TM build on the Desktop (in its folder) - correct as of today, might overhaul in the future.

jandem, in the fuzzing repo as long as you modify the stuff your fuzzer generates, the rest of us will pick it up just by `hg pull`-ing to tip, assuming you pushed.
Attachment #515154 - Attachment is obsolete: true
(In reply to comment #10)

> decoder, I have it working (splicing jandem's generated files into jsfunfuzz
> scripts prior to running the latter) in a private Mozilla repository, would you
> like access? (jandem's getting access soon - this will make things much easier)

Sure :)

> 
> I'm not sure how langfuzz will fit in but here's how it's done:
> 
> jandem's fuzzer generates files which then get appended into jsfunfuzz. The
> script is then run by a js shell, where jsfunfuzz begins its random generation
> of fuzz data. If a crash is found by jandem's fuzzer, the edited jsfunfuzz
> script will display the error before it even generates its random data.

To integrate LangFuzz here, all I need is a set of files, one "test" per file that can be run. This would suffice for both learning and mutating. For learning only, the file doesn't need to be runnable standalone, I only need to be able to parse it.

> > Note though that E4X is unsupported as the grammar I am using does not support
> > it.
> 
> Could it be trained for E4X? jandem's fuzzer didn't support E4X too till it got
> trained.

I doubt that because it requires to modify my ES3 grammars to generically understand E4X. In order to parse E4X (required to learn/mutate scripts with E4X), you would need to modify my ES3 ANTLR grammar to parse E4X. The fuzzer itself does nothing more than parsing using a grammar/parser class generated from grammar (and that's intended to keep it language independent).
(In reply to comment #13)
> To integrate LangFuzz here, all I need is a set of files, one "test" per file
> that can be run. This would suffice for both learning and mutating. For
> learning only, the file doesn't need to be runnable standalone, I only need to
> be able to parse it.

To run 4test.py alone for your purposes, try:

python -u 4test.py ./js .

This will output a current.js file in the current working directory.

python -u <fuzz script> <js shell> <location where current.js gets output>

I don't think the location works other then current working directory, but the original code that jandem wrote to make it this way is still present in 4test.py, albeit commented out.


> I doubt that because it requires to modify my ES3 grammars to generically
> understand E4X. In order to parse E4X (required to learn/mutate scripts with
> E4X), you would need to modify my ES3 ANTLR grammar to parse E4X. The fuzzer
> itself does nothing more than parsing using a grammar/parser class generated
> from grammar (and that's intended to keep it language independent).

Ok, just clarifying.
Depends on: 637010
(Assignee)

Updated

7 years ago
Depends on: 637014
Depends on: 637202
(Assignee)

Updated

7 years ago
Depends on: 637397
(Assignee)

Updated

7 years ago
Depends on: 637403
Depends on: 637011
Depends on: 637205
Depends on: 638735
Depends on: 639343
Depends on: 639412
Depends on: 639413
Depends on: 640116
Depends on: 640608
Depends on: 641327
(Assignee)

Updated

7 years ago
Depends on: 641479
(Assignee)

Updated

7 years ago
Depends on: 641491
(Assignee)

Updated

7 years ago
Depends on: 641525
(Assignee)

Updated

7 years ago
Depends on: 641535
(Assignee)

Updated

7 years ago
Depends on: 642894
(Assignee)

Updated

7 years ago
Depends on: 643182
Depends on: 643234
Depends on: 643242
Depends on: 643243
Depends on: 643244
Depends on: 643245
Depends on: 643249
Depends on: 643250
Depends on: 643669
No longer depends on: 643669
Depends on: 643670
Depends on: 643847
Depends on: 645505
(Assignee)

Updated

7 years ago
Depends on: 645629
(Assignee)

Updated

7 years ago
Depends on: 645632
Depends on: 645641
(Assignee)

Updated

7 years ago
Depends on: 645645
(Assignee)

Updated

7 years ago
Depends on: 645666
(Assignee)

Updated

7 years ago
Depends on: 645985
Depends on: 646366
Depends on: 647464
Depends on: 648992
(Assignee)

Updated

7 years ago
Depends on: 649439
(Assignee)

Updated

7 years ago
Depends on: 649939
Depends on: 650330
Depends on: 652177
(Assignee)

Updated

7 years ago
Depends on: 652806
Depends on: 654073
Depends on: 655940
Depends on: 655950
Depends on: 656226
Depends on: 656252
Depends on: 657198
Depends on: 657586
Depends on: 658803
Depends on: 660538
Depends on: 662132
Depends on: 663690
Depends on: 665289
Depends on: 665914
(Assignee)

Updated

7 years ago
Depends on: 666003
Depends on: 666599
Depends on: 667507
Depends on: 672104
Depends on: 673469
Depends on: 673788
Depends on: 673792
Depends on: 675921
Depends on: 678141
Depends on: 678234
Depends on: 680976
Depends on: 682168
Depends on: 683966
Depends on: 684922
Depends on: 690285
Depends on: 690650
Depends on: 695902
Depends on: 696492
Depends on: 697155
Depends on: 697255
Depends on: 698074
Depends on: 698581
Depends on: 698899
Depends on: 698944
Depends on: 699166
Depends on: 700464
Depends on: 700501
Depends on: 703818
Depends on: 705859
Depends on: 705873
Depends on: 705895
Depends on: 707337
Depends on: 707816
Depends on: 709863
Depends on: 713944
Depends on: 716743
Depends on: 724467
Depends on: 724579
(In reply to Gary Kwong [:gkw, :nth10sd] from comment #5)
> Created attachment 515159 [details]
> objectTester.js

Braindump of command to run objectTester.js :

./js objTester.js 2>&1 | grep -v "strict warning" | grep -v "warning:" | sort | uniq | sort > objectTResults.txt
Depends on: 724654
Depends on: 724702
Depends on: 724784
Depends on: 724788
Depends on: 724798
Depends on: 728033
Depends on: 729364
Depends on: 729793
Depends on: 730152
> > Created attachment 515159 [details]
> > objectTester.js

Second braindump for xpcshell:

LD_LIBRARY_PATH=. ./xpcshell objectTester.js 2>&1 | grep -v "strict warning" | grep -v "warning:" | sort | uniq | sort | grep "\"," > resultsObjectTester.txt
Depends on: 732758
No longer depends on: 729364
No longer depends on: 729793
No longer depends on: 724579
Depends on: 733255
Depends on: 735869
Depends on: 735957
(Assignee)

Comment 17

2 years ago
Closing this bug as it's not really used anymore and shows up in my bz dashboard. jsfunfuzz now does this afaik.
Status: NEW → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → FIXED

Updated

2 years ago
Group: core-security → core-security-release
Group: core-security-release
You need to log in before you can comment on or make changes to this bug.