1237403 - Differential Testing: Different output message involving disassemble

Status: RESOLVED INVALID

(Core :: JavaScript Engine: JIT, defect)

Description

[Gary Kwong [:gkw] [:nth10sd] (NOT official MoCo now)]

// Adapted from randomly chosen test: js/src/jit-test/tests/asm.js/testBug878495.js
print(
    disassemble("-r", function() {
        (function() {
            "use asm"
            return {}
        })()
    })
)

$ ./js-dbg-32-dm-darwin-dd6d447fc6e0 --fuzzing-safe --no-threads --ion-eager testcase.js
flags: LAMBDA CONSTRUCTOR
loc     op
-----   --
main:
00000:  lambda (function () {
            "use asm"
            return {}
        })
00005:  undefined
00006:  call 0
00009:  pop
00010:  retrval

Source notes:
 ofs line    pc  delta desc     args
---- ---- ----- ------ -------- ------
  0:    3     0 [   0] newline
  1:    4     0 [   0] colspan 9
  3:    4    10 [  10] xdelta
  4:    4    10 [   0] setline  lineno 7
  6:    7    10 [   0] colspan 12

[native code]


$ ./js-dbg-32-dm-darwin-dd6d447fc6e0 --fuzzing-safe --no-threads --ion-eager --no-fpu testcase.js
flags: LAMBDA CONSTRUCTOR
loc     op
-----   --
main:
00000:  lambda (function () {
            "use asm"
            return {}
        })
00005:  undefined
00006:  call 0
00009:  pop
00010:  retrval

Source notes:
 ofs line    pc  delta desc     args
---- ---- ----- ------ -------- ------
  0:    3     0 [   0] newline
  1:    4     0 [   0] colspan 9
  3:    4    10 [  10] xdelta
  4:    4    10 [   0] setline  lineno 7
  6:    7    10 [   0] colspan 12

flags: LAMBDA CONSTRUCTOR
loc     op
-----   --
main:
00000:  newobject ({})
00005:  return
00006:  retrval

Source notes:
 ofs line    pc  delta desc     args
---- ---- ----- ------ -------- ------
  0:    4     0 [   0] newline
  1:    5     0 [   0] colspan 12
  3:    5     0 [   0] newline
  4:    6     0 [   0] colspan 12
  6:    6     6 [   6] colspan 9


Tested this on m-c rev dd6d447fc6e0.

My configure flags are:

LD=ld CROSS_COMPILE=1 CC="clang -Qunused-arguments -msse2 -mfpmath=sse -arch i386" RANLIB=ranlib CXX="clang++ -Qunused-arguments -msse2 -mfpmath=sse -arch i386" AS=$CC AR=ar STRIP="strip -x -S" HOST_CC="clang -Qunused-arguments -msse2 -mfpmath=sse" AUTOCONF=/usr/local/Cellar/autoconf213/2.13/bin/autoconf213 HOST_CXX="clang++ -Qunused-arguments -msse2 -mfpmath=sse" sh /Users/skywalker/trees/mozilla-central/js/src/configure --target=i386-apple-darwin9.2.0 --enable-macos-target=10.5 --enable-debug --enable-more-deterministic --with-ccache --enable-gczeal --enable-debug-symbols --disable-tests

python -u ~/funfuzz/js/compileShell.py -b "--enable-debug --enable-more-deterministic --32" -r dd6d447fc6e0

Note the [native code] portion. This seems to have happened before m-c rev 54be5416ae5d in Nov 2014, so setting needinfo? from Luke and Benjamin, since this involves "use asm" and they reviewed/fixed bug 878495 from which this testcase hails from.

Comment 1

[Benjamin Bouvier [:bbouvier] (inactive)]

In the first mode (without --no-fpu), asm.js is used; in the second mode, as we're on 32 bits and --no-fpu, asm.js is disabled. So while the former sees an asm.js code (not interpreted, without a script), the latter sees a simple (non-asm.js) function (interpreted, with a script).

Apparently, disassemble() uses a JSScript to compute the informations it then displays. In the first mode (with asm.js), we don't have a JSScript, so we just show "native code" instead.

One way to fix this would be to regenerate the JSFunction/JSScript from the asm.js's module/function source code, to feed the Disassembler with it. That is something we haven't ever done, and it looks like a lot of complexity for a function only available in the shell (this feature is not exposed to the web platform, as far as I know).

To work around that, fuzzers might just want to discards tests involving both of "use asm" and "disassemble". That's an unfortunate solution but the work involved to fix this properly would be high complexity / low reward. Luke, what do you think?

Comment 2

[Gary Kwong [:gkw] [:nth10sd] (NOT official MoCo now)]

We always run with --fuzzing-safe, so can you put these special-case stuff in under --fuzzing-safe?

Comment 3

[Gary Kwong [:gkw] [:nth10sd] (NOT official MoCo now)]

Or should we just ignore differential testing errors with testcases involving disassemble, akin to bug 1237464? (See the resolution at the bottom of that bug)

i.e. add "disassemble" to a blacklist similar to:

https://github.com/MozillaSecurity/funfuzz/commit/047cb93b3858e1df35752a196043cb2ab92c61fe

Comment 4

[Benjamin Bouvier [:bbouvier] (inactive)]

(In reply to Gary Kwong [:gkw] [:nth10sd] from comment #3)
> Or should we just ignore differential testing errors with testcases
> involving disassemble, akin to bug 1237464? (See the resolution at the
> bottom of that bug)
> 
> i.e. add "disassemble" to a blacklist similar to:
> 
> https://github.com/MozillaSecurity/funfuzz/commit/
> 047cb93b3858e1df35752a196043cb2ab92c61fe

Yes, that's what I was suggesting (maybe unclear) as a workaround in comment 1. Ideally, it should be ignored when:

1) there is "disassemble"
2) the comparison is between asm.js and non-asm.js

But I don't know if the funfuzz can do this?

Comment 5

[Gary Kwong [:gkw] [:nth10sd] (NOT official MoCo now)]

For now, we can do one or the other. I think we should go with (1) for the time being. Thanks!

Updated blacklist:

https://github.com/MozillaSecurity/funfuzz/commit/7feca3856cbc125182ac3a9fb05fd68772ed79a5

Resolving INVALID as this is not a bug in SpiderMonkey.

Comment 6

[Liz Henry (:lizzard)]

Marking unaffected for 46.