1362536 - ASan failures and assertions on 32-bit mpi fuzzing runs

Status: RESOLVED FIXED

(NSS :: Libraries, defect)

Description

[Tim Taubert [:ttaubert] (inactive)]

Maybe related to bug 1362392?

See https://treeherder.mozilla.org/#/jobs?repo=nss-try&revision=d53e1c7ca29f9c27585904ced06a3109e4b7be86&selectedJob=96988443

There are a few segfaults in s_mp_exch(), but also two compare assertions in mpi-div and mpi-mod.

Comment 1

[Daniel Veditz [:dveditz]]

Is this consistent or a one-time thing? The stack in that log looks like it's all in the fuzzing harness. Any additional information that you can provide that points to a security problem?

Comment 2

[Franziskus Kiefer [:franziskus]]

This is more or less the same issue as bug 1362392 but found by the fuzzer.
From what I can tell so far this is a mix of build issues (our fault and not really a bug) and a real issue in mpi-div on 32-bit. So if mpi-div really turns out to being broken in certain cases on 32-bit, this is a security issue as we'll compute wrong values in a predictable way (it's used here [1]).

[1] https://searchfox.org/nss/search?q=symbol:_Z6mp_div&redirect=false

Comment 3

[Franziskus Kiefer [:franziskus]]

So mp_div for 32-bit is broken because we throw away bits that we'd need by casting mp_word to mp_digit [1]. I'm surprised that we don't see things failing because of this. The solution is probably to remove the 32-bit only code, which works as far as I can tell. I'll prepare a patch to fix this.

[1] https://searchfox.org/nss/rev/135560d23ec7688044cfb20b958bd528806eba80/lib/freebl/mpi/mpi.c#4262

Comment 4

[Franziskus Kiefer [:franziskus]]

This was fixed in bug 1362392