1227100 - [Static Analysis][Division by zero] Function makePrimefromPrimesShaweTaylor from lib/freebl/pqg.c can cause a division by 0

Status: RESOLVED INVALID

(NSS :: Libraries, defect, P3)

Description

[Andi [:andi]]

The Static Analysis tool Scan-Build added a division by 0 error on variable outlen, it's vallue is set to 0 when hashtype is inconsistent.

Comment 1

[Andi [:andi]]

Attachment: Bug 1227100.diff

Comment 2

[Andi [:andi]]

Comment on attachment 8690790 [details] [diff] [review]
Bug 1227100.diff

Hello Robert,

Can you please take a look other this patch?

THX

Comment 3

[Tim Taubert [:ttaubert] (inactive)]

Comment on attachment 8690790 [details] [diff] [review]
Bug 1227100.diff

Review of attachment 8690790 [details] [diff] [review]:
-----------------------------------------------------------------

::: security/nss/lib/freebl/pqg.c
@@ +515,5 @@
>      int i;
>      int iterations;
>      int old_counter;
>  
> +	if (outlen <= 0)

We should probably rather check `hashlen` and move the definition of `outlen` to here. if `hashlen == 0` that means we were passed an invalid `hashtype`. This should be an assertion as both external call sites should never pass an invalid type.

Comment 4

[Andi [:andi]]

Thanks for reviewing this, I'll update the patch shortly.

Comment 5

[Andi [:andi]]

This should be closed since the current code changed and the issue is no longer present.