1448571 - AES GCM crashes app

Status: RESOLVED INVALID

(NSS :: Libraries, defect)

Description

[microshine]

Attachment: main.cpp

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36

Steps to reproduce:

I'm trying to encrypt message with AES-GCM key via PKCS#11 interface

If pAAD for CK_AES_GCM_PARAMS is NULL and ulAADLen is 0 it works. But if pAAD is not empty, then app crashes

I added my test app

OS: MacOS High Sierra 10.13.3


Actual results:

My app crashes on C_EncryptInit function. As I can see from error stack this error comes from gcm_HashMult


Expected results:

C_EncryptInit must return CK_RV

Comment 1

[Franziskus Kiefer [:franziskus]]

I guess you mean CK_GCM_PARAMS? The attached main.cpp doesn't compile because of that and some other reasons (like missing cstdlib and others). I'm happy to look at the code if you have something that compiles and crashes.

Comment 2

[microshine]

This code crashes my app

```
    CK_MECHANISM genMech = {
        CKM_AES_KEY_GEN,
        NULL,
        0
    };
    
    CK_BBOOL attrTrue = CK_TRUE;
    CK_BBOOL attrFalse = CK_FALSE;
    CK_ULONG attrValueLen = 128 >> 3;
    
    CK_ATTRIBUTE genTemplate[] = {
        { CKA_TOKEN,        (void *)&attrFalse,     sizeof(CK_BBOOL) },
        { CKA_SENSITIVE,    (void *)&attrFalse,     sizeof(CK_BBOOL) },
        { CKA_EXTRACTABLE,  (void *)&attrFalse,     sizeof(CK_BBOOL) },
        { CKA_DERIVE,       (void *)&attrFalse,     sizeof(CK_BBOOL) },
        { CKA_SIGN,         (void *)&attrFalse,     sizeof(CK_BBOOL) },
        { CKA_VERIFY,       (void *)&attrFalse,     sizeof(CK_BBOOL) },
        { CKA_DECRYPT,      (void *)&attrTrue,      sizeof(CK_BBOOL) },
        { CKA_ENCRYPT,      (void *)&attrTrue,      sizeof(CK_BBOOL) },
        { CKA_UNWRAP,       (void *)&attrFalse,     sizeof(CK_BBOOL) },
        { CKA_WRAP,         (void *)&attrFalse,     sizeof(CK_BBOOL) },
        { CKA_VALUE_LEN,    (void *)&attrValueLen,  sizeof(CK_ULONG) },
    };
    CK_OBJECT_HANDLE hKey = NULL;
    rv = fl->C_GenerateKey(hSession, &genMech, genTemplate, sizeof(genTemplate)/sizeof(CK_ATTRIBUTE), &hKey);
    if (rv != CKR_OK) {
        print_error(rv, "C_GenerateKey");
        return 1;
    }
    
    CK_BYTE iv[] = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6 };
    // CK_BYTE* aad = NULL;             // no error on C_EncryptInit
    CK_BYTE aad[] = { 1, 2, 3, 4, 5 };  // error on C_EncryptInit
    CK_AES_GCM_PARAMS params = {
        iv,                 //pIv
        sizeof(iv),         // ulIvLen
        sizeof(iv) << 3,    // ulIvBits
        aad,                // pAAD
        sizeof(aad),        // ulAADLen
        128                 // ulTagLen
    };
    
    CK_MECHANISM encMech = {
        CKM_AES_GCM,        // mechanism
        &params,            // pParameter
        sizeof(params)      // ulParameterLen
    };
    
    rv = fl->C_EncryptInit(hSession, &encMech, hKey);
    if (rv != CKR_OK) {
        print_error(rv, "C_EncryptInit");
        return 1;
    }
```

Comment 3

[Franziskus Kiefer [:franziskus]]

The only way this compiles is when you're using a pretty old version of NSS (that doesn't even have the final AES-GCM code). As far as I can see CK_AES_GCM_PARAMS was used around 3.15 6 years ago.

Comment 4

[microshine]

I use NSS v3.35

Comment 5

[Franziskus Kiefer [:franziskus]]

Then this code can't compile as CK_AES_GCM_PARAMS is not a type. That should give you a hint on the problem.

Comment 6

[microshine]

I found problem. I use wrong structure for AES GCM params

I updated structure and C_EncryptInit works without errors

I use (cryptoki version 2.40) http://docs.oasis-open.org/pkcs11/pkcs11-base/v2.40/errata01/csd01/include/pkcs11-v2.40/pkcs11t.h

typedef struct CK_GCM_PARAMS {
    CK_BYTE_PTR       pIv;
    CK_ULONG          ulIvLen;
    CK_ULONG          ulIvBits;
    CK_BYTE_PTR       pAAD;
    CK_ULONG          ulAADLen;
    CK_ULONG          ulTagBits;
} CK_GCM_PARAMS;

NSS (cryptoki version 2.20)

typedef struct CK_GCM_PARAMS {
    CK_BYTE_PTR pIv;
    CK_ULONG ulIvLen;
    CK_BYTE_PTR pAAD;
    CK_ULONG ulAADLen;
    CK_ULONG ulTagBits;
} CK_GCM_PARAMS;

Comment 7

[Franziskus Kiefer [:franziskus]]

Good to hear that it's working now. I'll close this bug then. Since CK_GCM_PARAMS is a public type in NSS we can't change it.

Comment 8

[microshine]

Thank you for your help