1348279 - CryptoKeys cannot be read from IndexedDB: The object could not be cloned.

Status: RESOLVED WONTFIX

(Core :: DOM: Web Crypto, defect, P3)

Description

[Stefan Sechelmann]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.21 Safari/537.36 MMS/1.0.2459.0

Steps to reproduce:

We store CryptoKeys in IndexedDB. There a few types of keys: PBKDF2, HKDF, AES-KW. ECDH and ECDSA keys are exported to JWK before storing due to #1048931.
Last week (Mar. 2017) Firefox 52 became stable and seems to introduce problems with the storage of CryptoKeys in IndexedDB. Unfortunately there is litte more I can say about this: I see profiles that work with 52 and do not with 51 and the other other way round. The error that we see when accessing a row containing a CryptoKey is: The object could not be cloned. I will try and attach a test db once I can create one that does not contain sensitive data.

Comment 1

[Stefan Sechelmann]

Ok, here come the steps to reproduce: Open up Firefox version 51.0.1 and run this script on, e.g., https://www.mozilla.org (Attached db is on this origin):
```
console.log('generating and storing keys...');
Promise.all([
  crypto.subtle.generateKey({name: 'AES-KW', length: 256}, false, ['wrapKey', 'unwrapKey']),
  crypto.subtle.generateKey({name: 'AES-CBC', length: 256}, false, ['encrypt', 'decrypt']),
  crypto.subtle.importKey('raw', new Uint8Array(32), {name: 'HKDF'}, false, ['deriveKey', 'deriveBits']),
  crypto.subtle.importKey('raw', new Uint8Array(32), {name: 'PBKDF2'}, false, ['deriveKey', 'deriveBits'])
])
.then(([aes_kw_256, aes_cbc_256, hkdf_256, pbkdf2_256]) => {
  let storeName = 'keys';
  let openRequest = indexedDB.open('cryptoKeyTests');
  openRequest.error = console.error;
  openRequest.onsuccess = () => {
    console.log('db open');
    let db = openRequest.result;
    let t = db.transaction([storeName], 'readwrite');
    t.oncomplete = () => {
      console.log('complete.');
      db.close();
    }
    t.onerror = event => {
      db.close();
      console.error(event);
    }
    console.log('storing keys...');
    let store = t.objectStore(storeName);
    try {
      store.add({id: 1, aes_kw_256});
      store.add({id: 2, aes_cbc_256});
      store.add({id: 3, hkdf_256});
      store.add({id: 4, pbkdf2_256});
    } catch (error) {
      console.error(error);
    }
  }
  openRequest.onupgradeneeded = function () {
    console.log('upgrade needed');
    let db = openRequest.result;
    if (!db.objectStoreNames.contains(storeName)) {
      db.createObjectStore(storeName, {
        keyPath: 'id'
      });
    }
  };
})
.catch(console.error);
```

close Firefox and open version 52 with the same profile. Open DevTools and try to display the key -> No contents to be seen here. Try enumerating the db by running, e.g., this script:

```
enumerateIndexedDB('cryptoKeyTests', console.log, console.error);

function enumerateIndexedDB(dbName, successCallback, errorCallback) {
  let openRequest = indexedDB.open(dbName);
  openRequest.onerror = event => errorCallback(new Error(event + ''));
  openRequest.onsuccess = () => {
    let result = new Map();
    let db = openRequest.result;
    let storesNames = Array.from(db.objectStoreNames);
    for (let storeName of storesNames) {
      enumerateObjectStore(dbName, storeName, (r) => {
        result.set(storeName, r);
        if (result.size === storesNames.length) {
          db.close();
          successCallback(result);
        }
      }, error => {
        db.close();
        errorCallback(error);
      });
    }
  };
}
function enumerateObjectStore(dbName, storeName, successCallback, errorCallback) {
  let result = [];
  let openRequest = indexedDB.open(dbName);
  openRequest.onerror = event => errorCallback(new Error(event + ''));
  openRequest.onsuccess = () => {
    let db = openRequest.result;
    let trans = db.transaction([storeName], 'readonly');
    let store = trans.objectStore(storeName);
    let cursorRequest = store.openCursor();
    cursorRequest.onerror = event => errorCallback(new Error(event + ''));
    cursorRequest.onsuccess = () => {
      let cursor = cursorRequest.result;
      if (cursor) {
        try {
          result.push(cursor.value);
        }
        catch (error) {
          console.warn(dbName + '->' + storeName + '[' + cursor.primaryKey + ']: ' + error);
          errorCallback(error);
        }
        cursor.continue();
      }
      else {
        db.close();
        successCallback(result);
      }
    };
  };
}
```

Result:
cryptoKeyTests->keys[1]: DataCloneError: The object could not be cloned.  Scratchpad/1:41:11
DOMException [DataCloneError: "The object could not be cloned."
code: 25
nsresult: 0x80530019
location: Scratchpad/1:38]  Scratchpad/1:19:9
cryptoKeyTests->keys[2]: DataCloneError: The object could not be cloned.  Scratchpad/1:41:11
DOMException [DataCloneError: "The object could not be cloned."
code: 25
nsresult: 0x80530019
location: Scratchpad/1:38]  Scratchpad/1:19:9
cryptoKeyTests->keys[3]: DataCloneError: The object could not be cloned.  Scratchpad/1:41:11
DOMException [DataCloneError: "The object could not be cloned."
code: 25
nsresult: 0x80530019
location: Scratchpad/1:38]  Scratchpad/1:19:9
cryptoKeyTests->keys[4]: DataCloneError: The object could not be cloned.  Scratchpad/1:41:11
DOMException [DataCloneError: "The object could not be cloned."
code: 25
nsresult: 0x80530019
location: Scratchpad/1:38]  Scratchpad/1:19:9
Map { keys: Array[0] }

Thats it, the profile and keys are no more usable in Firefox :-( I attach the test profile that I used to create this description up in the bug description.
Help is greatly appreciated.

Comment 2

[Stefan Sechelmann]

Attachment: storage folder with corrupted indexeddb on https://www.mozilla.org

Comment 3

[Ben]

Between Firefox versions 51 and 52, there's a 1-byte difference looking at the blob containing the webCrypto key stored in the sqlite db. If the key is stored in version 51, this byte is 0x09; in version 52, it's 0x0a. It's the byte at position 0x3a of the blob:

xxd key1-51.0.1.txt
00000000: b801 0000 0501 04f1 ff01 0734 0800 ffff  ...........4....
00000010: 0200 0080 0400 ffff 6964 0112 2800 0001  ........id..(...
00000020: 0000 0003 00ff ff0a 0d18 2461 6573 5f6b  ..........$aes_k
00000030: 775f 3235 3609 2001 010c 0980 ffff 0128  w_256. ........(
00000040: 0800 01c0 010f 9000 2000 0000 a1b4 8d13  ........ .......
00000050: 278f 902c 26a7 a25b c257 5640 3cb6 773c  '..,&..[.WV@<.w<
00000060: 701b bb1a 1334 e861 ff64 a9b3 0529 3a01  p....4.a.d...):.
00000070: 0038 0600 0000 4100 4500 5300 2d00 4b00  .8....A.E.S.-.K.
00000080: 5705 1e05 5811 0128 0100 0000 0000 0013  W...X..(........
00000090: 00ff ff                                  ...

xxd key1-52.0.1.txt
00000000: b801 0000 0501 04f1 ff01 0734 0800 ffff  ...........4....
00000010: 0200 0080 0400 ffff 6964 0112 2800 0001  ........id..(...
00000020: 0000 0003 00ff ff0a 0d18 2461 6573 5f6b  ..........$aes_k
00000030: 775f 3235 3609 2001 010c 0a80 ffff 0128  w_256. ........(
00000040: 0800 01c0 010f 9000 2000 0000 4abb 1028  ........ ...J..(
00000050: 069c 3c5e 33b5 21f4 59ad 3e7a 9a72 66da  ..<^3.!.Y.>z.rf.
00000060: 76ea e8ce 4da8 3040 a9b1 1943 0529 3a01  v...M.0@...C.):.
00000070: 0038 0600 0000 4100 4500 5300 2d00 4b00  .8....A.E.S.-.K.
00000080: 5705 1e05 5811 0128 0100 0000 0000 0013  W...X..(........
00000090: 00ff ff

This behaviour is the same for all four keys in the provided test.

Comment 4

[Tim Taubert [:ttaubert] (inactive)]

This is all very weird. Trying to find a regression with mozregression is tough. I have a feeling that it could be related to bug 768074 and/or bug 964561.

Jan, what do you think? Is it possible that somehow we can't read IndexedDB data containing CryptoKeys created with Fx 51 in Fx 52 and above?

Comment 5

[Jan Varga [:janv]]

Well, it's hard to say if it's IndexedDB who causes the 1 byte difference.
I could try to debug this, but I need the key.txt.

Anyway, if this is a regression in IndexedDB then I guess we would see much more similar issues.

Comment 6

[Ben]

Attachment: key1-51.0.1.txt

Comment 7

[Ben]

Attachment: key1-52.0.1.txt

Comment 8

[BugBot [:suhaib / :marco/ :calixte]]

In the process of migrating remaining bugs to the new severity system, the severity for this bug cannot be automatically determined. Please retriage this bug using the new severity system.

Comment 9

[BugBot [:suhaib / :marco/ :calixte]]

The severity field is not set for this bug.
:freddy, could you have a look please?

For more information, please visit BugBot documentation.

Comment 10

[Frederik Braun [:freddy]]

Wrong component.

Comment 11

[BugBot [:suhaib / :marco/ :calixte]]

The severity field is not set for this bug.
:keeler, could you have a look please?

For more information, please visit BugBot documentation.

Comment 12

[Dana Keeler (she/her) [:keeler]]

I'm fairly sure the changed byte appears in the serialization before the data from webcrypto is written out, so this is probably due to changes elsewhere (indexeddb or structured cloning). In any case, after 6 years, I think it's safe to assume the relevant keys have been regenerated.