Bugzilla

For context, each decoded token is [buffered by LlamaRunner](https://searchfox.org/firefox-main/source/toolkit/components/ml/backends/llama/LlamaRunner.cpp#106). I suspect this breaks UTF8 sequences, as `llama_decode` is token-wise, not character-wise. 

The error we see is defined [here](https://searchfox.org/firefox-main/source/js/public/friend/ErrorNumbers.msg#102). This error message is only ever produced by [`InflateUTF8ToUTF16`](https://searchfox.org/firefox-main/source/js/src/vm/CharacterEncoding.cpp#279), in Javascript. This function is called when a C++ string from LlamaRunner crosses IPC boundary, the parent being in JS the C++ string (UTF8) is inflated to UTF16 (JS standard as noted by Greg). This is consistent with either the model outputing incomplete UTF8 sequences, or our buffering causing issues.  

To understand the issue and make sure we don't regress around this, I started by writing a minimal reproducer. To exercise the failing path end-to-end, I asked Claude to write a minimal gguf that always outputs a 3-byte UTF-8 
codepoint. I added a small test to load and run the minimal gguf. The test runs the generation with: 
- a large enough buffer size (`minOutputBufferSize`) so that `LlamaRunner` buffers the whole model response
- a buffer size of 1

On Nightly, the first test case passes and the second fails with exactly the message I get when running Link Preview on the link I gave above

For context, each decoded token is [buffered by LlamaRunner](https://searchfox.org/firefox-main/source/toolkit/components/ml/backends/llama/LlamaRunner.cpp#106). I suspect this breaks UTF8 sequences, as `llama_decode` is token-wise, not character-wise. 

The error we see is defined [here](https://searchfox.org/firefox-main/source/js/public/friend/ErrorNumbers.msg#102). This error message is only ever produced by [`InflateUTF8ToUTF16`](https://searchfox.org/firefox-main/source/js/src/vm/CharacterEncoding.cpp#279), in Javascript. This function is called when a C++ string from LlamaRunner crosses WebIDL boundary, when bound to JS the C++ string (UTF8) is inflated to UTF16 (JS standard as noted by Greg). This is consistent with either the model outputing incomplete UTF8 sequences, or our buffering causing issues.  

To understand the issue and make sure we don't regress around this, I started by writing a minimal reproducer. To exercise the failing path end-to-end, I asked Claude to write a minimal gguf that always outputs a 3-byte UTF-8 
codepoint. I added a small test to load and run the minimal gguf. The test runs the generation with: 
- a large enough buffer size (`minOutputBufferSize`) so that `LlamaRunner` buffers the whole model response
- a buffer size of 1

On Nightly, the first test case passes and the second fails with exactly the message I get when running Link Preview on the link I gave above