1550269 - scriptProcessorNode.onaudioprocess does not fire as expected and misses many seconds of audio data

Status: UNCONFIRMED

(Core :: Web Audio, defect, P3)

Description

[Charles]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:66.0) Gecko/20100101 Firefox/66.0

Steps to reproduce:

I created mic recorder for a product initiative for my company (Istation). I did this in the following way:

// First I created the audio graph nodes
var audioCtx = new (window.AudioContext || window.webkitAudioContext)();
const numChannels = 1;
const bufferSize = 2048;
const sampleRate = audioCtx.sampleRate;
const analyserNode = audioCtx.createAnalyser();
analyserNode.fftSize = 2048;
const rawDataProcessorNode = audioCtx.createScriptProcessor(bufferSize, numChannels, numChannels);
rawDataProcessorNode.onaudioprocess = processAudioData;
var dynamicsCompressor = audioCtx.createDynamicsCompressor();

// Later, in the getUserMedia success callback, I connect them up like so
const stream = mediaStream;
const sourceNode = audioCtx.createMediaStreamSource(stream);
sourceNode.connect(dynamicsCompressor);
dynamicsCompressor.connect(analyserNode);
analyserNode.connect(rawDataProcessorNode);
rawDataProcessorNode.connect(audioCtx.destination);

To gather the data off the mic, I am constantly listening to the channel data from the onaudioprocess event, and push the channel data into a audio chunks buffer if I need to record them at that time, and encode the chunks to wav later when I need to cut the recording off:

function processAudioData(ape) {
if (isRecording) {
const channelData = ape.inputBuffer.getChannelData(0);
audioChunks.push(new Float32Array(channelData));
samplesCount += ape.inputBuffer.getChannelData(0).length;
}
}

Actual results:

Firefox on Windows 10 is giving me chopped up recordings. The recordings get cut off in midstream when they should not be, IE that isRecording flag is set to true and I have not toggled it off at that point in time. This results in a recording that cuts out many seconds of audio, usually 25 - 30 seconds at a time. The way the behavior manifests, it seems like the thread processing the SPN goes to sleep periodically and stops firing the events as it should.

My tester has reproduced this issue on FF for MacOs as well, but my mac mini does not exhibit this behavior (FF 66.0.4 on OSX 10.14.4). For win10, we've reproduced this going back to FF 63.0.3.

Expected results:

The onaudioprocess event should continue to fire for the life of the node. This code is run on Chrome, Safari, and mobile Safari on MacOs, Win10/8/7, and iOS with no such issue.

Comment 1

[Karl Tomlinson (:karlt)]

Much of our testing uses ScriptProcessor in a similar way to this, and so the behavior reported here is surprising.

Is it possible that data is getting lost between the mediaStream from getUserMedia() and the Web Audio nodes?

Our testing has numberOfOutputChannels set to zero, but a non-zero value should not cause this behavior.

Comment 2

[Charles]

I set the numberOfOutputChannels to zero to test that, and I'm getting more data from the stream, but I'm still missing a lot of the audio from the mic.

I don't know how to check if the data is getting lost between the stream and the node.