(In reply to Jean-Yves Avenard [:jya] from comment #28) > (In reply to Byron Campen [:bwc] from comment #21) > > 4. If we're going to need to implement this logic in lots of places, and especially if the answer to 3 is "yes", should we have a single class that handles this as a passthrough, similar to how MediaChangeMonitor handles (most) detection of frames that require a decoder reinit? > > There's not a lot of places, and from what I can see, this only occurs in the Apple VT decoder , for h264 content, so really, pretty unused all things considered. > > The issue with MediaChangeMonitor is that it behaves differently depending on the type of the decoder wrapped (annexb vs avcc). > > To me, if there are any errors, the WebrtcMediaDataDecoderCodec shouldn't be handled them, they should be returned to libwebrtc and let it do whatever it needs to do if an error occur. This will signal the remote to send a new SPS/PPS and send a new keyframe. webrtc.org might have an error code that will cause it to re-create the decoder. I can look into it. But what should we do about the non-webrtc cases? > Attempting to retry something that already fails is doomed to fail again. H264 is very deterministic, particularly the Apple VT decoder. If it failed before, it will fail again. It doesn't randomly work and sometimes not. It is not random, but how thorough we are in resetting things determines whether we end up in the "keeps failing" case or not. If our reset destroys and re-creates the AppleVTDecoder, we're fine from then on. If we just reset the VT session, then we keep failing/resetting. It looks like there's something in the bitstream that has changed in a way that is incompatible with the const state in AppleVTDecoder, and for whatever reason MediaChangeMonitor has not noticed, or has not decided to reset the decoder. > May be worth capturing the content and lodge a bug with the Apple media team rather than attempting to get around it. *In addition* to what we're doing here, yes. But we still need a workaround, because a fix here is probably going to be slow.
Bug 1626364 Comment 30 Edit History
Note: The actual edited comment in the bug view page will always show the original commenter’s name and original timestamp.
(In reply to Jean-Yves Avenard [:jya] from comment #28) > (In reply to Byron Campen [:bwc] from comment #21) > > 4. If we're going to need to implement this logic in lots of places, and especially if the answer to 3 is "yes", should we have a single class that handles this as a passthrough, similar to how MediaChangeMonitor handles (most) detection of frames that require a decoder reinit? > > There's not a lot of places, and from what I can see, this only occurs in the Apple VT decoder , for h264 content, so really, pretty unused all things considered. > > The issue with MediaChangeMonitor is that it behaves differently depending on the type of the decoder wrapped (annexb vs avcc). > > To me, if there are any errors, the WebrtcMediaDataDecoderCodec shouldn't be handled them, they should be returned to libwebrtc and let it do whatever it needs to do if an error occur. This will signal the remote to send a new SPS/PPS and send a new keyframe. webrtc.org might have an error code that will cause it to re-create the decoder. I can look into it. But what should we do about the non-webrtc cases? > Attempting to retry something that already fails is doomed to fail again. H264 is very deterministic, particularly the Apple VT decoder. If it failed before, it will fail again. It doesn't randomly work and sometimes not. It is not random, but how thorough we are in resetting things determines whether we end up in the "keeps failing" case or not. If our reset destroys and re-creates the AppleVTDecoder, we're fine from then on. If we just reset the VT session, then we keep failing/resetting. It looks like there's something in the bitstream that has changed in a way that is incompatible with the const state in AppleVTDecoder, and for whatever reason MediaChangeMonitor has not noticed, or has not decided to reset the decoder. > May be worth capturing the content and lodge a bug with the Apple media team rather than attempting to get around it. *In addition* to what we're doing here, yes. But we still need a workaround, because Apple is probably going to be slow rolling out a fix.
(In reply to Jean-Yves Avenard [:jya] from comment #28) > (In reply to Byron Campen [:bwc] from comment #21) > > 4. If we're going to need to implement this logic in lots of places, and especially if the answer to 3 is "yes", should we have a single class that handles this as a passthrough, similar to how MediaChangeMonitor handles (most) detection of frames that require a decoder reinit? > > There's not a lot of places, and from what I can see, this only occurs in the Apple VT decoder , for h264 content, so really, pretty unused all things considered. > > The issue with MediaChangeMonitor is that it behaves differently depending on the type of the decoder wrapped (annexb vs avcc). > > To me, if there are any errors, the WebrtcMediaDataDecoderCodec shouldn't be handled them, they should be returned to libwebrtc and let it do whatever it needs to do if an error occur. This will signal the remote to send a new SPS/PPS and send a new keyframe. webrtc.org might have an error code that will cause it to re-create the decoder. I can look into it. But what should we do about the non-webrtc cases? > Attempting to retry something that already fails is doomed to fail again. H264 is very deterministic, particularly the Apple VT decoder. If it failed before, it will fail again. It doesn't randomly work and sometimes not. It is not random, but how thorough we are in resetting things determines whether we end up in the "keeps failing" case or not. If our reset destroys and re-creates the AppleVTDecoder, we're fine from then on. If we just reset the VT session, then we keep failing/resetting. It looks like there's something in the bitstream that has changed in a way that is incompatible with the const state in AppleVTDecoder, and for whatever reason MediaChangeMonitor has not noticed, or has not decided to reset the decoder. > May be worth capturing the content and lodge a bug with the Apple media team rather than attempting to get around it. *In addition* to what we're doing here, yes. But we still need a workaround, because Apple is probably going to be slow rolling out a fix. Do we already have a way to capture this content?