VideoFxProcessor does not render properly in some cases #2793
Description
What happened and what did you expect to happen?
We're trying to use VideoFxProcessor for background replacement in our browser app.
However, for some users, they intermittently get this issue where the background renders fine, but their face doesn't. See the image:
They either describe it as rapid flickering, or:
Mine would go black for a few seconds, then back to not black but flickering, then back to black.. constantly the whole time
Once they end up in this state, it doesn't fix itself unless they reload (we only start one VideoFxProcessor when they log in and it is reused across multiple calls).
Relevant parts of the codebase:
const isSupported = await VideoFxProcessor.isSupported(
processorLogger,
true,
).catch((err) => {
logger.error(`VideoFxProcessorError: ${err}`);
return false;
});
if (isSupported) {
this.videoFxProcessor = await VideoFxProcessor.create(
processorLogger,
{
backgroundBlur: {
isEnabled: false,
strength: 'medium' as const,
},
backgroundReplacement: {
isEnabled: true,
backgroundImageURL: getBackgroundImageUrl(effect),
defaultColor: undefined,
},
}
);
}
...
const transform = new DefaultVideoTransformDevice(processorLogger, device, [
this.videoFxProcessor,
]);Have you reviewed our existing documentation?
Reproduction steps
Unfortunately I do not have a reproducer. I only have reports and screenshots from two other people. Their system is not unusual either - recent Apple Silicon Macbooks with recent versions of Chrome. And that is not consistent for them either - one user reports only seeing it one in three meetings.
Amazon Chime SDK for JavaScript version
3.16.0
What browsers are you seeing the problem on?
Chrome, Electron
Browser version
Chrome 118, Electron 32.2