Enabling background removal
This document outlines information on how to successfully achieve background removal in Notch when using Disguise media servers.
What is background removal?
Notch has developed a node that uses NVIDIA’s Virtual Background technology to pick out human forms in video/images and remove them from the background, outputting a grayscale mask. This eliminates the need for green screen keying.
Nvidia developed Virtual Background for use on webcam feeds, video calls and streaming. It will therefore give the best results on video feeds that resemble those, such as a front-facing head and shoulders position.
What is needed to support background removal in Disguise?
Background removal requires three specific drivers and SDKs from NVIDIA:
- Graphics driver
- AR SDK
- Video Effects SDK
Background removal in Notch is supported on the GX 3 and GX 2X media servers since OS 24Q2 and 199B.947 respectively.
A useful table outlining the packages relevant to background removal that we ship in these OS images is below:
| Media Server | GPU Architecture | OS | NVIDIA graphics driver | NVIDIA AR SDK Version | NVIDIA Video Effects |
|---|---|---|---|---|---|
| GX 3 | Ampere | Current OS: 24Q2 | 551.52 | 0.8.2 | 0.7.2 |
| GX 3 | Ampere | Previous OS: 201F.1012 | 513 | 0.7.6 | 0.6.5 |
| GX 2C | Turing | Current OS: 199B.947 | 472.98 | 0.7.6 | 0.6.5 |
We are aware that NVIDIA make improvements to Virtual Background in newer versions of the NVIDIA Graphics Driver, AR SDK and Video Effects SDK. Please be aware that if you choose to update your graphics driver or SDKs then you will be running in an unvalidated state, and may experience the following side-effects:
- Dropped frames
- Tearing between outputs
- Failure to Genlock
- Failure to Apply Feed Settings
For this reason should users want to utilise these features note that you will be taking the above risks.
Can I upgrade any of the driver combinations manually?
We develop our own operating systems (OS) for all current Disguise media servers.
Any driver within an OS has been carefully selected and tested before being rolled out into a current OS.
Unless otherwise stated, we do not support manually updated drivers due to the unknown potential impact on performance, applying feeds and genlock.
In the event that any of these drivers have been updated and background removal is not working, we recommend updating the media server to the Current OS.
Will Disguise be updating any of the NVIDIA driver and SDK versions on the Current OS of the gx 3 and gx 2c?
We continue to update the Current OS’ of our media servers to enable functionality and general improvements over time. However, we will only add a later driver or SDK version to a Current OS after it has been rigorously tested by our engineering teams and beta tested by our users.