Virtual Cameras

Virtual cameras can be used as part of mixed reality or green screen workflows to allow shots to be captured from positions outside of the range of the physical cameras. By reprojecting the filmed action

What is a virtual camera?

Virtual cameras can be used as part of mixed reality or green screen workflows to allow shots to be captured from positions outside of the range of the physical cameras. By reprojecting the filmed action and virtual content to a new viewpoint, wide shots can be captured for small stages with limited space. A virtual camera can also be used to traverse a large virtual scene, such as a cityscape. It is possible to start by moving through the streets rendering the virtual scene, and end by transitioning into the view of the real camera on an xR stage, capturing the people on the stage within your virtual scene.

 

 

General Requirements

Virtual cameras are designed to be interchangeable with real cameras as far as possible for an MR/green screen setup. They should be interacted with in similar ways to a real camera.

Virtual cameras appear as an object in the stage, similar to a real camera, and can be linked to a ‘parent camera’, which is the real camera providing the filmed content from the stage.

Content can be mapped to the front and back plates of a virtual camera, and can be used in mixed reality sets in the same way as a real camera. Users can also control the position of a virtual camera in the same ways that are possible for a real camera.

Virtual camera positions can be defined in Global coordinates, or relative to the parent camera. Virtual camera positions can also be keyframed using AnimateCameraControl and AnimateCameraPreset layers.

Reprojection

Note that reprojection is done slightly differently to a real camera. For a real camera, the set extension is reprojected onto the set extension mesh to match the filmed content in order to avoid any discontinuities at the boundaries. For a virtual camera, the filmed content is reprojected using a planar reprojection to match the fixed set extension in order to ensure the set extension remains stable when the real camera moves. This means that there may be more obvious discontinuities between filmed content and set extension when using a virtual camera and moving its parent camera than when moving a real camera. This effect will be more obvious the closer the virtual camera’s view is, so it is recommended not to move the parent camera quickly when using a virtual camera which is only offset by a small amount from its parent.

Setting up the virtual camera:

  1. Create a VirtualCamera in the Cameras list in the stage

  2. Set the ‘Parent camera’ property to the desired real camera.

  3. Add a ‘Live action position marker’ located at the average position of live content on the stage (e.g. people). This will ensure that the live content appears with the correct perspective relative to the virtual content when the virtual camera moves.

  4. ‘Max zoom in factor’ can be set to restrict how much the filmed content can be zoomed into before purely virtual content is displayed. This prevents excessive pixelation when moving the virtual camera towards the live action.

  5. Set the lens settings under Physical->Lens. These can be defined locally in the virtual camera, set to follow the zoom of the parent camera (with an optional scaling applied), or set to follow all the intrinsics of the parent camera (with an optional zoom scale).

    6. To control a virtual camera:

    There are various ways for controlling the position of the virtual camera. You can either:

    • Use the object manipulators to move the VirtualCamera object around in the stage

    • Edit the values in the VirtualCamera editor. Note that values can be defined in a Global coordinate system, or Relative to the parent camera.

    • Use a tracking source: A tracking source can be used to control the pose and lens intrinsics for a virtual camera in the same way as a real camera

    • Keyframe camera animations using AnimateCameraPreset or AnimateCameraControl.

To keyframe virtual camera animations:

Virtual camera animations can be keyframed similarly to standard cameras using the AnimateCameraControl or AnimateCameraPreset layers.

  • AnimateCameraControl:
    • Add an AnimateCameraControl layer
    • Set the camera to the VirtualCamera you wish to animate
    • Set the coordinate system to Global or Relative as desired
    • Keyframe virtual camera properties. In addition to the standard properties for a normal camera, zoom scale can also be keyframed
  • AnimateCameraPreset:
    • Add an AnimateCameraPreset layer
    • Set the camera to the VirtualCamera you wish to animate
    • Add new VirtualCameraPositions to the Position property
    • Set the coordinate system to Global or Relative as desired
    • In addition to the standard properties for a normal camera, the virtual camera’s zoom can be set. Toggle the ‘Animate zoom scale’ option to choose whether to animate zoom scale or global field of view
    • Keyframe several VirtualCameraPositions to animate the camera between these positions. Positions in Global and Relative coordinates can be mixed, and the camera will move between the coordinate systems.

Limitations

  • If the virtual camera is significantly off-axis from the real camera, the live objects (e.g. people) will become skewed. The projection can be thought of as looking at a 2D cardboard cutout from a different angle.

  • All live action must be completely within the frame captured by the real camera. If not, it will be cut off during the reprojection.

  • If the lens is heavily distorted, there may be distortion artefacts. A radial mask can be set in the editor to remove these effects, which are most pronounced at the edges of the image.

  • If you attempt to move the virtual camera forwards towards the live action, the picture will become progressively more pixelated. This is because you are zooming on the captured image at fixed resolution. This can be mitigated by setting the ‘Max zoom in factor’.

  • If the virtual camera moves behind the live action, it will render virtual content only. disguise Technologies

  • If the parent camera moves quickly, there may be some noticeable discontinuity between filmed content and the set extension. This may be more noticeable if the virtual camera is near the parent camera. For fast camera moves it is recommended to use a real camera rather than a virtual camera.