Depth Workflows

With Designer’s depth tools, you can use black-and-white depth maps to deform meshes in real time, creating parallax and volume from flat plates or video content. The depth video workflow ties together Depth Map Meshes, Depth Map Mappings, and the Depth Video Layer into a single, repeatable setup detailed here.

What is the depth video workflow?

The depth video workflow uses a Depth Map Mesh as a deformable geometry, whose shape is driven by a depth map image or video. Depth can be applied in two main ways:

• Depth Map Mapping — the recommended, flexible approach that lets you drive a mesh from any layer that outputs a depth map.
• Depth Video Layer — a specialised content layer that sends a depth map video directly to a Depth Map Mesh, useful when mappings are not available (for example, inside a Layer Stack).

In both cases, the mesh occupies a frustum defined by Field of view and Min/Max depth, and the mesh vertices’ z positions are updated dynamically based on the depth values in the content.

Depth video workflow

This section groups the how-to guides that make up a complete depth workflow:

• Set up a Depth Map Mesh
• Control the mesh using a Depth Map Mapping
• Refine depth and frustum settings

Example of a Depth Map Mesh driven by depth content

Workflow elements

1. Add Depth Map Mesh object to the stage.
2. Assign the Depth Map Mapping.
3. Create a Depth Video Layer on the timeline.
4. Assign Depth map source content such as an image or video clip.
5. Target the screen or object using the Depth Map Mesh.

---

Set up a Depth Map Mesh

A Depth Map Mesh is the core of the workflow: it’s the geometry that will be deformed by the depth map.

1. In the Stage view, select the target stage object (for example, a screen or prop).
2. In the object’s Mesh property, create a new Depth Map Mesh.
3. Confirm that the mesh appears in the stage with its default frustum (you can enable Show frustum on the mesh later if needed).

At this point, you have a mesh ready to receive depth information but it will still appear flat until you map depth content to it.

Control the mesh using a Depth Map Mapping

Use this path when you want maximum flexibility and the ability to reuse existing layers or complex setups.

1. Add or identify a layer on the timeline that outputs a depth map (for example, a Video layer playing a depth-map plate).
2. Place the depth-map media into the project structure (for example, under objects/VideoFile) and assign it to the layer.
3. Open the Content Mapping panel and create a new mapping of type Depth Map Mapping.
4. In the mapping, add the target screen (which uses the Depth Map Mesh) to the Screens list.
5. Scrub the timeline or play the track. The screen’s Depth Map Mesh should now deform in real time based on the brightness values in the depth map (black/white representing relative depth).
6. Adjust any Common Mapping Properties (crop, scale, transforms) to align the depth map with the visible content on the screen.

This approach lets you keep your depth content in standard layers and reuse it across different meshes or screens simply by changing mappings.

Refine depth and frustum settings

Once the mesh is being driven by depth content, refine its properties to get clean parallax and minimise artefacts.

1. Depth type
   • Choose how the depth map is interpreted:
     • Raw – 0 is near plane, 1 is far plane.
     • Relative inverse – inverse normalised, where 0 is far plane and 1 is near plane (the default for many depth maps).
2. Resolution
   • Set the mesh Resolution to match (or broadly match) the resolution of the depth map texture. Higher resolution gives smoother deformation but uses more vertices.
3. Field of view
   • Adjust the mesh’s Field of view so that the frustum encompasses the region of interest in your scene. This controls how “wide” the depth volume is relative to your stage.
4. Min/Max depth
   • Set Min depth (near plane) and Max depth (far plane) in metres.
   • These values convert the normalised depth in the texture into real-world distances.
5. Depth offset
   • Use Depth offset to shift the frustum’s eye point relative to the mesh origin. This is useful to centre the mesh around the average depth of your scene.
6. Blur / Dilate radius
   • Increase the Blur and Dilate radius to soften edges and reduce artefacts around objects, especially where depth maps have noise or hard transitions.
7. Show frustum
   • Enable Show frustum to visualise the volume in the stage and confirm that the mesh’s depth matches your camera and scene scale.

Advanced workflows

Once the basic depth video workflow is working, you can combine it with other Designer tools:

• Composite with other content
  • Map standard video plates or 2.5D assets onto the same screen or related geometry, using the Depth Map Mesh as a foreground or background element.
• Use Layer Stacks
  • Place a Depth Video Layer inside a Layer Stack for plates that already include effects (e.g. blur, colour adjustments) before or after depth deformation.
• Animate or track the mesh
  • Remember that the Depth Map Mesh is just another object; it can be parented, animated, or driven by tracking devices in more complex virtual production setups.
• Match camera depth of field
  • Combine the depth video workflow with camera Focus settings to add depth-of-field to your rendered scene, aligning the mesh frustum to the expected focus range of the camera.

---

Related information

• Depth Map Mesh
• Depth Video Layer
• Depth Map Mapping
• 2.5D Workflow