OmniCal Calibration

Calibration is the automatic process of calculating camera and projector poses and settings using the blobs detected during OmniCal Capture. Calibration also produces a consistent 3D point cloud formed from all blobs.

OmniCal Stage Plan

In the Stage editor, under Projectors, click OmniCal Calibration to open the Calibration editor. This widget controls the main OmniCal behaviour. Right-click on any Capture Plan to open its OmniCalStagePlan editor.

In the OmniCalStagePlan we recommend working through the sections from top to bottom: Capture, Calibration, Alignment, and Mesh Deform (if required).

Calibration

The calibration process triangulates the position of the blobs detected in a capture as well as the relative positions of cameras and projectors and their lens intrinsics. From this point, you no longer need access to the physical stage to continue the calibration process.

Calibration Stages

OmniCal calibration has several stages:

1. Camera Calibration
   • Camera poses are calibrated based on the detected blob information and pre-calibrated lens data.
2. Camera Graph Calculation
   • Cameras are connected to each other on a graph, depending on their mutual overlap.
   • This graph is saved in the project, and re-used in later RigChecks, to ensure that alignments remain compatible.
3. Point Cloud Reconstruction
   • A combined point cloud is generated from all blobs seen by all camera pairs.
4. Projector Calibration
   • Projector poses and intrinsics are calibrated using the point cloud and projected blob coordinates.
5. Bundle Adjustment
   • An iterative process that optimises the pose and intrinsics for all cameras and projectors by minimising the total reprojection error.
6. Final Point Cloud Reconstruction

Calibration Settings

Click Setup Calibration… to open the OmniCal Calibration Setup widget.

In most cases the default settings suffice. If calibration results are unsatisfactory despite good capture quality, adjust one setting at a time to clearly assess its impact.

Note

Most calibration settings include tooltips within Designer to provide contextual information on when and how to adjust them.

Camera Calibration settings

In the Camera Calibration separator we can set the Calibration Model to one of three modes:

• Auto: Attempt to detect appropriate model. Only manually select another mode if ‘Auto’ fails.
• Epipolar: Used with scenes “with depth”, i.e. where blobs are not just projected on a flat surface, but are spread across 3 dimensions.
• Homography: Used with scenes where blobs are mostly projected on a flat surface, or the depth provided is very shallow (e.g. < 1cm).

Bundle Adjustment settings

In the Bundle Adjustment separator we can set Bundle Adjustment target to one of five modes:

• Auto
• Cameras
• Cameras and Projectors
• Cameras then Projectors
• Cameras then each Projector

The number of iterations controls how much time is spent on iteratively improving the overall calibration result. There are two separate iterations settings, that are relevant, depending on the selected BA target:

• Iterations (combined)
  • Used for Cameras and combined Cameras and/then Projectors modes.
  • High values will lead to longer calibration duration, especially for large numbers of cameras and projectors.
• Iterations (individual projectors)
  • Used for Cameras then each Projector mode.
  • This mode is faster, so the iterations can be set much higher than for the combined modes.

Point Cloud Reconstruction settings

A setting under Point Cloud Reconstruction that is often useful is Camera graph optimisation. This should be enabled for camera setups where all cameras overlap with each other, so that they form a closed loop, i.e. the first and last camera overlap with each other. Enabling the camera graph optimisation prevents gaps or discontinuities where the point cloud boundaries meet. This will give more consistent reconstruction and calibration results for typical setups for 360-degree projections inside of or on domes, cylinders, spheres or the walls of a room.

Camera graph BA iterations controls a separate Bundle Adjustment pass for the camera graph optimisation. The default is 500, but it is rare to need more than 50 iterations. This step is much faster than later Bundle Adjustment steps that involve cameras and projectors.

Projector Calibration settings

Projector Calibration Algorithm of Projector Calibration has three modes:

• DLT: Direct Linear Transformation
• Zhang
• Auto: will try all available algorithms and select the best result.

Perform Calibration

Once you have finished adjusting the Calibration Settings, click Calibrate in the Plan editor and confirm with clicking “Yes” to start the calibration process. Depending on the number of cameras and projectors, this can take from a few seconds to tens of minutes.

After starting a calibration, a new window will open that shows its progress. This window also contains detailed messages about the current calibration stage. Once the calibration has finished, you can close this progress window. The messages from the progress window are saved to the OmniCal report, for later viewing.

OmniCal Report

The OmniCal report is useful for assessing the success and quality of a calibration, and for potential trouble-shooting.

Click Open Report to see a summary of the current Plan or RigCheck Result calibration. A new window will open which details the calibration success using RMS reprojection errors and other metrics. The report also shows the detailed messages from the earlier calibration progress window.

Error values below 1 pixel are considered good, and above 1px usually point to something going wrong in the process (physical setup, scene, configuration, …). The report values are displayed using a colour between Green and Red to better highlight problematic camera or projector errors. If everything is green, then the calibration was a success.

Note

Projector calibration depends on the quality of the earlier camera calibration step. Therefore any larger camera errors will likely result in either high projector errors or unreliable calibration results. This can be useful to determine at what stage in the OmniCal process an issue arises.

OmniCal Score

The OmniCal Score is a simple way to assess and compare calibration quality between Plans or RigChecks. It was introduced in r32.4 as a “single value” measure that shows whether a calibration was successful, and how good it is.

The basis of the OmniCal Score is the rounded up aggregate projector RMS error, enhanced with penalty terms for any issues that were identified during the calibration. These added penalties make the OmniCal score more useful for identifying success than standard RMS errors, which sometimes can be low despite unrealistic calibrated parameters. While the OmniCal Score is not an RMS value itself, it uses the same open-ended range, where 0 is the theoretical best value, and a value of 1 or less is considered ideal.

The OmniCal Score contains penalty terms for issues identified during the calibration, including, but not limited to:

• Projector with high RMS error.
• Projector with invalid z-clipping values.
• Projector result is mirrored or pointing backwards.
• Projector result with non-square pixels or skewed image axes.
• Projector result with unrealistic focal length, lens shift or lens distortion coefficients.

The OmniCal Score is also contained in RigCheck API responses, to make it easy for API callers to compare a result against an earlier one.

Camera Table

This table lists each OmniCal camera of the current calibration against its calculated RMS error. It also shows whether the calibration resulted in square pixels, i.e. the aspect ratio for a single pixel in the camera is 1:1. Non-square camera pixels are usually the result of incorrect input data, e.g. wrong camera focal length, incorrectly scaled Feed Rects or incorrect settings in the physical projector.

Below the table is the aggregate camera RMS error, which combines all camera errors into a single value for easier comparison between different calibrations.

Projector Table

This table lists each projector of the current calibration against its calculated RMS error. It also shows whether the calibration resulted in square pixels, i.e. the aspect ratio for a single pixel in the camera is 1:1. Non-square projector pixels are usually the result of incorrect input data, e.g. incorrectly scaled Feed Rects or incorrect settings in the physical projector.

Below the table is the aggregate projector RMS error, which combines all projector errors into a single value for easier comparison between different calibrations. The OmniCal Score is based on this value.

Calibration and Point Cloud visualisation

Once calibration is complete, you can view the point cloud and pixel errors for each projector. To do so, go to the OmniCal Calibration widget and open the Visualiser section. Set the Visualisation Source to Plan and set the Visualisation Mode to All or Only Point Clouds.

The generated point cloud will now be shown in the stage visualiser.

Note

The point cloud is drawn using the Plan’s current alignment settings, so it may not be aligned well to the stage until QuickAlign has been performed. The cloud visualisation is helpful in achieving an initial manual alignment, which will make QuickAlign easier to understand.

Point Cloud Visualisation

The Point Cloud visualisation option provides several visualisation modes that help with analysing issues such as interference from other light sources, or movement during a capture. The default and most useful mode is ProjectorReprojError, which visualises the individual reprojection errors for each projected blob after projector calibration.

Point Cloud Visualisation	Description
Off	Disable point cloud visualisation.
Observations	Draw number of camera pairs that see a cloud point. The observation count affects the 3D reconstruction: Red=0 (failed), Blue=1 (OK), Green=2+ (better quality).
InlierScore	Draw cloud point inlier category using colours. Red: invalid or outlier, Orange: outlier, Yellow: projector calibration outlier, Blue: camera calibration outlier, Green: inlier.
InliersOnly	Only draw cloud points that are inliers (Blue) after applying camera calibration classifiers such as RanSaC or LMedS.
OutliersOnly	Only draw cloud points that are outliers (Red) after applying camera calibration classifiers such as RanSaC or LMedS. Reveals unstable points, possible scene movement, mismatched blob codes etc.
Deform	Draw distance of cloud points to aligned mesh (along point normal), using Mesh Deform threshold settings.
ProjectorInliersOnly	Only draw cloud points that are inliers (Blue) after projector calibration.
ProjectorOutliersOnly	Only draw cloud points that are outliers (Red) after projector calibration.
ProjectorReprojError	Draw cloud point reprojection errors after projector calibration. The point colour indicates the error quality between Green=good, Yellow=acceptable and Red=bad/outlier. Helps identify problematic points or projection areas.

Point labels

With the Point labels option a text label can be rendered for each cloud point. The text content depends on the option value. The default value is Off.

Note that rendering cloud point labels has a significant impact on performance.

Point labels	Description
Off	Disable point labels.
GridCode	Draw x, y index that corresponds to the blob in the projector grid. This can be helpful for identifying points as part of OmniCal troubleshooting.
ProjectorReprojError	Draw cloud point reprojection errors after projector calibration. This is a single RMS error value per point and given in pixels. Lower values are better. Values larger than 1px may indicate problems with the capture, such as light interference, movement etc., especially if they appear in clusters.

Re-running calibrations

Re-run a calibration by modifying a setting and clicking Calibrate again. Note that issues introduced during image capture (lighting changes, obstructions, etc.) will require re-running Capture.

Re-running the calibration with different settings is useful:

• If the calibration quality was decent, but not yet good enough. Tweaking camera or projector calibration settings can help in some cases to get better results.
• If there were incorrect calibration settings (e.g. camera calibration was set to “Homography”, but the blobs land on non-planar surfaces).
• Simply to experiment with settings, e.g. when trouble-shooting issues.

Limitations of OmniCal

OmniCal can accurately calibrate projectors, including their particular lens distortion coefficients. Any rectilinear lens is supported, including ultra-wide-angle / ultra-short-throw lenses with throw ratios around 0.35, even if they use built-in mirrors.

However, OmniCal currently does not support fish-eye lenses.

Another minor limitation is that OmniCal cannot correct chromatic aberration in projector lenses. This is a less common phenomenon in projection (compared to photography), but it is noticeable at the corners of some ultra-short-throw lenses.