Calibration, Alignment & Mesh Deform

Calibration is the automatic process of calculating camera and projector poses and settings using the blobs detected during Capture. Calibration also results in a consistent 3D point cloud formed from all blobs.

Alignment is a user-guided process to align the calibrated data with the coordinate system in Designer. OmniCal also includes the ReShape feature to adjust 3D model vertices based on the user alignment.

Mesh Deform is an automatic process that enhances and adjusts the existing 3D model by inserting new vertices based on the aligned point cloud.

OmniCal Stage Plan

In the Stage editor, under Projectors, click OmniCal Calibration to open the Calibrator widget. This widget controls the main OmniCal behaviour.

Once you have created a Capture Plan, you can click on it to open the OmnicalStagePlan widget.

In the OmnicalStagePlan we recommend working through the sections from top to bottom: Capture, Calibration, Alignment, and Mesh Deform (if required).

Calibration

The calibration process triangulates the position of the blobs detected in a capture as well as the relative positions of cameras & projectors and their lens intrinsics. From this point, you no longer need access to the physical stage to continue the calibration process.

Click Setup Calibration… and the widget OmniCal Calibration Setup will appear.
In most cases the default settings suffice. If calibration results are unsatisfactory despite good capture quality, adjust one setting at a time to clearly assess its impact.

Note

Most calibration settings include contextual help within Designer.

Camera Calibration settings

In the Camera Calibration separator we can set the Calibration Model to one of three modes:

• Auto: Attempt to detect appropriate model. Only manually select another mode if ‘Auto’ fails.
• Epipolar: Used with scenes “with depth”, i.e. where blobs are not just projected on a flat surface, but are spread across 3 dimensions.
• Homography: Used with scenes where blobs are mostly projected on a flat surface, or the depth provided is very shallow (e.g. < 1cm).

Bundle Adjustment settings

In the Bundle Adjustment separator we can set Bundle Adjustment target to one of five modes:

• Auto
• Cameras
• Cameras and Projectors
• Cameras then Projectors

Iterations(combined) of Bundle Adjustment to run for cameras and combined cameras & projectors modes. High value will be slow for large numbers of cameras and projectors.

• Cameras then each Projector

Iterations(combined) of Bundle Adjustment to run for Cameras then each Projector modes. Fast so can be set much higher than combined iterations.

Point Cloud Reconstruction settings

A setting under Point Cloud Reconstruction that is often useful is Camera graph optimisation. This should be enabled for camera setups where all cameras overlap with each other, where they form a closed loop, i.e. the first and last camera overlap with each other. Enabling the camera graph optimisation prevents gaps or discontinuities where the point cloud boundaries meet. This will give more consistent reconstruction and calibration results for typical setups for 360-degree projections inside of or on domes, cylinders, spheres or the walls of a room.

Camera graph BA iterations controls a separate Bundle Adjustment pass for the camera graph optimisation. The default is 500, but it is rare to need more than 50 iterations. This step is much faster than later Bundle Adjustment steps that involve cameras and projectors.

Projector Calibration settings

Projector Calibration Algorithm of Projector Calibration has three modes:

• DLT: Direct Linear Transformation
• Zhang
• Auto: will try all available algorithms and select the best result.

Perform Calibration

Once you have finished adjusting the Calibration Settings, click Calibrate in the Plan editor and confirm with clicking “Yes” to start the calibration process. Depending on the number of cameras and projectors, this can take from a few seconds to tens of minutes.

Calibration and Point Cloud visualisation

Once calibration is complete, you can view the point cloud and pixel errors for each projector. To do so, go to the OmniCal Calibration Calibrator widget. In the Visualiser, set the Visualisation Source to Plan and set the Visualisation Mode to All or Only Point Clouds. Set the Point Cloud Visualisation to ProjectorReprojError to see reprojection errors for each individual blob. Observations will show for each Blob by how many cameras it was seen. These and other visualisation modes may help with analysing issues such as interference from other light sources, or movement during a capture.

The generated point cloud will appear in the stage visualiser at this point but will not be aligned to the stage, unless you previously performed an alignment.

Calibration Report

Click Open Report to see a summary report of the calibration. Error values below 1 pixel are considered good, and above 1px usually point to something going wrong in the process (physical setup, scene, configuration, …). The report values are displayed using a colour between green and red to better highlight problematic camera or projector errors. If everything is green, then the calibration was a success.

Re-running calibrations

Re-run a calibration by modifying a setting and clicking Calibrate again. Note that issues introduced during image capture (lighting changes, obstructions, etc.) will require re-running Capture.
Re-running the calibration with different settings is useful:

• If the calibration quality was decent, but not yet good enough. Tweaking camera or projector calibration settings can help in some cases to get better results.
• If there were incorrect calibration settings (e.g. camera calibration was set to “Homography”, but the blobs land on non-planar surfaces).
• Simply to experiment with settings, e.g. when trouble-shooting issues.

Limitations of OmniCal

OmniCal can accurately calibrate projectors, including their particular lens distortion coefficients. Any rectilinear lens is supported, including ultra-wide-angle / ultra-short-throw lenses with throw ratios around 0.35, even if they use built-in mirrors.

However, OmniCal currently does not support fish-eye lenses.

Another minor limitation is that OmniCal cannot correct chromatic aberration in projector lenses. This is a less common phenomenon in projection (compared to photography), but it is noticeable at the corners of some ultra-short-throw lenses.

Alignment

Alignment is the process of registering or aligning the coordinate system of the captured point cloud with that of Designer.

Perform Alignment

1. In the plan editor, roughly edit the point cloud Alignment via Position, Rotation and Scale. This does not need to be exact at all, just close enough in terms of scale and rotation that further steps are made easier. Once the point cloud is somewhat close to your projected surfaces in the visualiser, click QuickAlign to open the tool for detailed alignment. In QuickAlign you should see a reprojected wireframe of the first projection surface.

2. Once the point cloud is roughly aligned, you can perform a Quick Align. Left-click Quick Align to open the Quick Align editor. The top two views relate to cameras which can be chosen from the view tab. The bottom two views relate to the cursor location in the camera view. This is essentially a zoomed in view, for better view finding.

3. There are some automatic options to automatically align the point cloud to the stage. Click Quick Align, and choose Alignment Estimate which is automatic alignment estimation based on point cloud and all projection surfaces.

   Align to plan cameras which is the automatic alignment based on positions of plan cameras.

   Align to plan which aligns to plan cameras and projectors.

   Align to point cloud is useful for alignment using 2D camera reference points. Use this option when you want to apply the current alignment data to the stage.

4. If the Alignment assistance tools don’t provide a good enough alignment, create manual alignment points and line them up in multiple cameras. At least 3 alignment points need to be lined up in at least 2 cameras each. It’s best to edit the alignment points one by one in each camera view. Alignment points are best placed on mesh vertices that correspond to distinct features in the physical model (e.g. a corner of a building). Not every camera will see every alignment point, so additional points may be needed.

5. Create AlignmentPoints (AP). Click near a point on the wireframe to create an AP for this mesh vertex. Alternatively, AlignmentPoints can be imported from existing QuickCal projector ReferencePoints.

6. Drag it to the corresponding point in the image. Do the same for the corresponding point in the second camera view.

7. Repeat this process for a minimum of three points.

   1. A Red point means this point is not being used as part of the alignment, but has been added to the view.
   2. A Yellow point means it has been aligned in the current camera, but it is not being used in the calculations.
   3. A Orange point means it has been aligned in a different camera.
   4. A Green point means it has been aligned in both cameras and is being used in the calculation.
   5. If selected points flash, and you can use the arrow keys to move them around.
   6. When points are selected, use SHIFT + arrow keys to move the point around & hold CTRL & left-click for fine control.

Multi-screen Alignment

If there is more than one screen in the plan, it may be helpful to use Multi-screen alignment. This allows aligning surfaces in the stage based on the captured images and a common master screen. If the relative offset and rotation between all surfaces in the stage matches the relationships of the actual physical surfaces, then this may not be needed.

• The master screen should be aligned first.
• The master screen acts as a registration base for all other screens. Reshaping or scaling of the master screen will therefore affect all other screens.
• When aligning secondary screens, they will be moved to the correct position relative the Master screen.
• When stage automation is used to control physical screen positions, it is best to choose a non-moving surface as the master screen.

An example for how to align multiple surfaces can be seen in the demo video from the Overview page, at around 2:40 minutes.

Alignment Re-shape

If the proportions of the model match the physical surface exactly, then the alignment should fit perfectly. But if the proportions are not correct, you may need to perform Alignment re-shape. Reshape allows adjustments to mesh vertices based on lined-up alignment points. This can be useful for adjustments such as stretching or squeezing the projection surface, or even moving some internal mesh elements (such as a window) by a certain amount.

Enable Re-shape

1. To enable Re-shape for an alignment point hold SHIFT and left-click it. This will turn it into a reshape point which is indicated by a diamond marker (45 degree rotated).
2. Left-click the point again (without holding SHIFT) to turn it back to a non-reshaped alignment point. The reshape process does not use the captured point cloud, but simply adjusts mesh vertices and mesh proportions according to the calibrated cameras, captured images and alignment point data provided by the user.

An example for how to use reshape points can be seen in the demo video from the Overview page, at around 4:24 minutes.

Mesh Deform

This may take a while, depending on mesh size, number of vertices and point cloud size.

If the depth of the mesh in does not match the real world object, it will require mesh deforming. Mesh deform will deform the mesh to match the point cloud.

It is a good idea to try the default settings for mesh deform first, to see if you get a good deform. If not, proceed to tweak the settings as required.

You can quickly preview Mesh deform results by changing the Point Cloud visualisation mode to Deform in the calibration editor. The results are updated in real-time so you can try out different deform settings using this. The lines indicate where each point on the mesh would be moved to.

Note

You must have added re-shape points to the screens for Mesh deform to work.

Perform Mesh Deform

1. Click Mesh deform from the Alignment tab of the Plan editor.
2. Select the screen you wish to deform by left-clicking the screen property in the deform editor.
3. Click Deform.