Quick Calibration

This sub-chapter explains the process of using QuickCal to line up projectors to real world screens.

Overview

QuickCal is a user-driven process of calibrating a virtual projector’s position, orientation and lens properties to match the real-world projector. QuickCal is based on reference points on given 3D meshes for a projector’s target surfaces, and user-defined 2D image coordinates that are manually assigned to these 3D reference points. As long as the 3D meshes are good matches to the real-world projection surfaces, QuickCal can accurately calculate the projector parameters.

The basic process of QuickCal is relatively simple : drop reference points onto the 3D model of the projection surface, and then ‘line up’ by dragging each point in the projector’s output raster until it hits the corresponding point on the real surface. Once you’ve done enough points (about 10-15 per projector) Designer can work out exactly where the projector is, and what lens qualities it has.

3D mesh accuracy

When calibrating projectors using QuickCal it is crucial that the 3D mesh object is modelled accurately to the physical model. The reason for this is that Disguise’s calibration algorithm assumes that the virtual reference points link exactly to their corresponding real-world points. The best way to generate a highly accurate 3D model is to laser scan the physical model, or to laser cut the physical model based on the same 3D file you later use in Designer. For building projections we recommend a mesh with a +-5mm error margin.

Using QuickCal

To start using QuickCal, first move the virtual projector into roughly the right position and orientation, and set up its lens qualities to roughly match the real projector. You don’t have to be particularly accurate in this, but when you’re trying to select projectors in the visualiser for editing, it’s easier when they’re in roughly the right place. Then move on to creating reference points, and then line them up.

Tips & Tricks

General QuickCal background information

The calibration goal is to calculate a projector’s projection matrix so that it produces physically valid projections on its assigned projection surfaces in the real world (the scene). This means that the matrix does not need to be absolutely physically correct, but should be correct for at least this projection area (or rather volume). Indeed, it helps to think of the calibration as a process that works on a volume in space, rather than on 2-dimensional surfaces.

The calibration process uses as main inputs:

3D Reference Points (in world space)

These are usually created from vertices of the mesh/object. Note that the mesh needs to match the real object, at least in the vertices used for creating Reference Points. In d3 Reference Points are 3D objects themselves, and a certain point can be used by several Projectors.

2D coordinates (in the projector image plane)

These are created during line up of a Reference Point in a given Projector. In Designer the Projector Configuration stores a pair that links the 2D coordinate to the 3D Reference Point.

Projector matrix

The projection matrix is a linear transformation that represents the projector’s position, orientation and internal sensor and lens properties. Note that Designer currently does not support correction of non-linear lens distortion in QuickCal. Lens distortion is only calculated and compensated for in OmniCal.

In Designer the projection matrix is applied to all 3D meshes in the Surfaces list, so they get transformed and projected on to the virtual Projector’s image plane. This creates the projector’s output image that is visible in the Feed Scene.

Choice of Reference Points and Order of line-ups

A Reference Point and the 2D line-up coordinate together form a correspondence point pair. Several such pairs are required for a calibration. QuickCal supports several algorithms that have different requirements on the minimum number of points (e.g. 4, 6, or more).

It doesn’t matter for the calibration itself which object/mesh the Reference Points come from, as long as that 3D point exists in the real world as a recognisable feature. Object corners are usually good choices.

It is possible to use Reference Points from several objects for a single Projector.

Important requirements are:

• The real-world equivalents of the Reference Points are actually visible by the physical projector (lie inside its light cone).

• All Reference Points that are used for a projector:

  • span a large enough 3D volume. Often this is referred to as “adding enough depth”. The main issue is to ensure that the Reference Points do not all lie on the same (virtual) plane in 3D space.
  • should evenly cover a large enough area in the Projector’s image. It’s a good idea to use Reference Points that get lined-up in all four corners of the projector output image.

Additionally, if you want to verify the wireframe of an object during QuickCal line-up, then the objects needs to be added to the Projector’s Surfaces list.

It is worth to keep in mind that calibrating creates a working volume for a projector, using the 3D Reference Points as a helper. So the volume in which the projector “is valid in” is always limited by the extent of Reference Points you have lined up. As an example, lining up 6 points that are all in the same image corner of the projector will likely produce a calibrated matrix that doesn’t work well for the rest of the image. The same goes for points that are in close proximity in the real world, i.e. the calibrated matrix will only be valid in the small volume within those points.

The order in which Reference Points are created and lined up does not affect the calculation of the projection matrix. The calibration will produce the same result no matter what points are lined up first.

However, to help the calibration work out things faster, it makes sense to line up Reference Points from extreme parts of the scene first, before adding points that are closer to each other. Otherwise there may not be a good visual output until enough points are lined up so that they cover the whole scene with enough depth.

Short QuickCal Rules

1. leave the Calibration Method setting at Auto, unless you really have a good reason to force a specific algorithm. Auto will choose the best algorithm based on how many Reference Points you have, and what their 3D relationship is.
2. Don’t “cheat” with your line-up coordinates unless you absolutely have to.
3. Never “cheat” until all Reference Points have been added and lined-up.
4. Line up the extreme parts of the mesh(es) and projector image first (e.g. image corners Left/Right/Top/Bottom and Close/Far in the scene). Then improve on that by adding more points. E.g. concentrate on Reference Points in areas where the projected images are still “off” when looking at the real object.

Limited Scene Depth and number of Reference Points

Meshes/Objects that are flat or have little depth are more difficult to use for calibration. This is even more problematic if the real world object has few or no visible features, like a wall where only 4 corners are easily usable as Reference Points.

However, with 4 correspondence point pairs it’s already possible to do a simple calibration (usually IterLM algorithm). This calibration is not able to calculate internal lens parameters such as lens shift or throw ratio. It will take values for these parameters from the manual projector settings. Note that these settings are still editable when using QuickCal with under 6 points, or if certain algorithms are set. It can be difficult to get these values right by hand, but if the throw ratio and lens shift of the physical projector is roughly known, then adjusting this can help improve the calibration significantly.

With 6 or more correspondence point pairs a different algorithm is used (usually Zhang algorithm), which also calculates lens shift and throw ratio parameters automatically. If the Reference Points provide little or no depth, then this algorithm has problems finding a good projector matrix. This is because mathematically it is very difficult to differentiate between some internal lens parameters and 3D parameters. For example throw ratio (or focal length, if you prefer) and distance to an object are interchangeable to a certain degree. Think of the famous Dolly Zoom effect used by Hitchcock in the film “Vertigo”, where the face always stays the same size, only fore- and background objects get smaller or disappear. This ambiguity can only be resolved by more depth. It is necessary to line up more points from the scene background/foreground to constrain the calibration better, essentially thereby increasing the calibration volume.

In practice, it may be preferable in certain rare situations to stick with 4 or 5 points, rather than adding a 6th, because the switch to the different algorithm may make the visual result worse rather than better.

Correspondence Point influence on calibration

A correspondence point pair is not like guide points for splines, or a fixed warp or rule that associates the two to each other.

Instead, they are inputs into an iterative algorithm that tries to solve a matrix so that all of these point pair conditions are (more or less) satisfied. But due to mesh/reality discrepancies, line-up inaccuracies, constraints of the optical model, even rounding errors, etc. there will never be a perfect matrix that fits all these pairs. There are many matrices though that will be good enough, so that the error for each of these point pairs is small (e.g. ideally below 1 pixel). If the inaccuracies in the input data are high, then some correspondence point pairs will basically contradict each other strongly. In that case the overall error will be high, too (e.g. 5-10 pixels, or even more). If the error is due to mesh/object discrepancies, then “cheating” might help (see below).

Cheating

If the 3D model and the physical object don’t match exactly, then QuickCal allows to “cheat” a bit. “Cheating” isn’t recommended, but often it’s not possible to get a better 3D model while being on-site.

There are several ways to “cheat”:

• Moving the line-up cursor slightly off from where it should be, so that the overall calibration looks better and the reprojection error gets smaller.

Note

The advanced option ‘Disable reference point bounds’ allows moving the 2D line-up coordinate outside of the projector’s image, e.g. to a negative value.

• Moving a 3D Reference Point away from its original vertex position. In QuickCal this can be done in “Manage” mode by holding the Shift key and moving the 3D Reference Point left, right, up or down. This will move the point in 3D space according to the current visualiser view, and allows placing the point in a non-vertex position. By default, Reference Points snap to the nearest mesh vertex, indicated by a green cross.

The problem with any of these tricks is to know which Reference Points to cheat with, and how it eventually affects the calibration.