Registration and Mosaicing
This document corresponds to the registration and mosaicing example folder within a VIAME desktop installation. This directory stores assorted scripts for performing registration and mosaicing, either across an image sequence with a certain amount of overlap between frames, or across modalities (e.g. optical and thermal imagery) for more specialized use cases.
Mosaic generation
The generate_mosaic_for_list script shows the simplest way to
generate a mosaic. The main program it invokes, create_mosaic.py,
also supports additional options and functionality.
A basic invocation of create_mosaic.py is as follows (assuming
setup_viame.sh or setup_viame.bat has been run):
create_mosaic.py --step 1 mosaic.jpg homographies.txt image_list.txt
This generates a mosaic image named mosaic.jpg from a file
containing homographies, here homographies.txt, and a file listing
images, one per line, here image_list.txt. The homography file
can be generated using one of the stabilization pipelines, as is done
in the generate_mosaic_for_list script.
The --step option controls what fraction of the input frames are
drawn in the output. --step 1, as above, will draw every frame,
--step 2 will draw every other frame, --step 3 will draw every
third frame, and so on. Drawing fewer frames will make the process go
faster, but drawing too few frames can create gaps.
Three other options are available for frame selection. --frames N
draws N regularly spaced frames. For example, --frames 2 would
only draw the first and last frame while --frames 3 would also
draw the middle frame. Either --step or --frames must be
used, but not both. --start and --stop can be used,
individually or together, to draw only frames from a particular range.
For example, with --start 3 --stop 8, only frames 3, 4, 5, 6, and
7 will be considered. Note that frames are counted starting from 0
and that the value passed to --stop is excluded. --step and
--frames are considered relative to any range specified using
--start and --stop. --start and --stop are necessary
when a homography sequence has a “break” where the reference frame
(the last number of each line in a homography file) changes; all
selected frames must have the same reference frame.
Here are some examples of how the frame selections options affect which frames are drawn from a 10-frame sequence:
--step 1: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9--step 2: 0, 2, 4, 6, 8--frames 2: 0, 9--frames 4: 0, 3, 6, 9--start 3: 3, 4, 5, 6, 7, 8, 9--stop 8: 0, 1, 2, 3, 4, 5, 6, 7--start 3 --stop 8: 3, 4, 5, 6, 7--start 3 --step 2: 3, 5, 7, 9--start 3 --frames 4: 3, 5, 7, 9
Several options control the drawing process itself. --zoom Z
scales the output image by a factor of Z. For example, if after
some invocation create_mosaic.py mosaic.jpg ..., mosaic.jpg
would be 20,000 x 10,000 pixels, then create_mosaic.py -Z 0.25
mosaic.jpg ... would result in mosaic.jpg being approximately
5,000 x 2,500 pixels (it might not be exactly that due to rounding of
image sizes). If the full-resolution image is not needed, then
passing a value less than 1 will reduce memory consumption and also
make the creation process faster.
The --reverse option will draw the images in the opposite of their
usual order. Normally, later frames are drawn on top of earlier ones.
With --reverse, the initial frames will be on top instead.
Generating two versions of a mosaic, one using --reverse and the
other not, can be useful in evaluating how well the homographies align
the images.
The last option, --optimize-fit, applies an extra homography to
the output that, when combined with each selected homography, attempts
to minimize the overall distortion of the images in the output,
keeping them near their original size and shape. Without this option,
aside from the limited effects of --zoom, images are transformed
exactly as described in the input homography file, except for a global
translation to keep the rendered mosaic in bounds. This option is not
guaranteed to always compute the same transformation for a given
input, but in practice the result is usually indistinguishable.
If you have coregistered image sequences, e.g. from a multi-camera
platform, create_mosaic.py can also handle that. The basic form
is:
create_mosaic.py --step 1 mosaic.jpg homogs1.txt images1.txt homogs2.txt images2.txt
That is, the homography files and image lists associated with
additional sequences are added in alternating fashion. Appropriate
homography files are for instance produced by the
suppressor_sea_lion_3-cam pipeline, or anything using
many_image_stabilizer. All the previous options still apply, but
note that frame selection applies individually to each sequence. Thus
passing --step 2 --stop 6 instead of --step 1 above would draw
images 0, 2, and 4 from the first sequence as well as images 0, 2, and
4 from the second sequence. The order of drawing in this case is
(sequence 1) 0, (sequence 2) 0, (sequence 1) 2, (sequence 2) 2,
(sequence 1) 4, (sequence 2) 4.
Build Requirements
These are the build flags required to run this example, if building from the source.
In the pre-built binaries OpenCV is enabled by default, though not ITK which is required for cross-modality registration.