Skip to content

Mosaicing_Digital_Images

Jump to bottom
John Cupitt edited this page Mar 29, 2017 · 1 revision

title: Mosaicing Digital Images permalink: /Mosaicing_Digital_Images/

Origin

This page has been written as part of the NIP2 Beginners Guide.

Introduction

The process of mosaicing images can be separated out into four main steps:

  1. Capturing or generating the images.
  2. Preparing the images for mosaicing.
  3. The actual process of making the joins to assemble the image.
  4. Balancing out tonal difference between all of the separate images.

This section will discuss some of the typical image preparation steps, the actual mosaicing process and final balancing stage.

Image preparation.

sec:im_prep In order to simplify the mosaicing procedure and allow the use of the balancing functions[#note44\ 1] it is recommended that any additional image processing required to prepare the images is perform first and a complete set of prepared images is generated and saved before they are put together. Image preparation procedures often include but are not limited to the following processes:

Calibration: For more information relating to image calibration see section [#sec:calibration 3.4]sec:calibration, further details on how image can be calibrated with nip2 can be found in the standard nip2 user guide.

Cropping: Images may need to be cropped to remove borders, extra information around scanned objects, or even to minimise the size of images with very large overlaps[#note45\ 2]. [Main Window][Toolkits: Image:Crop], for further details see section [#sec:crop 5.9.3]sec:crop.

Rotation: When images are scanned from transparencies, slides, X-ray plates, etc. it is often possible for some to have been rotated incorrectly during scanning. [Main Window][Toolkits: Image: Transform: Rotate: ... ], for further details section [#sec:rotate 5.9.1]sec:rotate.

Flipping: Again when objects are scanned it is often possible for some to have been scanned upsidedown. [Main Window][Toolkits: Image: Transform: Rotate: Flip: ... ], for further details section [#sec:flip 5.9.2]sec:flip.

Resize : Individual images can have been produced at a very high resolution for archiving purposes. However when it comes to mosaicing large groups of these it may not be advisable to mosaicing them at full resolution due to the limitations of your computer hardware[#note46\ 3]. [Main Window][Toolkits: Image: Transform: Resize: ... ], for further details section [#sec:nipresize 5.9.4]sec:nipresize.

Image Manipulation: Depending on how the images where produced it may be necessary to edit out dust and scratches or even remove images identification tags, such as the lead number that can be used to identify individual X-ray plates when they are processed, see section [#sec:clone 5.11]sec:clone.

Mosaicing Images

sec:mosaicing nip2 has a selection of functions designed to help join many small images together. They are separated into functions that require a single tie-point to mosaic images and those that require two tie-points.

Which function to use?

Single tie-point functions: For simple mosaicing, where both images are known to be the same scale and do not need to be rotated, only one pair of tie points is required. These functions are commonly used to mosaic sets of simple images, such as IR-reflectograms, that have been captured with controlled image capture systems[#note47\ 4]. Once a pair of tie-points have been fixed, one on each image, the software will move one image, left-right and/or up-down, with respect to the other until the selected tie-points overlap. This kind of movement is known as translation or a first order transformation.

Two tie-point functions: For more complex mosaicing procedures, where images may need to be resized or rotated, two pairs of tie points are required. These functions are commonly used to mosaic sets of more complex images such as scanned X-ray plates, which where positioned and organised by eye. Once a two pairs of tie-points have been fixed, two on each image, the software will move, resize and/or rotate one image with respect to the other until both pairs of selected tie-points overlap. This kind of movement is known as an affine or second order transformation.

One point mosaicing, an example of basic mosaicing of IR images.

nip2 includes a set of simple infrared images you can use to practise assembly using the . CLMB on the [Open] button to get an open file dialog (see section [#sec:load 5.3]sec:load for an explanation of all the features of the open file window). DLMB on the second or lower data directory listed in the left hand column to enter nip2's main data directory. The Examples directory should appear in the directories list. DLMB on it and then DLMB on 1_point_mosaic directory as it appears. The 1_point_mosaic directory contains a set of eight example infra-read images and a complete workspace showing how they are mosaiced together. Select all of the jpeg images and CLMB on [OK], figure [#fg:joins 6.1]fg:joins shows how these images fit together.

[Image:nipbeginguide022.png](/Image:nipbeginguide022.png "wikilink")
Figure 6.1: Schematic diagram showing how the sample IR images join up.
fg:joins

Mosaics are assembled by repeatedly joining pairs of images. Open image view windows for cd1.1.jpg and cd1.2.jpg[#note48\ 5], and position them side-by-side on the screen, see figure [#fg:joins2 6.2]fg:joins2. You should be able to make out the overlapping area.

[Image:nipbeginguide023.png](/Image:nipbeginguide023.png "wikilink")
Figure 6.2: Ready to join cd1.1.jpg and cd1.2.jpg
fg:joins2

Place a point on each image[#note49\ 6] on a feature you can see in both images, see figure [#fg:joins3 6.3]fg:joins3. You don't need to be exact: nip2 just uses the point you select as the start point for a search[#note50\ 7], but you can adjust the position of the points if required. You can use the display control bar to brighten the image if the overlap is very dark, see section [#sec:im_view 5.6]sec:im_view.

[Image:nipbeginguide024.png](/Image:nipbeginguide024.png "wikilink")
Figure 6.3: Marking a tie-point
fg:joins3

Join the two images together use the function:

  • [Main Window][Toolkits: Tasks: Mosaic: One Point: Left to Right]

nip2 will use the two points you marked to determine the approximate overlap, and then search for the best way to join the two images and then enter the joined image as a new object in your current column. Open a viewer for the new image, and check that it joined them together correctly, see figure [#fg:joins4 6.4]fg:joins4[#note51\ 8].

[Image:nipbeginguide025.png](/Image:nipbeginguide025.png "wikilink")
Figure 6.4: Joining cd1.1.jpg and cd1.2.jpg
fg:joins4

Try moving one of your regions slightly. You should find that the output image does not change. Despite the error in selection, nip2 was still able to find the correct overlap. If you try moving one of them a long way suddenly, the output image will jump and will obviously mismatch. nip2 has just found the best overlap available within the 10 pixel margin of error. Delete the joined image[#note52\ 9], and use the following function instead:

  • [Main Window][Toolkits: Tasks: Mosaic: One Point: Manual Left to Right]

This functions operate in the same way as the previous mosaic function, but does not do a search. Try moving the tie-points again and the joined image will change every time you move a point. The manual mosaic functions are useful when the overlap is too small for the search to work correctly, or when the overlap area is very smooth and contains too few features for the search to find the exact overlap for you. Delete the result of the manual process and use the first mosaic function again. Now pop up viewers for the next pair, cd2.1.jpg and cd2.2.jpg, and join them too. Pop up viewers for the two rows, and position them one above the the other. Again, you should be able to see the overlapping area and be able to place a pair of tie-points, then run the following function:

  • [Main Window][Toolkits: Tasks: Mosaic: One Point: Top to Bottom]

Mosaicing proceeds in this manner: you make repeated left-right joins to assemble each row (or repeated top-bottom joins, if the data was grabbed in columns). When two rows have been assembled, you then make a top-bottom join to link the rows together. Eventually, you will have assembled the whole mosaic[#note53\ 10], see figure [#fg:all 6.5]fg:all.

[Image:nipbeginguide026.png](/Image:nipbeginguide026.png "wikilink")
Figure 6.5: The completed mosaic of the sample IR images.
fg:all

Two point mosaicing, an example of more complex mosaicing of X-ray images.

nip2 includes a set of mock X-ray images you can use to practise two point mosaicing with[#note54\ 11]. CLMB on the [Open] button to get an open file dialog[#note55\ 12]. DLMB on the second or lower data directory listed in the left hand column to enter nip2's main data directory. The Examples directory should appear in the directories list. DLMB on it and then DLMB on 2_point_mosaic directory as it appears. The 2_point_mosaic directory contains a set of six mock x-ray images, a copy of the finished mosaic and a complete workspace. Select the six example* images and CLMB on [OK], figure [#fg:joins2 6.2]fg:joins2 shows how these images fit together.

[Image:nipbeginguide027.png](/Image:nipbeginguide027.png "wikilink")
Figure 6.6: Schematic diagram showing how the mock X-ray images join up.
fg:2ptpos

Mosaics are assembled by repeatedly joining pairs of images. Open image view windows[#note56\ 13] for example_im_01.jpg and example_im_02.jpg and position them side-by-side on the screen. You should be able to make out the overlapping area. Place a point on each image[#note57\ 14] on a feature you can see in both images, see figure [#fg:joins3 6.3]fg:joins3. You don't need to be exact: nip2 just uses the point you select as the start point for a search[#note58\ 15], but you can adjust the position of the points if required. You can use the display control bar to brighten the image if the overlap is very dark, see section [#sec:im_view 5.6]sec:im_view.

[Image:nipbeginguide028.png](/Image:nipbeginguide028.png "wikilink")
Figure 6.7: Two pairs of tiepoints selected for images one and two.
fg:2pt_pts

Join the two images together use the function:

  • [Main Window][Toolkits: Tasks: Mosaic: Two Point: Left to Right]

nip2 will use the four points you marked to determine the approximate overlap, rotation and scale, and then refine them to produce the joined image as a new object in your current column. Open a viewer for the new image, and check that it joined them together correctly. Now open up viewers for the next pair, example_im_03.jpg and example_im_04.jpg, and join them too. Pop up viewers for the two rows, and position them one above the the other. Again, you should be able to see the overlapping area and be able to place a two pairs of tie-points, then run the following function:

  • [Main Window][Toolkits: Tasks: Mosaic: Two Point: Top to Bottom]

Mosaicing proceeds in this manner: you make repeated left-right joins to assemble each row (or repeated top-bottom joins, if the data was grabbed in columns). When two rows have been assembled, you then make a top-bottom join to link the rows together. Eventually, you will have assembled the whole mosaic[#note59\ 16], see figure [#fg:2pt_complete 6.8]fg:2pt_complete.

[Image:nipbeginguide029.png](/Image:nipbeginguide029.png "wikilink")
Figure 6.8: The completed mosaic of the mock X-ray images.
fg:2pt_complete

Assembly hints and tips

Mosaic assembly is normally very easy and painless. There are a few factors you should bear in mind when you are deciding how to assemble an image (particularly a large image):

  • You can open up a mosaic image object[#note60\ 17] and change a few options, such as the blend width. If you want to change the defaults for a whole workspace, change the Mosaic defaults options in your Preferences.
  • nip2 does not do sub-pixel interpolation. As a result, each join will on average cause a positioning error of about 0.5 pixels, even if your input images contain no geometric distortion. If there are distortions in your input images (there usually are distortions in infrared images), then errors can be as much as 1–2 pixels per join. These errors accumulate as the number of images you join becomes larger. If you join a strip of 10 images together you can expect a total error of about 5 pixels. If you join two strips like this together top-bottom, you can therefore expect a mismatch of about 2.5 pixels at each end of the join. You can minimise the effect of these errors if you assemble your images differently. Suppose you have a 10 by 10 mosaic to build. Instead of making and joining 10 strips of 10 images each, make four 5 by 5 sub-mosaics (one for each quadrant) and then join these four quadrants together. The errors will now be more evenly spread over the image and therefore will be less visible.
  • sec:pieces Some operating systems limit the number of files a program can have open at once: this in turn limits the size of the mosaics you can assemble in one go. If you are having problems putting together very large mosaics, try building your image in sections (top, middle and bottom, perhaps), and later load up and join these larger pieces.

Other Mosaic related functions.

Mosaic Balancing

sec:balance If your camera, capture system ,etc. is not perfectly stable you will find your assembled mosaic will have a nasty `blocky' appearance as the tonality between sub-images varies. nip2 has some functions for fixing this. Select your assembled mosaic, and CLMB on:

  • [Main Window][Task: Mosaic: Mosaic Balance]

After a few seconds, (to quite a few minutes depending on the size of your mosaic), a new, balanced image should appear, see figure [#fg:balance 6.9]fg:balance[#note61\ 18].

[Image:nipbeginguide030.png](/Image:nipbeginguide030.png "wikilink")
Figure 6.9: The complete mosaic of the smaple IR images before and after the balancing process.
fg:balance

Balancing troubleshooting

Like assembly, mosaic balancing is normally automatic and painless. You may sometimes have problems if the image is very large, or needs dramatic corrections:

  • Each nip2 image file has an associated history, recording the operations on that image since it was loaded from a file. You can view the history of an image by CLMB on [Image view][View: Image header]. The automatic balancer uses the history to work out how you built your mosaic. The balancer knows about left-right and top-bottom joins, but not anything else! If the history has other stuff recorded in there, you'll see unhelpful error messages like unable to open tmp/xxx.v, or more than one root. If you need to perform corrections to any of your sub-images, do them, save the image, load it again, and then build the mosaic. This will make sure the history of the image you are trying to balance only contains mosaic operations.
  • On some systems the balancer can run out of memory or out of file descriptors on very large mosaics. If your mosaic is made up of more than a few hundred images, and you are having balancing problems you may have hit one of these limits. The solution (as with mosaic assembly) is to assemble and balance your mosaic in smaller pieces.
  • The balancer outputs a 32-bit float image. If you save this directly you may have problems viewing it in other programs. You can solve this problem by converting it to more standard 8-bit, before you save it, using:
    • [Main Window][Image: Number Format: 8 bit unsigned]
  • If your grey-card correction is not accurate, you will find that the balancer will magnify any problems you have. Suppose your lighting and camera set-up always produces images which are brighter on the right than the left, and suppose, due to some problem with your grey-card correction, this effect is not completely removed. You will find that when you balance a mosaic, the small differences between left and right edges of your sub-images will have been smoothed out, but they will have caused a large difference in brightness between the extreme left edge of your final image and the extreme right. nip2 includes several functions which can help to fix this problem, the most commonly used being:
    • [Main Window][Task: Mosaic: Tilt Brightness: Left to Right]
    • [Main Window][Task: Mosaic: Tilt Brightness: Top to Bottom]

Mosaic Rebuild.

  • [Main Window][Tasks: Mosaic: Rebuild]
[Image:nipbeginguide031.png](/Image:nipbeginguide031.png "wikilink")
Figure 6.10: An example column entry for the **Rebuild** function.
fg:rebuild

If an object is imaged several times, at different exposures or wavelengths, several sets of very similars images will be produced. The Rebuild function allows the user to mosaic the first set of images and the use all of the tie-points defined for the first set of images to automatically mosiac other sets. The function provides the user with two options, old and new, see figure [#fg:rebuild 6.10]fg:rebuild. For example:

  • First set of images:set_one_01.jpg to set_one_xx.jpg.
  • Second set of images: set_two_01.jpg to set_two_xx.jpg.
  • The first set of images would be loaded into nip2 and then mosaiced together as described.
  • The finsihed mosaic would be selected and then the Rebuild function run.
  • In the Rebuild object, see freffg:rebuild, the old option would be edited to read one and the new option would be edited to read two and then hit [Return].
  • nip2 will then process all of the second set of images to produce a completed mosaic based on the tie-points defined for the first set.

Printing

Once you have assembled a good mosaic, you will want to print it, or to use it in other computer programs. The best way to do this is to save the final image in TIFF or JPEG format[#note62\ 19], and then load it into the new application. There are a couple of points to bear in mind: first, like any image, mosaics look best on paper if you sharpen them up a little first.

  • [Main Window][Filter: Convloution: Sharpen]

A special sharpening function has been developed to process IR images produced with a vidicon camera:

  • [Main Window][Filter: Convloution: Custom Convolution]
  • CRMB on the matrix button, select Replace from file. DLMB on the second or lower data directory listed in the left hand column to enter nip2's main data directory. Change the Image type select option to All Files (*) and then select and load rachel.con.
  • This will usually produce an appropriately sharpened reflectogram.

Secondly, you will need to try several prints with different contrasts and brightnesses to get a good match between the paper and the screen, try

  • [Main Window][Image: Levels: Linear]
  • OR
  • [Main Window][Image: Levels: Power]

Finally, you may not need a full resolution image. For almost all printers there's no point going over about 300 dpi (dots per inch), or about 3000 by 2000 pixels for an A4 page. To reduce the size of an image, use one of the functions listed under[#note63\ 20]:

  • [Main Window][Image: Transform: Resize]

[1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20]

References

[1] Currently the mosaic balancing functions will only work on images produced just using the other mosaicing functions. All other image processing needs to be completed in the image preparation stage.

[2] nip2 is able to mosaic images with large overlaps, but if you are working with very large sets of images or just very large images it may speed up the mosaicing and balancing procedures if you crop your images. If you are unsure then don't crop the images, it is always better to have to much overlap than not enough.

[3] For example, X-ray plates scanned at 600dpi to produce mono, 16-bit, images can produce files greater than 100MB, mosaicing 50 of these together could produce a very large 2-3GB image. For more information relating to dpi, bit-depth, etc see chapter [#chap:dig_im 3]chap:dig_im

[4] In this context a controlled system can be described as a system where a painting is fixed in position and the camera is then positioned perpendicular to the painting, and then moved in a controlled manner, staying in plane and perpendicular to the painting. It is also possible to fixed the camera in position and move the painting in a controlled manner.

[5] see section [#sec:im_view 5.6]sec:im_view

[6] see section [#sec:im_view 5.6]sec:im_view for more information regarding making and moving points.

[7] nip2 can cope with misses of up to about 10 pixels.

[8] See the manual page for im_lrmosaic(3) if you are interested in a technical description of the method used by the mosaic functions to find the exact overlap.

[9] CRMB on the thumbnail and select Delete

[10] There is a workspace file, $VIPSHOME/share/nip2/data/ examples/1_point_mosaic/1pt_mosaic.ws, which you can load to mosaic all the images.

[11] The mock X-ray images that have been included are all 8-bit images, however the function will also work with 16-bit scanned X-rays.

[12] see section [#sec:load 5.3]sec:load

[13] see section [#sec:im_view 5.6]sec:im_view

[14] see section [#sec:im_view 5.6]sec:im_view for more information regarding making and moving points.

[15] nip2 can cope with misses of up to about 10 pixels.

[16] There is a workspace file, $VIPSHOME/share/nip2/data /examples/2_point_mosaic/2pt_mosaic.ws, which you can load to mosaic all the images.

[17] see section [#sec:complexobjects 5.2.3]sec:complexobjects

[18] See the VIPS manual page for im_global_mosaic(3) for a technical explanation of how automatic balancing works.

[19] see section [#sec:save 5.4]sec:save

[20] see section [#sec:nipresize 5.9.4]sec:nipresize

Build for Windows
Build for macOS
Build for OpenEmbedded
Build for Ubuntu
Build for Cygwin

Debugging
Hello World

HOWTO----Image-shrinking
Projects using libvips

Speed and memory use
Benchmarks

Clone this wiki locally