Windows on history

Did you ever see ...

This excellent blog post that displays how Russian photographer Sergey Larenkov uses Photoshop to merge old and new photos to produce some really compelling images that help to bring history alive?

Well, it got me wondering how tricky it must be to get exactly the right point from which to take the modern version of the photo, not knowing exactly where the original photographer was standing, what lens he used, what focal length, zoom etc. So it's actually quite difficult to get two suitable images for merging..... even if you can afford Photoshop (I can't) ....

So, given that writing software in general, and image processing in particular are what I do in my day job, I knocked up a - hopefully - simple-to-use application that not only corrects for slight differences in perspective, but also rescales images as necessary and provides an easy way to do the merging. As an example, taking a historical picture from the online archives of my home village ...

... together with the present day equivalent, taken by yours truly on a cold Sunday morning ....

... I was easily able to produce this merged composite ....

... which is not a bad result, if I say so myself.

Go on then, give us the software ....

Ok, if you have Windows 7 or Windows 8, or 8.1, you can download the installer from here and give it a bash. No guarantees, no warranty, etc. this is all an alpha release and subject to bugs, problems, incompatibilities, whatever, but if you want it, here's how you do it, using the above images as an example ...

Image merging the easy way ...

First off, you need at least two images. I say "at least" because you can merge multiple times and have an infinite number of them, but you can't have less than two. You just can't.

The images should be of the same scene, and should be taken from roughly the same position but do not need to be the same size or format, the program will take care of resizing and mapping one image onto the other to make it easier to merge them together. 

So, when you start the application, it looks something like this ...

The user interface is similar to Microsoft Office, with a ribbon at the top and some shortcut options for loading, saving etc. at the top left. Everything is controlled by the ribbon options, moving from left to right. Different options will be enabled/disabled as you proceed, so you really can't do anything inappropriate. Well, not with this software anyway ...

So basically, you follow these steps:

• Load two images – using the “Select images” panel on the ribbon
• Mark corresponding points – using the “Mark and merge images” panel
• Paint areas where you want the secondary image to show through – using the “Composite image” panel
• Finally, adjust the blur and save the result – using the “Save results” panel

These steps are described in detail next ....

Select the images

The images are arbitrarily called “primary” and “secondary”. The primary image is the one that the bulk of the content is expected to come from and will form the basis of the final result. The secondary image is the one from which selected areas will be merged into the primary image.

So, click the Primary and Secondary buttons - in any order - to load two images ...

You can pan and zoom each image and can adjust the vertical splitter bar to adjust how much of each image you see. Use the mouse scroll-wheel to zoom in or out and just click and drag with the left mouse button to pan the images. Each image can be panned/zoomed independently.

The next step is to mark a number of reference points on each image so that they can be correctly registered. Moving on …

Mark and merge

On either of the images, find a suitable reference point, such as a corner of a door or window frame, that you will be able to locate fairly accurately on the other image. Zoom in to get more accuracy and then right-click once with the mouse to mark a green spot at that location.

As you do so, a corresponding marker will be added to the other image in an approximately similar position. It’s not always easy to see, so you may need to zoom in or out to find it. 

When you’ve found the marker that was added to the other image, left-click and drag on the marker itself to move it to the exact position that corresponds to the reference point you chose in the first image. Get it?

Repeat this process until you have a reasonable number of marker pairs that are arranged over significant parts of the image. For example ...

If you left-click on a marker in one image, it turns to solid green and the corresponding marker in the other image also turns solid so you can see which ones match. 

If you have clicked a marker, you can use the Remove option from the ribbon to delete that marker pair from both images, or you can click Clear to remove all markers if you want to start again, and if the green colour is not visible enough, you can select a different colour using the Colour option.

Note that it's important that the marker points should identify medium-to-large, flat, two-dimensional areas, such as walls, doors, windows, signposts, etc. and should not all be in a single line. If they are, or if you don't have enough points, the next step might look a bit weird. No, it will look a bit weird ....

So, when you have marked at least 4 points (but ideally 7 to 10 - the more the merrier), click the Merge button on the ribbon to move to the next step...

The composite image ...

If everything went ok, you should see a composite image with a nice pink glow. This will be the primary image but with the secondary image – correctly positioned and sized – showing as a cross-faded ghost. 

If this is not happening, in other words, it does look weird, then as I said above, you probably didn’t choose enough good marker pairs in the previous step. In which case, click Undo Merge to return to the previous step and either adjust your marker positions or add some additional ones to get a better registration.

Using the images above, the following screenshot shows the sort of thing to expect. You can see that the primary image is unchanged (apart from looking pink!) but that the secondary image has been distorted – technically, it’s undergone a perspective warp – and scaled (stretched or shrunk) so that the reference points you marked are as near as possible coincident with those in the primary image. Use the Adjust crossfade slider to fade between the two images to check the registration ...

Now what you do, is to right-click and drag with the mouse to “paint” the areas of the composite image where you want the secondary image to show through. 

These areas will be shown in green as in the example below ...

To paint finer detail, just zoom in before painting – the size of the paintbrush doesn’t change, so the thickness of the brush depends on your zoom level. This is much easier (in my humble opinion) than having to select a particular brush size.

If you mistakenly paint some green areas where you didn’t want to, click the Erase button on the ribbon and then right-click and paint again to return to the pink colour. Click Paint to get back to normal.

After you’ve finished painting the areas you want merged, click Show Result ….

Nearly done ...

At this stage, the primary image will be merged with the warped secondary image in those places you have painted. It should look ok, but maybe not perfect.

By default, the areas will be blurred so that there is a smoother transition, which makes it easier on the eye if the images are not similar colours (as in the above example). This means you might want to paint bigger areas and let the blur fade nicely...

... alternatively, use the slider on the ribbon to adjust the blur radius until you get the visual effect you want. If you mess up, just click Undo Result to return to the previous step and adjust your painted areas.

When you are finally happy, click Save Result to save the final image. Note that you must click the large Save Result button on the ribbon – this is different to the small save at the top left, which saves details of the image filenames, marker positions and painted areas, so you can quickly load a marked and painted pair for editing later.

Ok, enough typing – get playing and see what happens. You have already seen one of my ideas – current and historic photos - but you can also experiment with identical scenes but with different people (I'll post some of my triplets later!), or maybe the same scene at different times of the day, or different times of the year, etc.

See what you think ..... it kept me amused ....