Advanced Tutorial: Converting Scanned Maps

Preface

This tutorial has been kindly contributed by TMJ user wicher based on his experiences with scanning paper maps for use within TMJ. It details some advanced techniques for aligning multiple scans and then converting the result into the projection and format required by TMJ.

Any further questions related to this tutorial can be posted in this forum thread, although please note that specific questions on using some of the third-party tools may be difficult to answer!

Introduction

This tutorial will guide you through a process of converting your map to a TMJ-friendly format. If you have a paper map or a digitized raster map and you want to use it offline, as a raster map in TMJ (MAP page), this tutorial is for you. (Note that the tutorial is not intended to vectorize your map, so you will not be able to use it in PLAN or NAV page for navigation)
All the software used in this tutorial is free.

I wrote this tutorial based on my own experiences. Using the steps described below, I successfully converted a paper map covering an area of more-less 120x100 km in size (scale 1:100000) to a TMJ raster map (.tmj file), with an actual error lower than 2 meters. The size of the resulting scanned image was 28000x22000 pixels.
The text in italics provides supplementary information only, like alternatives or motivation. You do not have to read/understand it to complete this tutorial..

- If you have only a paper map, proceed to point 1.
- If you have a digitized version of your map (multiple scans), proceed to point 2.
- If you have a digitized version of your map (single image file), proceed to point 3.

1. SCANNING

If you want to use your paper map in TMJ, you need to digitize it. To do that, you need a scanner. Scan your map piece-by-piece and make sure that the scanned regions overlap. The greater the overlap the better, but around 4cm should do the trick.
You could also use a camera, but the quality of the image and precision of the calibration will probably be far from perfect. If a camera is still your choice - point 2 may not work for you.

2. STITCHING THE SCANNED MAP

In this step, the scans of your map will be stitched into a single file. Linux users - switch to Windows (sorry!).

a) Download Image Composite Editor (ICE) by Microsoft (for free!)
I have tried two or three different stitching programs, including multi-platform Hugin, as explained here. I think that ICE is the best of them all - it is extremely simple to use (compare with Hugin), yet very accurate. Microsoft did well this time... If you do not use Windows or if you do not like ICE for some reason, feel free to use your favorite stitching software

b) Select your scans: File -> New. Wait...

c) An overview stitched map should pop up.
If something is wrong (missing files etc), make sure that the scanned regions overlap and that the quality of the scans is good enough.

d) Select Stitch -> Camera motion -> Planar Motion 1. Wait...
Why Planar Motion 1? (See this page)

e) Trim, if you want, select quality, click "Export to Disc", select destination folder. Wait...
If a weird, unnamed error pops up in a middle of the exporting process, it is most likely because you run out of space on your disk C:, or wherever you installed ICE. ICE does not require much of RAM, but it needs lots of disk space.

f) Your map is now stitched and ready for further processing.

3. WARPING - CONVERTING MAP'S PROJECTION

Once your map is stitched to a single file, you will need to convert its projection to rectangular projection used internally by TMJ. We will use the free GDAL Tools (http://www.gdal.org/) for that purpose.
No doubt, GDAL is state-of-the-art piece of software. Besides it is probably the only free software of this type - many commercial tools are based on it. For further info about the TMJ rectangular projection see "Some Further Notes on Map Projections..." on this page

a) Download & install GDAL (download page) and PROJ.4 (http://trac.osgeo.org/proj/). Ubuntu users: both these packages are available in the universe repository. Windows users: GDAL - download the "Minimalist windows executables" and follow the instructions in README.TXT; PROJ.4 - download the proj446_win32_bin.zip and follow the instructions in README.TXT.

b) Below is the contents of simple batch (bash) files that you will need to create (Win and Linux versions). Create a text file, name it convert.bat (convert.sh on Linux), copy the content to that file and save it in the directory where your stitched image is saved. Do not close the file yet, you will need to edit it.

c) There are a couple of lines that need to be modified in the batch file. Remember: DO NOT use spaces around equals sign. If you need to include a space in a content of a variable, use backslash in front of it (see examples below):

- INPUT_FILE: name of your original raster map file (stitched scans)

- OUTPUT_FILE: desired name of your output file (after projection conversion)

- INPUT_PROJECTION: projection of your original raster map file. You can find the projection details somewhere on your original map - it should be printed in the legend or in the edge of the map. Example (map of Denmark): "Datum: ETRS89, Projection: UTM, Zone: 32". You need to define the projection in the exact format used by GDAL tools. You can do it in two ways:

  + PROJ.4 format. Example: INPUT_PROJECTION='+proj=utm\ +zone=32\ +datum=WGS84'.
  The ETRS89 datum is not supported by GDAL so we use here the almost-the-same WGS84 datum. Good enough for us!
 
  + EPSG codes format. Go to online EPSG registry http://www.epsg-registry.org/ and find the EGPS code of your projection. For the above example, type "utm zone 32" in the "Name" field and hit Enter. You will get quite a few results, some are very similar to each other. In our case, the batch file must be edited as follows: INPUT_PROJECTION=EPSG:25832. Note the single colon. GDAL does not support some of the EPSG codes - if it returns an error, just pick another EPSG code compatible with (or similar to) your map projection.
 
- GCP_X: Ground Control Points mapping specific pixels on your map to their actual geographic coordinates. Syntax: GCP_x=PixelColumn\ PixelRow\ CoordEasting\ CoordNorthing. To get the GCP mapping, open your raster map file in any image editor (GIMP, IrfanView, Paint, whatever...), find a characteristic point on it (intersection, summit etc.) and note its pixel coordinates (column and row). Next find that same point in GoogleMaps (maps.google.com), right-click on it and click "What's here?". Copy the geographic coordinates from the search field. Fill the GCP_x entry accordingly. Repeat 7 times. Ideally, the GCPs should be distributed more less uniformly on the map, some of them close to the edges though. Note: you should determine the GCPs coordinates (both pixel and geographic) very precisely, so zoom into your image/GoogleMaps and select carefully possibly "narrow" sGCP points, for example a middle of an intersection, rather than "somewhere" on the intersection.
The GCPs will be used by GDAL to orient your raster file on a map (to turn it into Geo-referenced Tiff). The more GCPs entries the better, since the imprecision of pixel-coordinate mapping will be averaged. 7 works pretty well, but if 7 is not your number, change it (do not forget to edit the gdal_translate command though)

d) Save your file and run it (Windows users: just double-click it). Be patient. The resulting file is a converted Geo-referenced Tiff.

4. CONVERTING TO TMJ FORMAT

In this step you will convert your Geo-referenced map to a .tmj file with the use of the TMJ MapOrganizer.

a) Download & install TMJ Map Organizer.

b) Open your GeoTiff in any image editor. The image may look odd: most likely it will have rhombus-like shape with black background. Most probably it will also be unnaturally stretched. At this stage you can freely stretch/squeeze the map horizontally or vertically, to make it look more naturally. Do not rotate, bend or curve the image! Save the resulting image as a 24-bit .jpg (better for full-colour scans) or 8-bit .png file (better for clean limited-colour scans), do not close it yet.
Stephen (the author of TMJ) suggests stretching the image vertically by 1/cos(lat), where lat is the latitude of the middle of the map (see conversation here). It was not enough in my case, the image was still stretched horizontally too much, so I adjusted the image just to make it look good, and it worked like a charm. It seems that as long as you change the image vertical/horizontal dimensions only, you are free to do anything.

c) Pick two of GCPs you defined in point 2c), located possibly far away from each other. Note the pixel positions (column, row) of these points. Again: be as precise as possible (see point 3c, bullet "GCP_x"). Close your image file.

d) Open MapOrganizer. Go to: File -> Open Image File -> find your .jpg map file. Wait...
If MapOrganizer refuses to open your jpg file, it most probably means that it run out of RAM. To remedy the situation, invoke the program with the -Xmx argument or the "2 Start Map Organiser (Grabs 1GB of memory for handling large maps).cmd" shortcut. Note that 32-bit Windows systems (at least XP and older) do not let Java apps grab more than around 1.5G of RAM. If 1.5G is not enough - switch to Linux (for example Ubuntu Live CD)

e) Go to: Tools -> Calculate Bounds -> Fill the pixel position of your GCPs you obtained in the above point and their geographic coordinates (they sit in your convert.bat(sh) file) -> Hit "Calculate Top, Bottom, Left and Right Bounds". The TMJ should fill the corner coordinates accordingly.

e) Save the TMJ file: File -> Save TMJ File As... -> Browse to your location.

f) Copy your .tmj file to your mobile and load it in TMJ-Mobile (MAP page)

g) You can verify the correctness of the conversion by entering your GCPs as Waypoints and verifying their location visually in TMJ-Mobile.
To make your life a bit easier, you can create a .wpt file that you can later import into TMJ (Menu->File->Import Waypoints) This will save you from entering the coordinates manually. Name the file WhateverYouWish.wpt and fill it as follows:

Bash script (Linux version):

Batch file (Windows version):

Sources

http://www.gisnotes.com/wordpress/tag/gdal/
http://www.gdal.org/gdalinfo.html
http://trackmyjourney.co.uk/forum/10586

 
<< back to Tutorial Menu