Your exchanging pdf files with an editor for a journal publication. Each time you send a list of modifications to be applied and the modified pdf file comes back to you, usually with some additional modifications that you didn’t ask for. How to spot these change easily and to make sure they are relevant?

Problem

The solution that come to mind is to use a pdf diff tool, similar to what you usually use for text files or source code: diff, diff3, kdiff3, meld are such examples. However, a quick search on the internet couldn’t reveal any tool fulfilling the conditions (being free and open source are mandatory conditions).

Solution

The easy trick is to first convert these pdf to text file and then use the usual text comparison programs:

pdftotext file1.pdf
pdftotext file2.pdf
kdiff3 file1.txt file2.txt

and that’s it, you are now able to see what has been modified between the two files. Don’t count on that to be able to make a merge though!

A family gathering with ten different photographers shotting every moment. You end up with thousands of photos that need to be organized at least in sequence over the time (to make a slide-show, for example).

Problem

Some cameras name the photos as IMG_XXXX, others as DSC-XXX, etc. Without proper organization, you can’t simply view the photo in a time organized sequence.

Trick

Most digital cameras record the date and time in the photo exif header. The idea is to use this information to rename the photo according to the date and time when it was taken. Once the photos are renamed with the proper name format, it becomes easy to make them into a sequence.

Application

First, you’ll need the jhead program installed apt-get install jhead (on Debian/Ubuntu). This simple program does all the work for you. Just launch:

jhead -nf%Y%m%d-%H%M%S-photographerName *.jpg

in the directory of each photographer. Then, you can copy all photos within the same directory.

All cameras might not be set properly. There will at least be few minutes of difference between them. It can be few years if one camera time has not been set before. Choose one reference camera (enough shots in the series with a correct time) and try to estimate the time-difference with others. Then jhead can help you to change the exif header according to your evaluation:

jhead -da2006:08:05+17:41:40-2006:08:05+11:39:06 *-photographerName.jpg

to add about 6 hours and 2 min to all shots taken by photographerName in the series. The second argument (2006:08:05+11:39:06) is the current date-time of one of the shot. The first argument (2006:08:05+17:41:40) is the date-time according to the reference camera. All shots will be incremented by this difference.

Once the time is corrected, you need to rerun the renaming

jhead -nf%Y%m%d-%H%M%S-photographerName *-photographerName.jpg

One solution to create an image which will look closer to the reality is to use High Dynamic Range (HDR). HDR combines different shots of the same scene taken with different exposure to create a 16 bits or 32 bits image. Pixel in these images represents a physical value: the luminance.

Of course as monitor, printers, etc can’t represent such an image, an additional step is required: tone mapping. The tone-mapping is a projection from the real luminance value back to the 8 bits value. Of course a simple linear projection won’t do the trick. Several algorithms exists and can do that for you. These images are often referred as HDR which is not strictly correct.

Qtpfsgui, an open-source HDR workflow can help you create your own HDR images. Here are some examples.

In a church in Amsterdam, three shots at different exposure. The camera was stable so no registration step was required before the HDR generation.

And the result:

The same on a street of Shanghai. Here the camera was not still so a registration was first necessary:

And the result

Here you can note the ghost effect on people would didn’t want to stand still while I was taking the pictures. Within the one second between the first and the third shot, they moved and of course the registration process is not designed to compensate for that.

A good opportunity to try it out was to simplify my previous python tagging script.

Using the lib is easy from the examples on Google’s website:

coordinates = factory.CreateCoordinates()
coordinates.add_point3(lon,lat,alt)

to create coordinates for examples. All elements are created and combined together in a kmldom::Document which is passed to a kmldom::Kml then written to a file.

Here is the full script:
jpegtokml.py.

Just run as python jpegtokml.py

Enjoy !

NB: You’ll need to get the libkml cvs version >= r33 for this to work

We will now see how to make the generation of photo map automatic with python from this set of geotagged photos. We need to generate thumbnails of the images and the kml file. Here is the idea behind the script:

• Loop through all jpeg files in a folder (we assume the selection is already done and that you have only the photos you want to put on the map)
• Read the exif informations from the file
• Generate a resized version of the image adapted to browser visualization (800 pixels) and a thumbnail (160 pixels).
• Write the corresponding instructions in the KML file

Looping through all the jpeg files in the folder is the easy part with python (for fileName in os.listdir( folder ): combine with a matching based on fnmatch.fnmatch (fileName, '*.jpg')).

Reading the information is done using the pyexiv2. For example:

image = pyexiv2.Image(filename)

image.readMetadata()

orientation=image['Exif.Image.Orientation']

We can take advantage of the powerfull PIL library to resize the images:

im=Image.open(folder+'/'+fileName)

im.thumbnail(160,106)

As we had easily retrieved the orientation information from the exif data, we can rotate the photos: im=im.rotate(90). I choose to prefix the 800pix filename with ‘w’ and the 160pix filename with ‘_thb_w’ (to match my earlier SPGM gallery).

Intructions in the KML file look like that:

<placemark>

<name>Photo Name</name>

<description>

<!--[CDATA[<a href='http://www.website.sth/wherethefileis.jpg'

target='_blank'>

<img src='http://www.website.sth/wherethethumbnailis.jpg' />

</a>]]--></description>

<styleurl>#camera</styleurl>

<point>

<coordinates>-159.432961,21.875480,11.0</coordinates>

</point>

</placemark>

To make this output effortlessly (almost), I just used a python file from gpicsync which is very nice but lack the feature of using directly the GPS information from the photos.

Here is the result:

Photos on maps

The full script is just two python files:

• kmlGen.py (modified from gpicsync)
• gmGeneration.py which does the job

Just edit the gmGeneration.py file to set the folder and url variables to whatever you need and run

python2.5 gmGeneration.py

EDIT 2008-04-03: Using libkml, the script is now simplified as shown on the new post “Mapping photo using libkml”.

D’ici 5 ans au plus, tous les appareils photos intégreront un recepteur GPS, mais en attendant, il faut faire sans. Heureusement, il y a une variable commune qui va nous permettre de lier les deux: le temps.

La date et l’heure sont inscrites dans les fichiers photo, plus précisement dans les données EXIF. Le GPS enregistre évidemment ces info (info de base pour lui). Il faut s’assurer que l’heure de l’appareil photo est correcte: moins de la minute d’écart si possible ou sinon il suffit d’en connaitre le décalage; garder aussi à l’esprit le fuseau horaire dans lequel on se trouve (TU+1 ou TU+2 selon la saison en France), le GPS gardant les heures en TU en général.

Il est plus facile de vérifier l’heure correcte des photos si ces données sont directement dans le nom. On va commencer par renommer les images (on verra plus tard qu’il y a un autre intérêt à faire ce renommage):

jhead -nf%Y%m%d-%H%M%S *.jpg

On peut également ajuster l’heure selon le décalage constaté de l’appareil:

jhead -ta+0:05 *.jpg

pour ajouter 5 min par exemple. (Ne pas oublier de refaire l’opération de renommage pour rester consistant.)

Les photos sont prêtes.

D’un autre côté, on a un fichier gpx obtenu d’un tracker GPS. Ce fichier peut contenir également des points aberrants (nombre trop faible de satellites, pas de fix, etc). On va donc commencer par filtrer ces données

gpsbabel -i gpx -f track-all.gpx \
      -x discard,sat=4,fixnone,fixunknown \
      -o gpx -F track.gpx

(A noter que les options fixnone et fixunknown ne sont disponible dans GPSBabel qu’après application d’un simple patch.)

On est maintenant prêt pour utiliser l’outils gpscorrelate. Sous Ubuntu ou Debian, un simple apt-get install gpscorrelate-gui et on est paré.

L’utilisation est triviale: on lance l’interface gpscorrelate-gui, on charge les photos préparées précédemment, puis le fichier gpx, on n’oublie pas de spécifier le fuseau horaire de l’heure des photos, on clique sur le bouton correlate et ca y est. Les photos sont taggées. On peut le vérifier par la commande exiftools sur une des photos.

L’ajout de l’option est plutôt simple et le patch ci-dessous a été soumis pour une inclusion dans la prochaine version de GPSBabel, en attendant, vous pouvez l’appliquer vous même:

--- gpsbabel-1.3.5-beta20080217/discard.c       2008-02-06 03:29:21.000000000 +0100
+++ new/discard.c        2008-03-24 17:16:30.000000000 +0100
@@ -27,6 +27,8 @@
 static char *vdopopt = NULL;
 static char *andopt = NULL;
 static char *satopt = NULL;
+static char *fixnoneopt = NULL;
+static char *fixunknownopt = NULL;
 static double hdopf;
 static double vdopf;
 static int satpf;
@@ -43,6 +45,10 @@
                NULL, ARGTYPE_BOOL, ARG_NOMINMAX},
        {"sat", &satopt, "Minimium sats to keep waypoints",
                "-1.0", ARGTYPE_BEGIN_REQ | ARGTYPE_INT, ARG_NOMINMAX},
+       {"fixnone", &fixnoneopt, "Suppress waypoints without fix",
+               NULL, ARGTYPE_BOOL, ARG_NOMINMAX},
+       {"fixunknown", &fixunknownopt, "Suppress waypoints with unknown fix",
+               NULL, ARGTYPE_BOOL, ARG_NOMINMAX},
        ARG_TERMINATOR
 };

@@ -71,6 +77,12 @@
         if ((satpf >= 0) && (waypointp->sat < satpf))
                del = 1;

+        if ((fixnoneopt) && (waypointp->fix == fix_none))
+               del = 1;
+
+       if ((fixunknownopt) && (waypointp->fix == fix_unknown))
+               del = 1;
+
        if (del) {
                switch(what) {
                        case wptdata:

L’option est alors -x discard,fixnone ou -x discard,fixunknown. On peut bien sur combiner avec -x discard,fixnone,fixunknown.

The first inaccuracy source is that Google seems to round decimal latitude and longitude to 10^-6. That’s about 10 cm on the ground.

The second source of error depends on the processing of each satellite or aerial image. This is a process called orthorectification and its accuracy can vary widely. This error will depend on the area you’re looking at in Google Earth. You probably won’t have the same accuracy in Europe and Africa as it can depend on the digital elevation model (DEM) of the area.

If you know precisely the coordinates of a particular point that you can see on Google images, you can evaluate the accuracy on this area. Let’s see an example with a reference geodesic point.

Reference geodesic point

Zoom to the maximum: with a perfect orthorectification the blue mark should be exactly on the center of the white square. This is not the case, so we know that for this area we have an accuracy of about 1-2 meters.

This accuracy can be considered as quite good already (not enough though to be used as a reference) but probably worsen in mountainous areas.

One track example

To do that, you need to register to get your own Google API key and install the XML Google Maps plugin. Once you’ve done that, upload the kml file, link it and you’re done.

So simple that I’m speechless…

Et bien… des gens se sont déjà posé la question avant vous et ils ont créé Open Street Map. Le résultat est plutôt impressionnant.

Voyons voir comment contribuer…

Etape Intellectuelle: l’optimisation

Si vous aimez les challenges logiques, réfléchissez avant pour trouver le chemin optimal qui permet de parcourir toutes les rues avec le minimum de distance. Hum… en fait pour faire ca il faut une carte (cf étape 4).

Etape Sportive: le terrain

En vélo, c’est l’étape sportive de l’affaire: on trouve une zone avec une pente suffisante, de préférence non cartographiée et on fait toutes les rues dans un rayon de 500m, ca fait pas très loin, mais pas mal de distance parcourue.

Par exemple ca:

Etape Technique: récupération des données

Cf un post précédent pour sortir les données du tracker GPS. Générez un fichier gpx et un kml, ca servira.

Ouverture du fichier kml sous Google Earth, repérage du début et de la fin de la trace à uploader. Il faut repérer les heures (à la seconde près). Et extraction de la partie intéressante du fichier gpx:

gpsbabel -t -i gpx -f GPSDATA-20080225.gpx \
  -x track,start=20080225071441,stop=20080227 \
  -o gpx -F GPSDATA-20080225-new.gpx

Ensuite connexion à http://www.openstreetmap.org, login, upload du dernier fichier gpx.

Etape Artistique: le dessin

Laissez libre court à votre imagination pour dessiner les rues parcourues par dessus vos traces GPS.