Rating images painlessly with exiftool feat. ranger & sxiv

Background

I was looking for a way to classifying images by rating them on the fly. My first attempt was using darktable as suggested in a thread. Indeed, the auto-advance rating mechanism was quite handy. But it is still too heavy for this sole purpose. In darktable, user have to import images before editing metadata. When tens of thousands images are involved, the process of importing images can be quite time-consuming¹ Dozen hours for 950.000 images. as it creates for each image an XMP file to store metadata. And this also applies to rating, even though I configure it to improve performance (without OpenCL) the latency is counted by seconds. Moreover, XMP are also created for symlink of image. This was not plausible in my use-case² Statistical classification of images. For each class, it creates a directory in which each symbolic link is associated to the actual image. as it enforces me to keep multiple metadata files for the same image.

Lots of critics, but clearly darktable was not the right tool. It suits better on raw photo post-production as intended. I will present in this post two solutions to remediate the issues aforementioned.

Goals

After experiencing darktable, I know better what I am seeking:

1. Edit metadata in the image file itself. This has two advantages:
   1. Keep metadata even if the filename is changed.
   2. Get rid of XMP files.
2. Preview and select image without latency. Namely, preview images and rate them on the fly.
3. Metadata editing should follow symlink. To centralize metadata in the same place.
4. Batch rating. Rating a whole directory or multiple selected images at once.

Exiftool

ExifTool is a free and open-source software program for reading, writing, and manipulating image, audio, video, and PDF metadata.

Rating images with exiftool

Rating image with exiftool is very simple.

1
exiftool -rating=5 -overwrite_original_in_place <files>

The option -overwrite_original_in_place overwrite directly the file(s) instead of moving the original one to filename.ext_original. Use it wisely at your own risk.

To read back the rating:

1
exiftool -rating <files>

… or format yourself the output:

1
exiftool -p '$Rating $Filepath' -f <files>

And, of course the symbolic links are followed³ Beware, without -overwrite_original_in_place, symlink will be removed! !

But with exiftool alone, one cannot watch and rate image at the same time. This can be done by combine up exiftool with a file manager having preview ability or an image viewer. Next, I will show how to integrate exiftool capability in the file manager ranger and the image viewer sxiv.

File types supported by exiftool

In fact, the version 11.88 of exiftool already supports a large set of file types. Thus, what has been and will be said is not limited to images and rating.

ranger

ranger is a free and open-source CLI files manager I’m using for years. It is very handy to select images and preview them⁵ As long as you use the right terminal emulator, e.g. iTerm, kitty, urxvt, xterm etc. From my past experience, Elementary OS’s pantheon-terminal doesn’t work. .

Append the following snippet in ~/.config/ranger/commands.py. It will add the custom command rate_image <0-5> <files>.

1
# ~/.config/ranger/commands.py
2
class rate_image(Command):
3
    """:rate_image <0-5> <files>
4

5
    Command for rating image with exiftools.
6
    """
7

8
    def execute(self):
9
	import subprocess
10
	if self.arg(1):
11
	    rating_score = self.arg(1)
12
	else:
13
	    self.fm.notify("rate_image: a rating score is required!", bad=True)
14
	    return
15
	if self.arg(2):
16
	    files = self.arg(2)
17
	else:
18
	    cwd = self.fm.thisdir
19
	    cf = self.fm.thisfile
20
	    if not cwd or not cf:
21
		self.fm.notify("Error: no file selected for deletion!", bad=True)
22
		return
23
	    if len(cwd.marked_items) > 1:
24
		files = " ".join([f.shell_escaped_basename for f in cwd.marked_items])
25
		self.fm.mark_files(all=True, val=False)
26
	    else:
27
		files = cf.shell_escaped_basename
28
	command = "exiftool -rating=" + rating_score + \
29
		  " -overwrite_original_in_place " + files
30
	self.fm.notify("Run command: " + command)
31
	result = self.fm.execute_command(command, stdout=subprocess.PIPE)
32
	stdout, stderr = result.communicate()
33
	if result.returncode == 0:
34
	    # This is a generic function to print text in ranger.
35
	    self.fm.notify("Succeeded to rate image " + files + \
36
			   " with score " + rating_score + ".")

It remains to define some key bindings to be granted the full power of ranger. Append the following snippet to ~/.config/ranger/rc.conf.

1
# ~/.config/ranger/rc.conf
2
map r1 rate_image 1
3
map r2 rate_image 2
4
map r3 rate_image 3
5
map r4 rate_image 4
6
map r5 rate_image 5

The resulting workflow is as follows:

• Select images with SPC (single selection) or v (reverse selection).
• Press r and the rating 1 to 5.

sxiv

sxiv is a free, open-source, lightweight and scriptable image viewer. Add the following entries in ~/.config/sxiv/exec/key-handler⁶ If this file doesn’t exist, you can copy the sample one: mkdir -p ~/.config/sxiv/exec/ && cp /usr/share/doc/sxiv/examples/key-handler .

1
# ~/.config/sxiv/exec/key-handler
2
case "$1" in
3
# ...
4
"C-1")      tr '\n' '\0' | xargs -0 -I {} exiftool -rating=1 -overwrite_original_in_place "{}" ;;
5
"C-2")      tr '\n' '\0' | xargs -0 -I {} exiftool -rating=2 -overwrite_original_in_place "{}" ;;
6
"C-3")      tr '\n' '\0' | xargs -0 -I {} exiftool -rating=3 -overwrite_original_in_place "{}" ;;
7
"C-4")      tr '\n' '\0' | xargs -0 -I {} exiftool -rating=4 -overwrite_original_in_place "{}" ;;
8
"C-5")      tr '\n' '\0' | xargs -0 -I {} exiftool -rating=5 -overwrite_original_in_place "{}" ;;
9
esac

The resulting workflow is as follows:

• Open images with sxiv.
• Press C-x C-<1..5> to rate the current image.
• Or mark images with m⁷ See sxiv/config.def.h for more keybindings to mark images. , toggle thumbnails mode with RET and press C-x C-<1..5> to rate selected images.

Bonus

Some interesting tips are presented here. The main dependencies are zsh and GNU parallel, adapt it to fit your need.

Rate or view images of specific rating

The long command below search all JPG files (potentially symlink) of rating 3, 4 or 5 and view them with sxiv.

1
# 1. Find recursively all JPG under the current directory
2
# 2. Use GNU parallel, for each JPG
3
#    (a) noglob (zsh specific): disable zsh globing to prevent "No such file" error
4
#    (b) Build the string "$Rating $Filepath"
5
#    (c) Remove the last line of exiftool output which count the amounts of the files.
6
#    (d) Keep entry with $Rating in {3, 4, 5}. Remove the first space.
7
# 3. Use GNU parallel: view at most 250 found images with sxiv at a time.
8
find -L . -type f -regex ".*\.jpg" |
9
parallel -L 5000 \
10
  'noglob exiftool -p '\''$Rating $Filepath'\'' -f -q -q {} | \
11
   head --lines=-1 | \
12
   awk '\''$1 ~ /[345]/ {$1=""; print substr($0, 2)}'\'' ' |
13
parallel -L 250 -q sxiv "{}"

Some remarks:

• To see unrated images, use - as rating, then you can rate them with sxiv as above.
• This one-shot command is fast enough for hundreds of images. Above this amount, you may want to take some time to dump the result in a file as follows.

1
find -L -type f -regex ".*\.jpg$" |
2
parallel -L 5000 \
3
 'noglob exiftool -p '\''$Rating $Filename'\'' -f -q -q {} | \
4
  head --lines=-1 | \
5
  awk '\''$1 ~ /[^0-]/ {print $0}'\'' ' >> ratingdb.txt

And view images with specific rating with this command:

1
awk '$1 ~ /345/ {$1=""; print substr($0, 2)}' ratingdb.txt |
2
parallel -L 250 -q sxiv "{}"

• The best would be writing a python script to maintain an SQLite database, and adapt the script of ranger and sxiv above to update the database each time a file rating changed.
• shuf may be used after awk to shuffle images before viewing them.
• Actually, no one will type again and again those lengthy commands. I either use C-R in zsh with fzf for casual ones or add them as entries in pet, a manager of parametrizable snippet.

Migrate XMP rating into image metadata

If you migrate from darktable or have XMP files with rating, you can try the following commands.

1
zmodload zsh/mapfile
2
# 1. Find recursively all XMP under the current directory
3
# 2. Use GNU parallel, for each XMP
4
#   (a) Extract the rating with grep and awk; put it in $RATE.
5
#   (b) Append the image path to .xmp-$RATE-rating.db
6
find . -type f -regex ".*\.xmp" -print0 |
7
parallel -0  'TMP=$(grep -o "Rating=\"[012345]\"" {} | \
8
  awk '\''{print gensub(/Rating="([0-5])"/, "\1", "g", $1)}'\''); \
9
  echo {} >> xmp-$TMP-rating.db'
10
# 3. For each $RATE:
11
for i in {1..5}; do
12
    FNAME="xmp-$i-rating.db"
13
    # For each file in .xmp-$RATE-rating.db
14
    for f in "${(f)mapfile[$FNAME]}"; do
15
	# Use exiftool to rate it with the corresponding $RATE
16
	exiftool -rating=$i -overwrite_original_in_place "${f%.*}"
17
    done
18
done