Rename command with non-existing target directory

I’m trying to group files automatically into subdirectories using a command like this:

$ rename 's/(.)(.)(.+)/$1/$2/$1$2$3/' *.*

A dry run with the -n parameter shows me what I want:

test.jpg renamed as t/e/test.jpg

But the actual renaming fails with

Can't rename test.jpg t/e/test.jpg: No such file or directory

because the subdirectories do not exist yet.

How can I achieve this without creating all subdirectories manually beforehand?

There is difference between renaming and moving to somewhere. 
In this case the easiest way (in modern bash) is to loop through all files:

for f in *.*
    [ -d "$d" ] || mkdir -p "$d"
    mv "$f" "$d"


This makes use of the lesser known "Parameter Expansion / Substring Expansion" feature of bash.


    Expands to up to length characters of the value of var
    starting at the character specified by offset.

.  [Slightly paraphrased from bash(1).]

I don’t see where this is documented, but, offset is zero-based,
so 0 is the first character and 1 is the second. 
If offset is null (missing), it is taken to be zero.

Answered By: Costas

As an extension of Costas’s answer,

for f in *.*
    if [ "${f:0:1}" = . ]  ||  [ "${f:1:1}" = . ]
        printf 'Cannot handle "%s"n' "$f"
        if ! { [ -d "$d" ] || mkdir -p "$d"  &&  mv "$f" "$d"; }
  • I added the (first) if
    to handle filenames like M.Butterfly.jpg and Q.jpg,
    because you can’t (really) make directories like M/. and Q/.
    Likewise, .Dragon.mp4
    would try to create a directory called ./D.1

  • I added the && (between the mkdir and the mv) so,
    if the mkdir fails,
    we don’t try to move the file into the non-existent directory.

  • I added the second if because,
    if something fails, it makes sense for the script to terminate
    and let the user figure out (and fix) what’s wrong.
    (For example, what if you accidentally run the script in a directory
    where you don’t have write permission, or the file system is full
    and so you cannot create the new subdirectories?)

    I could have written this as

    [ -d "$d" ] || mkdir -p "$d"  &&  mv "$f" "$d  ||  break

    but these &&/|| sequences get confusing very quickly.

Another variation:

for f in *.*
    if [ "${f:0:1}" = . ]  ||  [ "${f:1:1}" = . ]
        printf 'Cannot handle "%s"n' "$f"
        if ! { [ -d "$d" ] || mkdir -p "$d"; }          # Note: no "mv" command in the loop.
rename 's/(.)(.)(.+)/$1/$2/$1$2$3/' *.*

This has the advantage that it invokes rename only once
instead of invoking mv a thousand times. 
It has a couple of drawbacks:

  • files like M.Butterfly.jpg and Q.jpg get passed to rename.
  • rename gets run even if mkdir fails. 
    (This is probably easy to fix. 
    One way would be to replace the break with exit,
    but that seems inelegant.)

I don’t have the same version of rename as you,
but I tried mv M.Butterfly.jpg M/.,
and it simply moved M.Butterfly.jpg into the directory M
So, if you use my second solution, when you list the directory M,
you will see subdirectories a, e, i, o, etc., (containing files
like Many.jpg, Memorable.jpg, Minnesota.jpg, Moments.jpg, etc.)
and a few plain files (like M.Butterfly.jpg). 
You can then deal with them as you please.
1 Actually, the commands will succeed,
but they will create directories called M, Q and D,
which might not be what you want.

This can be done entirely witin the rename script, because rename allows the use of any perl code, it’s not limited to just simple s/// operators.

The following works in the simple case that all files to be renamed are in the current directory and are passed to rename directly on the command-line without a ./ prefix (which is generally not a good idea because of the potential for filenames beginning with - which could be interpreted as command-line options for rename).

It won’t work if the filenames include a directory prefix, e.g. they are passed to rename via find or as ./*.*.

$ touch test.jpg test.txt
$ rename -n 'my ($a, $b) = (/(.)(.)/);
             mkdir $a unless -e $a;
             mkdir "$a/$b" unless -e "$a/$b";
             if (-e "$a/$b" && ! -d "$a/$b") {
               print STDERR "$_: $a/$b exists but is not a directory!n";
             } else {;
             }' *.*
rename(test.jpg, t/e/test.jpg)
rename(test.txt, t/e/test.txt)

A better version could use the File::Basename and File::Path modules, both of which are included with perl.

$ rename -n 'use File::Basename;
             use File::Path qw(make_path);

             my ($bn, $dn) = fileparse($_);
             my ($a, $b) = ($bn =~ /(.)(.)/);
             make_path("$dn$a/$b", { verbose => 1 });
             $_ = "$dn$a/$b/$bn"' ./*.*
mkdir ./t
mkdir ./t/e
rename(./test.jpg, ./t/e/test.jpg)
rename(./test.txt, ./t/e/test.txt)

This will create the two sub-directories under each file’s original directory. The make_path() function from File::Path already checks for existence of the target dir (and treats it as a fatal error if it exists but is not a directory, but you can implement your own error handling if you want it to just print a warning and move on to the next filename).

You could hard-code it to rename all files into a specific directory tree if you wanted to by setting $dn to the target top-level directory (you’ll still need to use the File::Basename module to extract just the basename from the full pathname).

Answered By: cas
Categories: Answers Tags: , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.