Can scp create a directory if it doesn't exist?

As far as I know, scp itself cannot do that, no. However, you could just ssh to the target machine, create the directory and then copy. Something like:

ssh user@host "mkdir -p /target/path/" &&
    scp /path/to/source user@host:/target/path/

Note that if you are copying entire directories, the above is not needed. For example, to copy the directory ~/foo to the remote host, you could use the -r (recursive) flag:

scp -r ~/foo/ user@host:~/

That will create the target directory ~/foo on the remote host. However, it can’t create the parent directory. scp ~/foo user@host:~/bar/foo will fail unless the target directory bar exists. In any case, the -r flag won’t help create a target directory if you are copying individual files.

If you are copying a group of files, not really. If you are copying a directory and all the contents below it, yes. Given this command:

$ scp -pr /source/directory user@host:the/target/directory

If directory does not exist in ~/the/target on host, it will be created. If, however, ~the/target does not exist, you will likely have issues – I don’t remember the exact results of this case, but be prepared for the intended scp to fail.

Yes, you can. According to scp’s man page:

man scp

…..

-r Recursively copy entire directories. Note that scp follows symâ
bolic links encountered in the tree traversal.

….

I use the following function before transferring btrfs snapshots over ssh:

check_remote_dir() {
        printf "ntesting remote directory: '$1' "

        if ssh -p $PORT $ROOT@$REMOTE "[ ! -d $1 ]"; then
                printf "nCreating: $1 on $ROOT@$REMOTEn"
                ssh -p $PORT $ROOT@$REMOTE "mkdir -p $1"
        else
                printf "[OK]n"
        fi
}

Just call the function in your script with:

check_remote_dir /my/remote/path

You could chain the command with the mkdir command and then reference to the folder you’ve created:

mkdir ~/new-folder/ && scp -P 22 <remote/url>:~/new-folder/

When restricted to only scp or sftp with no ssh available (by rssh), it is possible to create directories by copying empty directory tree with rsync and then copying files.

That is how I put public ssh key to remote host when ssh-copy-id does not work and .ssh directory does not exists:

rsync -e 'ssh -p22' -av -f"+ */" -f"- *" ~/.ssh backup@1.2.3.4:~/
scp -P22 ~/.ssh/id_rsa.pub backup@1.2.3.4:~/.ssh/authorized_keys

It is possible to combine this commands into one rsync adding one more include filter in the middle, if target file name is the same.

How about "smart" scp + ssh mkdir?

This is a script to create target directory hierarchy only once per directory (not a blind ssh mkdir -p each time).

Rationale

I’m copying motion recordings (video files) as they are written to local disk (event-driven).

• rsync doesn’t make sense because there is no point in doing full directory comparisons – I am transferring one file per event. If I restrict rsync to just one subdir or file, then it won’t be able to create the full target directory hierarchy.
• scp will work great but it doesn’t know if the target directory exists, and I don’t want to run an ssh mkdir -p for every file.

Solution

1. Create each target dir via ssh the first time there is a need (first time we are about to copy into it).
2. Cache target dir name in a an array so that we won’t ssh mkdir each time we see it again. Clear this array daily so that it doesn’t get huge.
3. scp the file.

Example code:

#!/bin/bash

TARGETHOST="some.hostname"
TARGETPATH="/data/backups/MOTION/"

ensure_dir() {
    if [ "$TODAY" != "$(date +%D)" ] ; then
        CREATED=()  # reset array daily and first time so it won't get huge
        TODAY=$(date +%D)
    fi  
    if [[ ! " ${CREATED[@]} " =~ " ${1} " ]]; then
        # create dir
        echo "Creating remote directory: $1"
        ssh $TARGETHOST "mkdir -p ${TARGETPATH}$1"
        CREATED+=("$1")
    fi  
}

# This line is specific to my use case, watching a directory for new files:
inotifywait -mr --format '%w %f' -e close_write --exclude '.jpg' /data/MOTION/ | while read filedir filename ; do
    RELDIR=$(echo $filedir | sed 's|/data/MOTION/||')
    ensure_dir "$RELDIR"
    echo "Copying '$filename' from '${filedir}' to '${RELDIR}'"
    scp -r -p "${filedir}${filename}" "${TARGETHOST}:${TARGETPATH}${RELDIR}${filename}"
done

Typical output, illustrating "smart" mkdir behavior:

Creating remote directory: 2019-12-28/movie-C6/
Copying 'test1.mp4' from '/data/MOTION/2019-12-28/movie-C6/' to '2019-12-28/movie-C6/'
Creating remote directory: 2019-12-28/movie-C4/
Copying '095700.mp4' from '/data/MOTION/2019-12-28/movie-C4/' to '2019-12-28/movie-C4/'
Copying '095841.mp4' from '/data/MOTION/2019-12-28/movie-C4/' to '2019-12-28/movie-C4/'
Copying '100410.mp4' from '/data/MOTION/2019-12-28/movie-C4/' to '2019-12-28/movie-C4/'
Creating remote directory: 2019-12-28/movie-C2/
Copying '102106.mp4' from '/data/MOTION/2019-12-28/movie-C2/' to '2019-12-28/movie-C2/'
Creating remote directory: 2019-12-28/movie-C5/
Copying '102318.mp4' from '/data/MOTION/2019-12-28/movie-C5/' to '2019-12-28/movie-C5/'
Copying '102326.mp4' from '/data/MOTION/2019-12-28/movie-C2/' to '2019-12-28/movie-C2/'
Creating remote directory: 2019-12-28/movie-C1/
Copying '102450.mp4' from '/data/MOTION/2019-12-28/movie-C1/' to '2019-12-28/movie-C1/'
Copying '102744.mp4' from '/data/MOTION/2019-12-28/movie-C2/' to '2019-12-28/movie-C2/'
Copying '103008.mp4' from '/data/MOTION/2019-12-28/movie-C2/' to '2019-12-28/movie-C2/'
Copying '103945.mp4' from '/data/MOTION/2019-12-28/movie-C6/' to '2019-12-28/movie-C6/'
Copying '104252.mp4' from '/data/MOTION/2019-12-28/movie-C6/' to '2019-12-28/movie-C6/'
Copying '104824.mp4' from '/data/MOTION/2019-12-28/movie-C6/' to '2019-12-28/movie-C6/'