Copy file from Server A to Server B
I have requirement to copy files from Server A to Server B
And then move the copied files Server A to Archive folder
I have written the script, but it takes the directory name differently.
Copying file from Server A to Server B is working, but moving file to Archive folder is failing
Script
#!/usr/bin/env bash
username="tempusr"
dir_origin="/data"
dir_destination="/export/home/tempusr/data"
Ip="123.456.789"
echo $dir_origin
echo "Uploading files to remote server...."
dt=$(date +%Y%m%d%H%M%S)
for file in /data/*.xml
do
name=${file%.*}
echo "Name is: "${name}
echo "File is: "/data/$file
echo "Coping files from Server A to Server B"
scp -rC $file $username@$Ip:$dir_destination
echo "New File is: " /data/processed/${name}_${dt}.xml
echo "Moving files from Server A /data to /data/processed and appending timestamp before file type"
mv $file /data/processed/${name}_${dt}.xml
done
echo "File upload to remote server completed! ;)"
Output I am getting is:
ISSUE: If you check the line File is:
its taking path as "/data//data/" but it need path as just /data/.
I am not getting, where I am wrong.
./transfer.sh
/data
Uploading files to remote server....
Name is: /data/1552653_123510007_1316325
File is: /data//data/1552653_123510007_1316325.xml
SERVERB password:
1552653_123510007_1316325.xml 100% 2204 1.6MB/s 00:00
New File is: /data/processed//data/1552653_123510007_1316325_20230914002942.xml
mv: cannot move '/data/1552653_123510007_1316325.xml' to '/data/processed//data/1552653_123510007_1316325_20230914002942.xml': No such file or directory.
Also, it asks for password every time and on this system sshpass and run apt install sshpass.
So is there any other way, where I can pass password directly in scp command or such that I don’t need to enter password every time.
Replace line:
mv $file /data/processed/${name}_${dt}.xml
with
mv $file /data/processed/${name##*/}_${dt}.xml
The ##*/ will strip of the path (everything until (including) the last slash) of the variable name.
For your other problem, as mentioned in the comments, set up ssh public key authentication without passphrase for your private key.
The issue can clearly be seen in your very first line of output,
Name is: /data/1552653_123510007_1316325
Everywhere you use $name you will be including the /data/ prefix.
So to fix this, either strip the prefix from $name or stop reincluding /data in your paths. Here’s one approach:
#!/bin/sh
username="tempusr"
origin="/data"
backup="/data/processed"
destination="/export/home/tempusr/data"
ip="203.0.113.1"
dt=$(date +%Y%m%d%H%M%S)
echo "Uploading files to remote server..."
for file in "$origin"/*.xml
do
[ -f "$file" ] || continue # In case there are no files to copy
name=${file%.*} # Strip extension
name=${name##*/} # Strip leading path
save="${name}_$dt.xml" # Backup filename with datetime suffix
echo "Name is: $name"
echo "File is: $file"
echo "Copying file from Server A to Server B"
scp -C -- "$file" "$username@$ip:$destination"
echo "New file is: $backup/$save"
echo "Moving file from Server A $origin to $backup and appending timestamp before file type"
mv -f -- "$file" "$backup/$save"
done
echo 'File upload to remote server completed! ;)'
You can use ssh public key certificates to avoid entering a password each time. This is well documented but the gist of it is as follows:
ssh-keygen -t ed25519 # Take defaults, do not overwrite an existing key
ssh-copy-id username@ip # Replace username@ip to match your script values
If ed25519 is not recognised you can try rsa instead, but make sure ssh-copy-id copies the correct public key.