Files sharing with ssh

From The World according to Vissie
Jump to navigation Jump to search

If you are a Gnome user, this is possible in Nautilus as well.

Instead of

fish://username@192.168.1.20

you have to enter

ssh://username@192.168.1.20

in the address bar – protocol is ssh:// instead of fish://.

sshfs

or you could try sshfs http://fuse.sourceforge.net/sshfs.html

Before attempting to mount a directory, make sure the file permissions on the target directory allow your user correct access. To mount, invoke sshfs to mount a remote directory:

sshfs USERNAME@HOSTNAME_OR_IP:/REMOTE_PATH LOCAL_MOUNT_POINT SSH_OPTIONS

For example:

sshfs sessy@mycomputer:/remote/path /local/path -C -p 9876 -o allow_other

Where -p 9876 stands for the port number, -C use compression and -o allow_other to allow non-rooted users have read/write access.

nc to copy

# SOURCE: 
tar -cf - /u02/databases/mydb/data_file-1.dbf | pigz | nc -l 8888
# TARGET:
nc server.addr 8888 | pigz -d | tar xf - -C /

http://intermediatesql.com/linux/scrap-the-scp-how-to-copy-data-fast-using-pigz-and-nc/

as per the above link, I want to try this!

Monitoring transfer progress with nc

Pigz/nc transfer might be significantly faster, but, it might not be the easiest to monitor. While scp has a nice progress bar, pigz/nc just gives you a blank screen for the entire duration of the transfer. Fortunately, it is very easy to correct if you drop in a pipe viewer tool within pigz/nc pipe.

> tar -cf - /u02/databases/mydb/data_file-1.dbf | \

 pv -s `du -sb /u02/databases/mydb/data_file-1.dbf | awk '{s += $1} END {print s}'` | \
 pigz  | nc -l 8888

script to automate nc copy

vim: /etc/acpi/lid.sh
#! /usr/bin/bash

FILE_FULL=$1
REMOTE_HOST=$2

FILE_DIR=$(dirname $FILE_FULL)
FILE_NAME=$(basename $FILE_FULL)
LOCALHOST=$(hostname)

ZIP_TOOL=pigz
NC_PORT=8888

tar -cf - -C $FILE_DIR $FILE_NAME | pv -s `du -sb $FILE_FULL | awk '{s += $1} END {printf "%d", s}'` | $ZIP_TOOL | nc -l $NC_PORT &
ssh $REMOTE_HOST "nc $LOCALHOST $NC_PORT | $ZIP_TOOL -d | tar xf - -C $FILE_DIR"

Some random notes to look at and test

In most cases I personally tested, I had good results with database files on lzop. Lzop suited our needs better than PIGZ.

for networks (<10MBit/s) I use xz with varying compression ratios, even xz -0 is a lot better than for example bzip2.

the source command needs to be ” … | nc -l -p8888″ … the “-p” is missing above. At least this is what I needed to make it work here.

nc over ssh?

tar -cf – -C $FILE_DIR $FILE_NAME | pv -s `du -sb $FILE_FULL | awk ‘{s += $1} END {printf “%d”, s}’` | $ZIP_TOOL | ssh $REMOTE_HOST ” $ZIP_TOOL -d | tar xf – -C $FILE_DIR”


nc with encryption !

I improved apon this script by adding gpg encryption via a pipe and a remote directory to place the file. it now takes 4 options and you have to create a gpg key and have it available on both the client and the server Here is how to create the gpg keys. i didn’t use any passwords…

https://fedoraproject.org/wiki/Creating_GPG_Keys

i just used SCP to move the keys.

http://www.phildev.net/pgp/gpg_moving_keys.html

you have to edit the variable ENCRYPT_KEY=’your-keyname’ in the script the kename must be inside ‘ ‘ it takes three options the name of the file you want to transfer the name of the server you want to trasfer to and the directory you want the file placed on that server. Enjoy.

vim: /etc/acpi/lid.sh
#! /bin/bash
FILE_FULL=$1
REMOTE_HOST=$2
REMOTE_FILE_PATH=$3
FILE_DIR=$(dirname $FILE_FULL)
FILE_NAME=$(basename $FILE_FULL)
LOCALHOST=$(hostname)
ZIP_TOOL=pigz
ENCRYPT_KEY=’your-keyname’
CRYPT_TOOL=gpg
ENCRYPT_OPTS=” –encrypt -r $ENCRYPT_KEYDECRYPT_OPTS=” –decrypt -r $ENCRYPT_KEYENCRYPT_TOOL=$CRYPT_TOOL $ENCRYPT_OPTSDECRYPT_TOOL=$CRYPT_TOOL $DECRYPT_OPTSNC_PORT=8888
tar -cf – -C $FILE_DIR $FILE_NAME | pv -s `du -sb $FILE_FULL | awk ‘{s += $1} END {printf “%d”, s}` | $ZIP_TOOL | $ENCRYPT_TOOL | nc -l -p $NC_PORT -q 0 &
ssh $REMOTE_HOST “nc $LOCALHOST $NC_PORT | $DECRYPT_TOOL | $ZIP_TOOL -d | tar xf – -C $REMOTE_FILE_PATH/$FILE_DIR