Occasionally I need to copy nontrivial amounts of data from one machine to another. I describe in this article three approaches to doing this on the command line, and explain why the third, using ssh and tar, is the best one.
As test data, I decided to use RVM’s hidden control directory, ~/.rvm. I deleted my non-MRI 1.9 rubies to reduce the transfer size.
I haven’t tested this, but I imagine that for installing rvm on multiple similar systems (e.g. those with compatible native compilers, libraries, etc.), it may be possible to save a lot of time by a full install of rubies and gems on only one machine, then on the others doing a minimal install of rvm and then copying the fully populated .rvm directory.
Not So Good — Using scp
Note: This approach requires that the ssh port (22) be open on the destination host, and sshd is running. On the Mac, this is done by enabling “Remote Login” in the Sharing Preferences.
A very simple way to do this is to use scp (secure copy, over ssh) with the -r (recursive) option. For example:
scp -r source_spec destination_spec
…where source_spec and destination_spec can be local or remote file or directory specifications. (I’ll use the term filespec to refer to both.) Remote filespecs should be in the format user@host:filespec. Don’t forget the colon, or the copy will be saved to the local host with a strange name! Here is an example that works correctly:
# To create ~/.rvm on the destination:
>time scp -rq ~/.rvm kbennett@destination_host:~/temp/rvm-copy/using-scp/
Password:
scp -rq ~/.rvm kbennett@destination_host:~/temp/rvm-copy/using-scp/ 25.38s user 40.99s system 3% cpu 31:12.66 total
When I tried this, I was astonished to see that the destination directory consumed more than twice as much space as the original! To easily get the amount of space consumed by a directory tree, with the size in human readable format, run du -sh directory_name. For example:
# At the source:
>du -sh .
427M .
# At the destination:
>du -sh .
1.1G .
Continue reading Copying (RVM) Data Between Hosts Using ssh, scp, and netcat →