cp and scp

[gusl]

I am in a directory with a number of folders, each of which has a file named out.csv

I would like to copy only these files, while preserving directory structure.

If I do scp -r gusl@ubc.ca:my-project/*/out.csv my-project/ then all the copies of out.csv get copied into the base of my-project, erasing each other.

I want to specify which files match separately from the root directory from which I am recursively copying.

Is this one of the rare uses of the --copy-contents option?

Comments

[bhudson.livejournal.com]

Let me suggest:
(a) a short shell script, or
(b) the GNU tar --include-files option (or something like that).

scp is slow when multiple files are involved, so you usually want to run tar cf - a b c | ssh -c 'tar xf -' (or something like that -- I'm not testing these suggestions, just giving something similar to the truth)

[en-ki.livejournal.com]

I second this. In general, if I want to do something at all intelligent with transferring files via ssh, I do

find ... | xargs tar cf - ... | ssh -c 'tar xvf - ...'

or, if find isn't smart enough, a shell, Perl, or Ruby script to identify the files; or I will use rsync -e ssh if it's available on both ends.

I'm not aware of --copy-contents and I don't see it in the man pages for scp from OpenSSH 4.3p2 or OpenSSH 5.2p1, which suggests at a minimum that it is unportable. tar and find are present on any Unix system, though non-GNU finds and tars can get pretty stupid. (The only place I see "--copy-contents" on the tubes is in the "cp" manpage, and no, it's not the right thing to use there.)

[robbat2.livejournal.com]

Easiest solution would be rsync:
# rsync -avP --include='foo' --filter 'hide,! */' src/ dst/

--copy-contents is for inodes that are neither directories or files (fifos, device nodes, named pipes, symlinks etc).

[infryq]

Bingo. It's totally worth reading the huge nasty manpage for rsync if you want to make a pasrtial directory tree copy like that more than once.

[tedesson.livejournal.com]

rsync is a good choice.

A suggestion that hasn't been mentioned is unision, which will keep directories in sync between computers. Good if you want to do this over time, or if you have a 'home', 'work' and 'travel' system that you want to keep the same files on. It's like rsync, but at a higher level.

[chrisamaphone]

agreed! <3 unison.

[chrisamaphone]

(and it's made by benjamin pierce, so you know you can trust its semantics :)