working with locally-mounted remote filesystems and remote CPUs

[gusl]

sshfs (SSH filesystem) is really really handy! By mounting a remote directory locally, I can view / edit / attach / copy remote files as if they were local, bypassing awkward commands and password prompts (not to mention the junk that is left behind after running scp).

The only thing that bothers me a bit is that when I run things while in a locally-mounted remote directory, I have the illusion that they are running remotely. But in fact, it is my own CPU that is running and heating up... and it's probably slow for reading and writing to file. (In an ideal world, it would be the other way around: a remote process would be accessing the files in my HD, but that's beside the point.)

I'm wondering if there's a nice way to start remote processes from locally-mounted remote drives... a way that (a) understands where I am running the process from (i.e. which directory I'm in), (b) that can resolve references to locally-mounted remote files into actual remote files, and (c) has a simple syntax, like e.g. a "*" after the command name to indicate "run remotely", so that Rscript* run.R | grep pattern would run Rscript on the remote CPU, and grep with the local CPU.

Comments

[demarko (from livejournal.com)]

isn't that essentially an RPC on the system that you're viewing locally?

I guess, if you think about it, FTP executes some pretty basic commands like that.

[gustavolacerda.livejournal.com]

yes, thanks! http://en.wikipedia.org/wiki/Remote_procedure_call

How would you make RPCs from a bash shell? Anything that approximately satisfies (a-c) above?

[demarko (from livejournal.com)]

I'm not entirely sure actually, I just know that's something that FTP "pretends" to do

[darius.livejournal.com]

Plan 9 could do this, I'm pretty sure. But in Unix/Windows I don't know.

[demarko (from livejournal.com)]

>_> Plan 9 can do a lot of things, if only someone could use it

[peamasii.livejournal.com]

Not to my knowledge, you'd have to mount your local drives remotely (using something like nfs/automount) and then run the remote processes in a shell.

[edanaher.livejournal.com]

A while back I wrote up a hack that I think does what you want:

http://www.contrib.andrew.cmu.edu/~edanaher/rem

Basically, "rem command" checks if it's in an sshfs-mount, and if it is, grabs the user, host, and directories from both sides, ssh's over, moves into the directory your in, and runs the command. It's probably pretty brittle - I don't use it for much beyond moving and md5summing files.

Also, note that it depends on the sshfs being named something in the form sshfs#user@host:dir (either via fstab or -ofsname), which seems to be required anyway for non-root unmounting. Of course, it's about 30 lines long so it shouldn't be too hard to tweak.

[robbat2.livejournal.com]

Look at OpenPBS/Torque basically.
That grep however really should be on the remote side.