Nc

Netcat is a very handy little tool for extremely basic, unencrypted inter-machine connectivity. Let's say you wanted to pipe a file between programs on two computers; you could use ssh. Here's a trivial example:

you@onebox:~$ echo testing123 | ssh me@twobox.mynetwork.local "mail -s test me@myemail.tld"

This works fine; assuming that you've got a mail program configured and working on otherbox, otherbox will send you an email with "testing123" in the body. The advantage to doing it this way is that ssh is a very mature technology with heavily tested encryption, authentication, and everything else you need to keep twobox secure even though it's reachable over a potentially untrusted network.

But what if you're on a completely trusted network, and you don't want the overhead (and slowdown) involved in encrypting the stream?

you@onebox:~$ echo testing123 | nc -l 3333

And:

you@twobox:~$ nc onebox.mynetwork.local 3333 | mail -s test me@myemail.tld

What you did here is open a listening port on TCP 3333 on onebox, and pipe your source data to it - then open a connection from twobox to onebox:3333 (TCP), and pipe whatever comes out of that connection to your mail command. The net effect is the same, but it takes place without encryption or authentication, which on a local, trusted, fast network can mean an order of magnitude or more in better throughput.

What if you've got A LOT of data to move from point A to point B? One thing you don't get is any indication of progress. pv comes to the rescue here. For example, recently I needed to move a ZFS snapshot from a pool named "backup" on one machine, to a dataset named "backup" on a pool named "data" on another machine:

me@onebox:~$ screen
me@onebox:~$ sudo zfs snapshot backup@1
me@onebox:~$ zfs list
NAME       USED  AVAIL  REFER  MOUNTPOINT
backup    1015G  1.69T  1015G  /backup
backup@1   153K      -  1015G  -
me@onebox:~$ sudo zfs send backup@1 | pv -s 1015G | sudo nc -l 3333

and

me@twobox:~$ screen
me@twobox:~$ sudo nc onebox.mynetwork.local 3333 | pv -s 1015G | sudo zfs receive data/backup

With that, I got my ZFS snapshot started synchronizing between my two machines. The two machines are both on a remote network from my workstation, so I started the process in a screen on both sides so it wouldn't get interrupted if my ssh session timed out, and I can resume the session if I need to. The pv command on both boxes means each box will show a nice progress bar with time elapsed, data transferred so far, current bandwidth, and estimated time remaining. And since I'm using nc instead of ssh for the actual transfer, my terabyte of data won't have to be encrypted, so it will go at gigabit speeds across that LAN rather than being limited by how fast the two machines can encrypt and decrypt it.

Remember, though - there is no security with netcat - that's the whole point! So only use nc on a completely trusted network.