Linux SSH Tunnel

TODO
http://nerderati.com/2011/03/17/simplify-your-life-with-an-ssh-config-file/

switchyomega
https://github.com/FelisCatus/SwitchyOmega/wiki/PAC-profile
If the scripts are contained in the profiles, the raw scripts will be sent to the browser. Otherwise, the URLs will be told to the browser, and the browsers will decide when to download them. However, if a result profile is a PAC profile, the extension will always try to download the script by the URL. The scripts will be executed to determine the proxies every time a request is made by the browser.

https://github.com/ClockworkNet/cmc
Once the utility is installed, background connections can be created by calling `cmc -O`.
This utility is largely deprecated by the addition of ControlPersist in OpenSSH 5.6. It allows ControlMaster sessions to be configured to timeout instead of relying on a utility like cmc to manage them.

cmc makes SSH ControlMaster sessions easy. SSH ControlMaster sessions have the following benefits:

• When using a SSH Bastion, you will only be prompted for (two-factor) authentication once.
  • Hosts behind the SSH Bastion can be accessed "directly" by proxying through the SSH Bastion (see workflow).
• Sessions to or through ControlMaster hosts do not need to create a new connection (SSH will be faster for most tasks)

    -O      open all ControlMasters defined in SSH_CONFIG

https://github.com/ClockworkNet/cmc/blob/master/docs/annotated_config.rst

4. ForwardAgent no specifies that the authentication agent will not be forwarded. This prevents administrators on untrusted remote servers from masquerading as you on any system on which you have your SSH public key. See SSH Agent Hijacking for more information.

This section is for a server on the Internet that acts as a SSH bastion. It provides access to servers behind a firewall.

# bastion
Host bastion bastion.example.com
    HostName bastion.example.com
    ControlMaster auto
    ControlPath ~/.ssh/master-%r@%h:%p

1. ForwardAgent yes specifies that the authentication agent will be forwarded to the remote server.
   • This is important for the bastion server as it allows public key sessions from the bastion to other servers (especially those behind the firewall). This means you will be able to connect to those servers without a password.
2. ControlMaster auto indicates SSH should listen for connections on a control socket. Additional sessions can connect to this socket and reuse the master instances (bastion's) network connection rather than initiating a new one.

1. ProxyCommand ssh -q bastion nc -w30 %h %p specifies the command to use to connect to the server.
   • This allows the connections to servers behind the firewall using the bastion server as a proxy. Any SSH client (ex. ssh command line, svn, Transmit) will see the production session as a single connection. It just works!

https://github.com/ClockworkNet/cmc/blob/master/docs/workflow.rst

1. You have already created a SSH key pair and added the secret key to your Mac keychain.
   • Test to see currently available keys:
     1. ssh-add -L
   • SSH key pair creation example:
     1. ssh-keygen -b 4096 -C USERNAME@COMPUTER_DESC
     2. ssh-add -K

1. Establish control sessions at the start of your day/session/etc.
   • cmc -o bastion or cmc -O
   • This establishes a control master connection in the background. It will stay connected and available until it is closed or connectivity is lost

Dynamic port forwarding
`ssh -fND 3000 bastion

-DsocksProxyHost=127.0.0.1 -DsocksProxyPort=3000
System.setProperty("socksProxyHost", "127.0.0.1");
System.setProperty("socksProxyPort", "3000");

curl --socks5 localhost:3000 "http://{$HOSTNAME}:8983/solr/admin/action=clusterstatus&wt=json”

https://community.hortonworks.com/articles/5469/working-with-firewalled-hdp-via-ssh-tunnel-and-soc.html

You can use SSH to open a local port that connects to a remote environment and behaves like a SOCKS proxy. Once this tunnel is established, you can configure your web browser to use the proxy and all web traffic will be routed over the tunnel and into the cluster environment (behind the firewall where the environment is open). The following command will open a tunnel to the machine gateway.hdp.cluster which has SSH enabled:

ssh -D 8080 -f -C -q -N username@gateway.hdp.cluster

Parameters map to the following:

• -D the local port to listen on
• -f send this ssh operation into the background after password prompts
• -C use compression
• -q quiet mode --> suppress warnings and diagnostic messages
• -N do not execute remote command or wait for the user to provide any commands

https://www.ssh.com/ssh/config/

Host

Restricts the following declarations to be only for those hosts that match one of the patterns given after the keyword. The pattern is matched against the host name given on the command line.

Dynamic forwarding in OpenSSH is done using the -D switch or DynamicForward configuration statement:
$ ssh -D1080 server
or:

# ~/.ssh/config:
host server
  DynamicForward 1080

As with static (-L) port forwarding, this command causes the SSH client to listen for TCP connections on the given port (here 1080, the standard SOCKS port). Note, however, that there's no argument specifying a target socket for the forwarding! That's because this sort of forwarding is completely flexible: each connection to port 1080 can go to a different remote socket, given at connection time via the SOCKS protocol. Set any network client with SOCKS support to use this SSH-forwarded port as its "SOCKS server,"

Just as with static forwarding, a dynamically forwarded port by default listens only on the loopback address. Use the -g switch to have it listen on all host addresses
https://blog.emacsos.com/use-socks5-proxy-in-curl.html

Suppose you have a socks5 proxy running on localhost:8001.

In curl >= 7.21.7, you can use

curl -x socks5h://localhost:8001 http://www.google.com/

Many tools use libcurl internally or use curl command in their installer script. If it's difficult to modify the command line itself, you can set proxy using environment variables.

env ALL_PROXY=socks5h://localhost:8001 PROGRAM [OPTION]...

If you want to overwrite system proxy settings, you may also need to set two more variables:

env http_proxy=socks5h://localhost:8001 HTTPS_PROXY=socks5h://localhost:8001 ALL_PROXY=socks5h://localhost:8001 PROGRAM [OPTION]...

Note that http_proxy is lower case, the other two is upper case.

https://sanctum.geek.nz/arabesque/ssh-socks-and-curl/

First of all we’ll make an SSH connection to the machine we’d like to act as a SOCKS proxy, which has access to the network services that we don’t. Perhaps it’s the only publically addressable machine in the network.

$ ssh -fN -D localhost:8001 remote.example.com

In this example, we’re backgrounding the connection immediately with -f, and explicitly saying we don’t intend to run a command or shell with -N. We’re only interested in establishing the tunnel.

If the tunnel setup fails, check that AllowTcpForwarding is set to yes in /etc/ssh/sshd_config on the remote machine:

AllowTcpForwarding yes

Note that in both cases we use localhost rather than 127.0.0.1, in order to establish both IPv4 and IPv6 sockets if appropriate.

We can then check that the tunnel is established with ss on GNU/Linux:

# ss dst :8001




-D [bind_address:]port
             Specifies a local ``dynamic'' application-level port forwarding.  This works by allocating a socket to listen to port on the local side, optionally bound to the specified bind_address.  Whenever a con-
             nection is made to this port, the connection is forwarded over the secure channel, and the application protocol is then used to determine where to connect to from the remote machine.  Currently the
             SOCKS4 and SOCKS5 protocols are supported, and ssh will act as a SOCKS server.  Only root can forward privileged ports.  Dynamic port forwardings can also be specified in the configuration file.

http://ericholscher.com/blog/2009/mar/21/really-easy-ssh-tunneling/
how to use the SOCKS proxy ability of SSH to allow you to tunnel your HTTP traffic through a remote server.

https://debian-administration.org/article/449/SSH_dynamic_port_forwarding_with_SOCKS

With standard SSH port forwarding, you could enter the command:

ssh -L 2525:mail.example.net:25 shell.example.org

This will forward port 2525 on your machine to port 25 on mail.example.net, by way of shell.example.org. You will then need to configure your mailer to send mail to localhost, port 2525, and use the authentication information for your mail account on mail.example.net

To avoid all this hassle, SSH also supports dynamic port forwarding via SOCKS. SOCKS defines a standard mechanism for a client to connect to a server by way of a proxy. SSH can serve as the proxy, allowing you to connect to shell.example.org and make connections from there to an arbitrary server such as mail.example.net. Simply run:

ssh -D 1080 shell.example.org

to make the connection to shell.example.org and start a SOCKS proxy on localhost port 1080.

In order to make use of the SOCKS proxy, you can either use applications which can speak SOCKS natively, or you can use a socksifier program like tsocks. tsocks provides a library used with LD_PRELOAD, which replaces the standard sockets functions like socket, connect, and sendto with functions that make use of a designated SOCKS proxy. The tsocks script runs a program with this library loaded. The library will read /etc/tsocks.conf to find out what SOCKS proxy to use. To configure tsocks to work with an SSH SOCKS proxy on localhost, edit the default /etc/tsocks.conf, change the server variable to 127.0.0.1, and comment out the path example.

Now that you have tsocks configured, you can run the following whenever you want to send mail via mail.example.net:

ssh -D 1080 shell.example.org
tsocks thunderbird

http://www.earthvpn.com/ssh-tunnelsocks-proxy-setup-tutorial-for-chrome-windows/

You can configure OpenSSH so you do not have to enter a password each time you connect to a server (remote system). To set up this feature, you need to generate a personal authentication key on the client (local system), place the public part of the key on the server, and keep the private part of the key on the client. When you connect to the server, it issues a challenge based on the public part of the key. OpenSSH then uses the private part of the key to respond to this challenge. If the client provides the appropriate response, the server logs you in.


host myserver
    Hostname 192.168.1.23
    Port 22
    User jdoe
https://www.cyberciti.biz/faq/linux-unix-ssh-proxycommand-passing-through-one-host-gateway-server/

Host fooserver HostName FooServer User vivek ProxyCommand ssh vivek@Jumphost nc %h %p

Save and close the file. Where,

1. Host fooserver : Set nickname of your choice.
2. HostName FooServer : Set the real remote server/host name.
3. User vivek : Set the real user name for remote server/host.
4. ProxyCommand ssh vivek@Jumphost nc %h %p : Specifies the command to use to connect to the server. In this example, I’m using nc command. Any occurrence of %h will be substituted by the host name to connect, %p by the port, and %r by the remote user name.

To test enter:
$ ssh fooserver

sftp -o 'ProxyCommand=ssh %h nc firewall.nixcraft.net.in 22' \ -o 'HostKeyAlias=firewall.nixcraft.net.in' \ vivek@server1.nixcraft.net.in
https://backdrift.org/transparent-proxy-with-ssh

#~/.ssh/config   Host superchunk.example.org ProxyCommand ssh user@bastion.example.org nc %h %p

In the above we are telling ssh that when it establishes a connection to superchunk.example.org to do so using the stdin/stdout of the ssh ProxyCommand as a transport. The ssh ProxyCommand then tells the system to first ssh to our bastion host and open a netcat connection to host %h (hostname supplied to ssh) on port %p (port supplied to ssh).

The result is a connection as if you were connecting from a trusted host:

$ ssh superchunk.example.org Password: user@superchunk.example.org's password: Last login: Wed Jun 25 12:05:47 2008 from 10.0.0.221 [user@superchunk ~]$

Now you may be wondering why it prompted me for two passwords. This is because we are effectively sshing into two systems one right after the other. This can be resolved through the use of pre-shared ssh keys or with more advanced methods such as kerberos ticket forwarding.

http://ufasoli.blogspot.com/2013/11/multi-hop-ssh-tunnel-howto-creating-ssh.html
If you prefer you can do all of the above steps in one giant SSH command using the -t flag to chain commands

ssh -v -L38080:localhost:38080 ufasoli@host1  -t ssh -v -L38080:localhost:38080 ufasoli@host2 -t ssh -v -L38080:localhost:8080 ufasoli@host3

https://blog.remibergsma.com/2013/05/28/creating-a-multi-hop-ssh-tunnel-by-chaining-ssh-commands-and-using-a-jump-host/

You could do this manually:

1	ssh -l user jump-host

and then from that server:

1	ssh -l user webserver.dmz

But using the -t switch, you can chain them together like this:

1 2	ssh -A -t -l user jump-host \ ssh -A -t -l user webserver.dmz

The -A switch enables forwarding of the ssh-agent. When using key based authentication, you’ll be able to login with typing the certificate’s password only once.

Using this technique, you can also build a SSH tunnel through the jump host:

1 2 3 4

ssh -A -t -l user jump-host \ -L 8080:localhost:8080 \ ssh -A -t -l user webserver.dmz \ -L 8080:localhost:8080

http://linuxcommand.org/man_pages/ssh1.html

     -t      Force pseudo-tty allocation.  This can be used to execute arbi-
             trary screen-based programs on a remote machine, which can be
             very useful, e.g., when implementing menu services.  Multiple -t
             options force tty allocation, even if ssh has no local tty.

     -A      Enables forwarding of the authentication agent connection.  This
             can also be specified on a per-host basis in a configuration
             file.

             Agent forwarding should be enabled with caution.  Users with the
             ability to bypass file permissions on the remote host (for the
             agent’s Unix-domain socket) can access the local agent through
             the forwarded connection.  An attacker cannot obtain key material
             from the agent, however they can perform operations on the keys
             that enable them to authenticate using the identities loaded into
             the agent.

     -l login_name
             Specifies the user to log in as on the remote machine.  This also
             may be specified on a per-host basis in the configuration file.

     -L [bind_address:]port:host:hostport
             Specifies that the given port on the local (client) host is to be
             forwarded to the given host and port on the remote side.  This
             works by allocating a socket to listen to port on the local side,
             optionally bound to the specified bind_address.  Whenever a con-
             nection is made to this port, the connection is forwarded over
             the secure channel, and a connection is made to host port
             hostport from the remote machine.  Port forwardings can also be
             specified in the configuration file

http://stackoverflow.com/questions/42505339/why-use-t-with-ssh

To have interactive prompt in your ssh shell, you need to have allocated PTY also on the server side. It is done automatically when you call ssh host.

When you allocate this PTY on server, then your local terminal and the remote one are exchanging some additional messages (Terminal control characters), which give the remote shell information about the size of your local terminal, the remote can update title of your window and so on. This is something you really don't want when you want to transfer files or just pass the output "as it is". It would modify that and you would get generally something else. Again, this is done automatically in case you use scp or just noninteractive script as ssh host my_script.

http://stackoverflow.com/questions/4426280/what-do-pty-and-tty-mean

"tty" originally meant "teletype" and "pty" means "pseudo-teletype".

In UNIX, /dev/tty* is any device that acts like a "teletype", ie, terminal. (Called teletype because that's what we had for terminals in those benighted days.)

A pty is a pseudotty, a device entry that acts like a terminal to the process reading and writing there, but managed by something else.

http://netsec.ws/?p=283
https://help.ubuntu.com/community/SSH/OpenSSH/PortForwarding

There are three types of port forwarding with SSH:

• Local port forwarding: connections from the SSH client are forwarded via the SSH server, then to a destination server
• Remote port forwarding: connections from the SSH server are forwarded via the SSH client, then to a destination server
• Dynamic port forwarding: connections from various programs are forwarded via the SSH client, then via the SSH server, and finally to several destination servers

Local port forwarding is the most common type. For example, local port forwarding lets you bypass a company firewall that blocks Wikipedia.

Remote port forwarding is less common. For example, remote port forwarding lets you connect from your SSH server to a computer on your company's intranet.

Dynamic port forwarding is rarely used. For example, dynamic port forwarding lets you bypass a company firewall that blocks web access altogether. Although this is very powerful, it takes a lot of work to set up, and it's usually easier to use local port forwarding for the specific sites you want to access.

For example, say you wanted to connect from your laptop to http://www.ubuntuforums.org using an SSH tunnel. You would use source port number 8080 (the alternate http port), destination port 80 (the http port), and destination server www.ubuntuforums.org. :

ssh -L 8080:www.ubuntuforums.org:80 <host>

Where <host> should be replaced by the name of your laptop. The -L option specifies local port forwarding. For the duration of the SSH session, pointing your browser at http://localhost:8080/ would send you to http://www.ubuntuforums.org/.

In the above example, we used port 8080 for the source port. Ports numbers less than 1024 or greater than 49151 are reserved for the system, and some programs will only work with specific source ports, but otherwise you can use any source port number. For example, you could do:

ssh -L 8080:www.ubuntuforums.org:80 -L 12345:ubuntu.com:80 <host>

This would forward two connections, one to www.ubuntuforums.org, the other to www.ubuntu.com. Pointing your browser athttp://localhost:8080/ would download pages from www.ubuntuforums.org, and pointing your browser to http://localhost:12345/ would download pages from www.ubuntu.com.

The destination server can even be the same as the SSH server. For example, you could do:

ssh -L 5900:localhost:5900 <host>

This would forward connections to the shared desktop on your SSH server (if one had been set up). Connecting an SSH client to localhost port5900 would show the desktop for that computer. The word "localhost" is the computer equivalent of the word "yourself", so the SSH server on your laptop will understand what you mean, whatever the computer's actual name.

ssh -f user@personal-server.com -L 2000:personal-server.com:25 -N
The -f tells ssh to go into the background just before it executes the command. This is followed by the username and server you are logging into. The -L 2000:personal-server.com:25 is in the form of -L local-port:host:remote-port. Finally the -N instructs OpenSSH to not execute a command on the remote system.

Another useful feature of port forwarding is for getting around pesky firewall restrictions. For example, a firewall I was behind recently did not allow outbound Jabber protocol traffic to talk.google.com. With this command:

ssh -f -L 3000:talk.google.com:5222 home -N

I was able to send my Google Talk traffic encrypted through the firewall back to my server at home and then out to Google. 'home' here is just an SSH alias to my server at home. All I had to do was reconfigure my Jabber client to use localhost as the server and the port 3000 that I had configured.

Imagine you’re on a private network which doesn’t allow connections to a specific server. Let’s say you’re at work and imgur.com is being blocked. To get around this we can create a tunnel through a server which isn’t on our network and thus can access Imgur.

$ ssh -L 9000:imgur.com:80 user@example.com

The key here is -L which says we’re doing local port forwarding. Then it says we’re forwarding our local port 9000 to imgur.com:80, which is the default port for HTTP. 

Multi-hop ssh tunnel - howto : Creating a SSH tunnel with port forwarding between multiple hosts

1.Connect the local machine to host1 (create the first tunnel)

?

1st tunnel

1	[ufasoli@local]> ssh -L38080:localhost:38080 ufasoli@host1

2.Connect to host2 from host1 (create the second tunnel)

?

2nd tunnel

1	[ufasoli@host1]>ssh -L38080:localhost:38080 ufasoli@host2

3.Connect to host3 from host2 (create the third and last tunnel)

?

3rd tunnel

1	[ufasoli@host2]>ssh -L38080:localhost:8080 ufasoli@host3

4.Checking the result

Now if everything went as expected you should be able to see your tomcat application by firing your favorite browser with the and entering the target remote URL as a localhost URL with the 38080 port, like for example http://localhost:38080/mywebapp

http://backdrift.org/transparent-proxy-with-ssh

A connection is established to the bastion host

+-------+ +--------------+ | you | ---ssh---> | bastion host | +-------+ +--------------+

Bastion host runs netcat to establish a connction to the target server

+--------------+ +--------+ | bastion host | ----netcat---> | server | +--------------+ +--------+

Your client then connects through the netcat tunnel and reaches the target server

+-----+ +--------------+ +--------+ | you | | bastion host | | server | | | ===ssh=over=netcat=tunnel======================> | | +-----+ +--------------+ +--------+

So there are 3 things we need to have happen behind the scenes:

1. Ssh to bastion host.
2. Run netcat command on bastion host.
3. Connect to netcat tunnel.

Here’s how to use the ssh proxycommand

#~/.ssh/config   Host superchunk.example.org ProxyCommand ssh user@bastion.example.org nc %h %p

In the above we are telling ssh that when it establishes a connection to superchunk.example.org to do so using the stdin/stdout of the ssh ProxyCommand as a transport. The ssh ProxyCommand then tells the system to first ssh to our bastion host and open a netcat connection to host %h (hostname supplied to ssh) on port %p (port supplied to ssh).

http://www.cyberciti.biz/faq/linux-unix-ssh-proxycommand-passing-through-one-host-gateway-server/

Passing through a gateway or two

Instead of typing two ssh command, I can type the following all-in-one command. This is useful for connecting to FooServer via firewall called ‘Jumphost’ as the jump host:
$ ssh -tt Jumphost ssh -tt FooServer
$ ssh -tt vivek@Jumphost ssh -tt vivek@FooServer
$ ssh -tt vivek@Jumphost ssh -tt vivek@FooServer command1 arg1 arg2
$ ssh -tt vivek@Jumphost ssh -tt vivek@FooServer htop
$ ssh -tt vivek@Jumphost ssh -tt vivek@FooServer screen -dR
Where,

• The -t option passed to the ssh command force pseudo-tty allocation. This can be used to execute arbitrary screen-based programs on a remote machine. Multiple -ttoptions force tty allocation, even if ssh has no local tty.

Say hello to the ProxyCommand

The syntax is:
$ ssh -o ProxyCommand='ssh firewall nc remote_server1 22' remote_server1
$ ssh -o ProxyCommand='ssh vivek@Jumphost nc FooServer 22' vivek@FooServer
##########################################
## -t option is needed to run commands ###
##########################################
$ ssh -t -o ProxyCommand='ssh vivek@Jumphost nc FooServer 22' vivek@FooServer htop
The netcat (nc) command is needed to set and establish a TCP pipe between Jumphost (or firewall) and FooServer. Now, my laptop (local system) is connected to Jumphost it now connected FooServer. In this example, the utility netcat (nc) is for reading and writing network connections directly. It can be used to pass connections to a 2nd server such as FooServer.

Update ~/.ssh/config file

Edit the $HOME/.ssh/config file using a text editor such as vi, enter:
$ vi ~/.ssh/config
Append the following configuration:

Host fooserver HostName FooServer User vivek ProxyCommand ssh vivek@Jumphost nc %h %p

Save and close the file. Where,

1. Host fooserver : Set nickname of your choice.
2. HostName FooServer : Set the real remote server/host name.
3. User vivek : Set the real user name for remote server/host.
4. ProxyCommand ssh vivek@Jumphost nc %h %p : Specifies the command to use to connect to the server. In this example, I’m using nc command. Any occurrence of %hwill be substituted by the host name to connect, %p by the port, and %r by the remote user name.

sftp -o 'ProxyCommand=ssh %h nc firewall.nixcraft.net.in 22' \ -o 'HostKeyAlias=firewall.nixcraft.net.in' \ vivek@server1.nixcraft.net.in

http://unix.stackexchange.com/questions/188285/how-to-copy-a-file-from-remote-server-to-local-machine

The syntax for scp is:

If you are on the computer from which you want to send file to a remote computer:

scp /file/to/send username@remote:/where/to/put

Here the remote can be a FQDN or an IP address.

On the other hand if you are on the computer wanting to receive file from a remote computer:

scp username@remote:/file/to/send /where/to/put

scp can also send files between two remote hosts:

scp username@remote_1:/file/to/send username@remote_2:/where/to/put

So the basic syntax is:

scp username@source:/location/to/file username@destination:/where/to/put

http://sshmenu.sourceforge.net/articles/transparent-mulithop.html

Netcat (often abbreviated to nc) is a computer networking utility for reading from and writing to network connections using TCP or UDP. Netcat is designed to be a dependable back-end that can be used directly or easily driven by other programs and scripts. At the same time, it is a feature-rich network debugging and investigation tool, since it can produce almost any kind of connection its user could need and has a number of built-in capabilities.

https://www.sans.org/security-resources/idfaq/what-is-a-bastion-host/2/11
A bastion host is a computer that is fully exposed to attack. The system is on the public side of the demilitarized zone (DMZ), unprotected by a firewall or filtering router. Frequently the roles of these systems are critical to the network security system. Indeed the firewalls and routers can be considered bastion hosts. Due to their exposure a great deal of effort must be put into designing and configuring bastion hosts to minimize the chances of penetration.