Another way to beat The GFW - tunnel out with your own secure proxy server
A normal Great Firewall connection error.
Quite a lot has been written recently on how to get around the increased restrictions on internet access from China. Yes it sucks, but there's a million or so ways around it. A lot of them use proxies. Unfortunately many are dependent on third-parties, and in particular, unknown third parties. In my mind, if you use open IP lists to gain unencrypted internet access, you're just asking for trouble.
One of the best ways to ensure unrestricted, and anonymous internet access is through the use of virtual private networks. There are a number of commercial solutions that are easy to install on PC, Mac or Unix and really don't cost all that much money. Nonetheless, they do cost money and often have secure web traffic quota restrictions which if you use the web a lot, could cause you some problems.
However if you're like me and want to save a few RMB and already host your own web-site on a Unix-based server you may want to set-up your own secure proxy server through the use of SSH tunnelling. Sounds difficult? Well, it isn't. Thanks to matt-man for hooking me up with this system.
The Concept
What we are going to do is to create a private proxy on your Unix webserver that you can use to access the internet. Access to the private proxy will only be allowed if you have login privileges to the webserver. To access the webserver, you will use an SSH connections to ensure traffic between your computer and your proxy server is encrypted (thus unobservable). The last step will allow you to get past the firewall.
Requirements.
- A Unix-based webserver outside the firewall (work, China, etc) that allows SSH connections.
- Knowledge and familiarity with FTP, SSH and basic Unix navigation commands (cd, pwd, find, mv).
- A little Unix compilation knowledge.
- The ability to read all of these instructions first and then follow them (very important).
Explanation of the Unix Lines here
Before we start, I'm just going to qualify that in the below examples of Unix line, DO NOT include the following components as part of the commands. These just signify where you should be issuing the commands from (like on the Unix server, or your own computer). I hope this explanatory. The $ sign signifies the command prompt.
webserver:~ user$
mycomputer:~ me$
Free Tools
elhttp - The extra light HTTP proxy server that runs on Unix systems (in this case, your webserver).
SSHKeychain - A key manager for Mac OSX that can also forward local ports (on your computer) over a SSH connection (to your webserver).
From what I've read, you could also give the more robust privoxy a shot instead of elhttp. I can only vouch that elhttp works under this situation.
I'm writing this assuming you use OSX for your home computer system (like me), but don't fret if you've got a PC because you can use many of the other SSH managers (that may have to be paid for) available for PC or Linux. If you don't want to spend the bucks or are a purist, you can use command line SSH programs like PC's PuTTY or OSX's good ol' Terminal instead. I'll cover how to do this in the below notes section.
Installing elhttp on your webserver
- Download the tgz archive of the latest elhttp version (I used elhttp-0.22.tgz). Let's call this file elhttp.tar.gz for this example.
- FTP the elhttp.tar.gz file to your desired folder or subdomain on your webserver (FTP). I prefer to use a subdomain that has its own unique user just for added security. This user must have SSH access enabled, or this whole thing won't work.
- SSH to your webserver and navigate to the folder or subdomain where you placed the elhttp.tgz archive.
- Untar and decompress the elhttp.tgz using the following command structure.
- This will produce a folder named elhttp. If this name differs from yours, you can rename the folder using the mv unix command.
- Navigate into the new elhttp folder using the cd unix command.
- Read the README text file in this folder on how to 'install' elhttp. For the bulk of you, you shall choose the linux option like below.
webserver:~ user$ tar -xvzf elhttp.tar.gz
webserver:~ user$ cd elhttp
webserver:~ user$ ./make linux
Configuration of elhttp on your webserver
We will now configure the elhttp proxy server. The commands that we will call will ensure that any proxy traffic will only be accepted from our own webserver (referred to this case as 127.0.0.1 or localhost). Also, we will specify that any traffic accepted for this proxy must come through the 4222 port (ssh connection port). We will also specify a network mask (in our case the classic subnet mask 255.255.255.0), a logfile (proxy_log.txt) and what type of connection methods we will allow (2, any connection method allowed like http or https for example).
webserver:~ user$ ./elhttp 4222 127.0.0.1 255.255.255.0 proxy_log.txt 2
Here's the syntax for the elhttp command.
elhttp [proxy port] [authorized ip] [network mask] [logfile] [CONNECT method flag]
A full explanation of the syntax of the command can be found here.
Ok, we've now setup our extra light http server! Now what we have to do is connect to it from our own computer. And how will we do this? By SSH of course!
Install and configure SSHKeychain to connecting to elhttp proxy server.
SSH stands for Secure Shell. You can read up on SSH at Wikipedia after you've connected to your elhttp installation, but for now let's get started with the installation and configuration of SSHKeychain, which will manage your SSH connection to your elhttp proxy server.
- Download the SSHKeychain binary from the developer's website. Install the program as directed.
- Start SSHKeychain
- Open Preferences/Display. Choose to display SSHKeychain in the Status Bar and Dock.
- Open the Preferences/Tunnels tab. Create a new tunnel by pressing the + button.
- General -> give your tunnel a name. Select the other options as desired (I leave these off as a default).
- SSH -> Enter details for your webserver. Hostname = my.domain.com, Port = 22, Username = myusername.
- Local Ports -> Local Port = 4222, Remote Host = localhost, Remote Port = 4222
- Remote Ports -> Leave blank.
- Save the settings.
The key steps here are the local port forwarding to our remote server that hosts the elhttp installation. Here we are pushing traffic from our computer through our computer's 4222 port. This will then access our webhost elhttp system by using localhost (i.e. 127.0.0.1) and the port elhttp allows to access the internet 4222.
Create a SSH tunnel to elhttp using SSHKeychain.
This is really easy. Earlier, you configured SSHKeychain to be present on both the Menu Bar and the Dock. Go to the menu bar icon for SSHKeychain (a set of keys), select it and the Tunnels submenu.
Select your Tunnel configuration. Wait a few moments while SSHKeychain contacts your webhost and it responds with a password challenge. Enter the password for the webserver account you are using to access your webserver. If the connection is allowed, you will notice that the Tunnel configuration in the menu bar listing will now have a tick.
Guess what? You're tunnel is connected, and now you can use elhttp to browse the internet. All you need to do is tell your web browser to use this connection.
Setting up your web applications to use your proxy server.
This is the easiest step left to do. What you need to do here is to ensure that your web applications that you want to use your elhttp server can find the right path to access the server.
Essentially, all you have to do is point your http and https proxies to the SSHKeychain program. To do this, put the following proxy options into either your Firefox browser or your OSX System Preferences/Network/Proxies settings. Have a look at the below image and fill in as indicated.
In this situation, localhost (127.0.0.1:4222) refers your web browser to connect to the internet via the 4222 port on your own computer.
Save and apply your proxy settings!
Notes about this whole setup.
The elhttp proxy that has been built here creates a tunnel through a network firewall. It does not anonymise your IP address at all. In this way you can accept cookies, leave a trail through the internet, and also access a lot of websites that may not usually allow the use of free proxy services that also anonymise your identity.
Your webserver is a highly dependent factor in this whole escapade. Again, this can only work if you have SSH access to your webserver. If you don't, you can't use this method, but maybe someone with SSH access to another server can set this up for you if you ask very very nicely. I can also imagine that your webhost may not allow the functioning of elhttp for various reasons, in particular some hacker's penchant for using elhttp's capabilities. For people with their own hosting providers,I can assure you this works on Dreamhost's virtual machine/shared servers and also other Apache-variants. They give you a lot of bandwidth to use, so I don't think they care too much about what you use this for. Very rarely, elhttp loses its configuration settings due to Dreamhost's tinkering. To fix this, just SSH to your account and reconfigure the proxy using the elhttp command used earlier. No worries otherwise.
SSHKeychain's main aim is to manage security keys, and thus will start managing Apple's default key manager, Keychain. Essentially what you will find is recurring requests to use your Apple User accounts authorisation password to access your stored passwords. I don't find this annoying, but you might. You can alter this relationship by reading more on the SSHKeychain online help.
I have also added SSHKeychain to my start up items, and if you wish, you can automatically create your SSH tunnel when SSHKeychain starts up.
Using an SSH tunnel to drive your web connections through can be a pain as it can slow your access. At least from Shanghai, I don't see any major slow-downs, unless I am accessing sites within China of course. So it is a good idea to be able to switch between normal network connection proxies and your elhttp connection in such situations. Do do this efficiently and quickly, I like to use the SwitchProxy Firefox extension plugin to allow changing of my proxy settings just that little bit easier. You should be using Firefox of course! Otherwise on Mac OSX, I would create new Network Locations to make proxy switching easier for Safari or other browsers. For non-Firefox using PC people, I'm sure there's an easy solution to your problem.
By the way, if SSHKeychain isn't on, and the tunnel isn't active, you can't use the 127.0.0.1:4222 proxy configuration to connect to the internet.
You can also port other traffic through your SSH tunnel including email, torrent downloads, video and audio streaming for example. However, that is up to you to configure if you want it. Commercial VPN solutions cover all these different type of connections and provide a cheap, complete solutions for unrestricted internet access.
Command Line Tunnelling
This is perhaps the most simple way on the command line to set up your SSH tunnel to your elhttp webserver.
mycomputer:~ me$ ssh -N -p 22 useraccount@my.domain.com -L 4222/localhost/4222
The above Endgadget link breaks down this ssh command as follows.
- ssh - the actual SSH command
- -N tells SSH we don't want to execute a remote command. Not terribly necessary, but makes it safer.
- user@my.domain.com = account and SSH host on the my.domain.com server.
- -L 4222/localhost/4222 This one creates the tunnel. It tells SSH to forward traffic from port 4222 on the local machine (your computer) to port 4222 on the remote machine (the elhttp proxy server you created).
When you create this connection, you will be challenged for the password for the user account you specified. Please note that the command line prompt will not change after this step as the connection is active. You can then configure your internet browser to utilise the connection as mentioned earlier. When you want to close the tunnel, hit Control+C. The elhttp proxy will then refuse connections.
Ok! If you read this carefully, and followed the directions you should now be able to reach sites like shown in these screenshots.
Pretty cool isn't it?
0 TrackBacks
Listed below are links to blogs that reference this entry: Another way to beat The GFW - tunnel out with your own secure proxy server.
TrackBack URL for this entry: /cgi-bin/mt/mt-tb.cgi/324.
I don't know too much about tunneling and proxies and all of this, but what are the advantages to using something such as elhttp? Why wouldn't following the steps detailed in http://lifehacker.com/software/ssh/geek-to-live--encrypt-your-web-browsing-session-with-an-ssh-socks-proxy-237227.php suffice?
Dude, you have way too much time on your hand. ;-)
I'm happy I don't live in China. I would start crying if I had to fry my brain with this geekstuff to stay in touch with the outside world. Now I can use all my brainpower on mind numbing computer games instead.
@Russell,
Essentially this is the same thing as both methods require (1) a SSH proxy server and (2) the SSH connection method to the proxy server. But the Lifehacker links to another one of their articles which shows you how to set up a SSH server on your own home computer/server.
The lifehacker example you cite is probably a good way to get past a work firewall as you'll probably have unrestricted network access at your home server. However, living in China is a bit different, and we want to poke outside the China internet access points (or overseas). This is why you need to install elhttp on a unix webserver outside of China's firewalls. Many bloggers/internet users in China host their own sites on Unix machines. elhttp can then be installed on those servers.
elhttp is a good way to create a simple proxy server on your hosted webspace. You just need to configure it to be secure as possible (hence elhttp will only route connections from the hosted server, and that requires a SSH connection and account user-password challenge).
@Peter
Ha! Maybe I just need to start a new hobby, like making drinks and soups with blenders eh? ;) Yeah, this is geeky. The beauty of this is that it only takes 10 minutes maximum to set this up. The verbose instructions are also meant to act as a tutorial.
Tim, I guess I wasn't quite clear with my question.
I am also in China, and I am making use of this SSH method, but I am doing so with an incomplete understanding. That is, I've thus far relied on friends to provide me with shell accounts to use, rather than setting up my own proxy (because, as you said, I am in China, setting it up at home doesn't improve much).
However, as far as I know, they didn't specifically install additional proxy software for me to use, only that they provided me with a shell account. If my webhost provided shell access (I know, I know, that they don't is perhaps a good reason to change) I'd have tried out this method through it, but I don't, so I haven't.
So, as I originally asked, what makes elhttp a necessary step? Am I doing something that is creating some massive vulnerability without knowing it?
Russell, The tunnel is just half of the story. Without something at the end of the tunnel point the traffic where to go, it won't connect to the internet.
Imagine an SSH connection as the tunnel, and the elhttp (or proxy) as the exit to the tunnel. Essentially SSH is a secure way of ensuring you connect directly between two computers via a network connection. In our example, we are making a tunnel to get past the great (fire) wall. But, a tunnel isn't any good without an exit or an escape plan.
In my example, you use an SSH tunnel connection to pass HTTP and HTTPS information from your computer to your unix server and vice versa. Once this connection is established, you send normal web traffic through this tunnel to your unix server. But when it reaches the unix server it needs to go somewhere. An SSH connection without a way to access the outside world is akin to a dead-end tunnel.
So on the server we need a man-hole. This is why we need our own proxy software on the server to route the traffic. The lifehacker article stipulates you need access to a ssh-compatible SOCKS proxy server to make their method work. In my example, elhttp acts as the proxy server. We need configure elhttp to handle traffic from the tunnel connection and route it to the wider internet. Your friends may have already set up a proxy handling software on server you have access to, or it may have been installed as a default on the server account. This is why you might not know about this situation.
The SSH tunnel between your computer and your unix server is encrypted using the SSH method. This will stop 3rd parties snooping on the traffic between your server and your computer (i.e. the great firewall). This makes traffic sent between your computer and your server secure. BUT... it does not protect you against content that is requested for or sent down the tunnel. So you could access cookies as well as malicious scripts, worms, viruses and the like. This method does not cloak your IP address, so you can't use it for hacking or to surf anonymously.
Also, the method I show above only pushes web browser HTTP and HTTPS traffic generated from your internet browser through the tunnel. It does not secure FTP, streaming protocols, or email client traffic. You will need to secure those using different methods.
I can't really explain this any more clearly. I'd check out a few Answers.com articles on how proxies work and relay information and also ssh connections if you have any further queries.
Sorry for any trouble I've evidently caused you. What I didn't understand in the case of the lifehacker article was where the socks proxy server was installed, set up, or configured.
The source of my confusion was not in how tunnels or proxying conceptually operates, but rather, why one would use elhttp instead of a socks proxy given that the impression I've gotten (possibly erroneously, I realize) is that in the latter case, there is less setup to be done.
Thanks for your attempts to clarify but it seems that I've been successively failing to clearly ask my question. I'll go ahead and try search engines again rather than continue to pester you.
No worries. Asking questions is how people learn. I was in your shoes a little while back so don't worry about it.
Tim: Whoops... Actually it was a cake. Anyway... Back to subject.... fast :-)
Hi,
I'm really sick of all this censorships. I'm in Iran and to be honest I think those who live in china have better life (in internet) because at least US companies won't restrict them so much. Any how I'm looking to have a private proxy server. I tried to find one from Google, but most of them are suggesting a software for this purpose. I download some of them and they either don't work (virtually any thing related to proxy is blocked here) or when they work it's works partially, meaning my IP changes to some thing else but I still face with Iran censorship. I've tried to find an open source project for this purpose but failed miserably. Now I have a hosting plan which support java plus some money in my moneybookers account.
My question is, what is your suggestion? Is there any thing (preferably in java) that I can run on my server and act as my personal proxy server? or is there any commercial service which give me private proxy server and how much it costs?
Best.
Commercial Virtual Private Network (VPN) providers is perhaps the most convenient option. I used the commercial VPN Anonybrowser for a while before learning how to create my the system I wrote about here. It is cheap (about USD$4.00 per month). There's a number of other companies offering similar services for the same rates. Wiitopia is another paid personal VPN service I have heard good things about.
The VPN services above are similar to what I described here, but they have more convenient authentication protocols (using security certifications) and more complex routing systems at the business end to make their clients browsing faster. For me in China, I haven't suffered from a slow-down in web traffic using the SSH tunnel + elhttp method described here.
There's also free community web sharing tools like Tor and Psiphon which is sort of like P2P sharing of internet access. In this case you are dependent on people sharing their unrestricted internet access, and thus sometimes can be really slow.
This is the beauty of SSH tunneling to your own proxy server. No slow downs, and is also secure so no one can view what you're browsing between the proxy server and your own computer.
The main thing is that if you want your own proxy server it must be outside the firewalled area, in your case Iran, and in my case China. You must also be able to access it using SSH. If your SSH connections are blocked you may need to go through further port-forwarding steps.
There's a few other web-based solutions you can stick on your website if it runs PHP. There's PHProxy (which I previously used), Squid and CGI-Proxy (used a little) which provide a HTTP interface to access certain domain addresses. Unfortunately not every one of these interfaces allow cookies, and sometimes they strip out cool web functions like Ajax and Javascript codings (making surfing a little less enjoyable).
That is why IMHO SSH tunneling with your own web proxy rocks. Faster than Tor/Privoxy doesn't suffer from the limitations of the web-based proxies, and doesn't cost USD4 per month for essentially the same service.
Great description of this process Tim! If you don't have access to a suitable machine in an uncensored country, but do have friends that have a windows machine on the internet in an uncensored country that don't mind you leeching their connection I would recommend looking into a Canadian project Psiphon -- http://psiphon.civisec.org/.
Hi Tim,
Thanks very much for hints. I signed up with them and it seems it works perfectly. Actually I don't have so much knowledge in this area and am a little worry. My main concerns are security and costs (not monthly fee). As I understand a server is shared with lots of people, so can it be dangerous if I insert important things like financial data? Actually financial things are the main reason I'm looking for a solution (some of companies are blocked here). And in regard to cost I'm ready to pay 10-15 per month, but don't want to pay for dialing to USA. It will be a big sum of money. I see it just work when I'm connected to internet with my ISP but still I'm a little worry.
Beside this, you talked about having your own proxy on your web site. I have a web site (running on linux) which support java too. How can I know if I can use the method you described above? Or can you suggest a hosting provider which let me to do it? Is my understanding correct that I will need vps to allow me have access to command line? Actually it seems support teams of hosting providers don't have enough information about details.
Thanks very much.
@Matthew - Thanks mate! Coming from a middle-ware guy that's a big complement!
@Dave,
With VPNs, you don't need to dial-out to the VPN provider. If you have any internet access, you can create a secure network tunnel to your VPN provider using their software systems.
You have to view a VPN service like a security guard. They provide you a way of protected internet access, but each VPN service has their own capabilties. Almost all of them will allow you to go through firewalls and to have unmonitored internet activity. However, they do not provide 100% protection. You have to take some responsibility for your own actions, just like walking down the street. Don't jump into the traffic for example.
If you use a VPN to connect to your bank, the traffic between the tunnel exit (VPN provider's connection to the internet) and your bank still needs to be encrypted. Talk to your bank/financial institution about their security methods. The good banks usually have 2 or 3 good ways to ensure that you are who you say you are when performing transactions.
Lastly, if you have paid for a VPN, you don't need to worry about elhttp or a proxy server. The VPN provider is your own paid-for proxy service! When you VPN is connected, it routes internet traffic through the VPN tunnel. Talk to your VPN provider about this for further detail.
Glad you found this informative.
-Tim
Hi every body,
I've done some research, some communication, find some good things and have some questions. Hoping you don't mind.
In regard to anonybrowser, I must confess their service is flawless as far as I can understand. Very good price ($4) for the functionality they are offering. I asked them if they can provide me with dedicated IP address (for some good reasons) and they said it will cost $10 per month. I know must of users won't need it at all. But if some one is going to use it for important things, I think it worth each penny to pay for dedicated IP.
My biggest problem with this service is that as soon as bots of government find out about their service it will be blocked completely. So I think it's better I have my own VPN server. After browsing the web, I find this hosting eapps.com. http://www.eapps.com/Docs/VPSStandardPrices.jsp and their standard plan.
You can see it's a VPS, with root access, dedicated IP and SSH. All for $10 per month. So my understanding is that I can use it as my hosting plus my vpn server. Please let me know if it's right or not. Please note that I think I can find my way based on your tutorial above and other resources. I just want to know it will do what I need.
Also my last question is that, is it possible to use ssl with vpn? I know the connection between vpn client and server is encrypted, but what about connection between vpn server and ssl based web server like paypal? Is it a good idea to use it for this purpose?
Still looking for freedom.
Best.