Scenario
I am currently living in China and as many of you know, access to the web here is filtered, meaning sites such as YouTube, Flickr, Blogger, WordPress are often unavailable. Whilst previously I overcame this problem by using TOR and FoxyProxy and setting up rules telling Firefox which URLs should be redirected through the proxy, TOR is not exactly speedy and so although this setup was free, it was not really suitable for smooth viewing of any blocked multimedia websites.
Solutions
I started to look around for other possible proxy solutions but most pay-for proxies are charged on a monthly basis and are fairly expensive. Furthermore, there is always the possibility that the authorities here are already aware of the IP ranges of some commercial proxy servers and so you run the risk of being blocked out yet again. What I really needed was a pay as you go proxy server where you would only be charged when you actually use the server and for the amount of bandwidth used. I then came accross a post on Duane Storey’s blog which described how he created an Amazon EC2 instance which could be used to make a proxy server available that was only to Iranian web users. Duane’s solution involves his own custom Fedora image which has Apache setup to run as a web proxy. This setup was pretty much exactly what I needed – except that I would need to make sure the Proxy server only allowed my own computer to connect rather than making it a public proxy server accessible to Iranians. After a bit of testing, I managed to get something going and was pleased to see that the proxy was very speedy indeed!
Cheap Proxy Server
Amazon EC2 Instances currently cost $0.10 USD per hour to run plus bandwidth charges (which for standard web browsing will be negligent). This gives Amazon EC2 the potential to be an incredibly cheap on-demand proxy server. Finally you only pay for what you use rather than paying an extortionate monthly rate.
Automation
The next thing to do was to automate the server setup process as entering all the necessary commands into a terminal can be cumbersome and time consuming. I used a unix shell script to accomplish the following steps:
- Create an Amazon EC2 Fedora instance which is configured to run Apache as a proxy server.
- Connect to Amazon EC2 instance via SSH and modify the Apache configuration file to only allow the client’s computer to connect to the server.
- Restart Apache to apply changes.
- (Optional) Assign an Amazon Elastic IP Address to the Instance which can be used to access the HTTP Proxy. The script prints the IP address assigned to the proxy server in the terminal – this IP address can then be pasted into your web browsers proxy configuration settings.
- (Optional) Create a secure SSH HTTP Tunnel to the proxy server on port 9999. Clients can connect by changing browser network settings to a SOCKS 5 proxy on localhost:9999.
- The script then waits for keyboard input “stop” upon which the SSH HTTP tunnel will be closed, the Amazon EC2 Instance shutdown and any Amazon Elastic IP addresses released.
The script is designed to work on Ubuntu (Jaunty 9.04) but should also work on other Linux Operating Systems and possibly on Mac OS X too. Before you can get it up and running there is some preliminary preparation which is necessary.
Step 1: Sign up for an account and setup AWS
Before you can use Amazon Web Services, you need to sign up for an account at http://aws.amazon.com/.
This will give you two keys that you will need for accessing your Amazon Web Services: your “access key” and your “secret access key”. Once complete, you then need to sign up for Amazon’s Elastic Compute Cloud (EC2) service. As part of signing up for EC2, you will create an X.509 certificate. You will need to save your private and public keys in ~/.ec2 on your local machine. All this is described in more detail in the AWS “Getting Started Guide” under Setting up an account.
Step 2: Download and install EC2 command line tools
The process of setting up the tools is described under Setting up the Tools in AWS “Getting Started Guide“. Just do what that page says and note down the location of the tools directory.
Step 3: Generate an SSH keypair, disable strict key checking
Follow the instructions here to generate an SSH keypair to be used in combination with your Amazon EC2 Instance. The RSA private key generated needs to be placed in your ~/.ssh folder and should be given 600 permissions.
chmod 600 your_rsa_key
Another important thing for the script to work is to set OpenSSH to disable Strict Host Key Checking for amazon domains only. This has only minor security implications and is vital if you want the whole server setup process to be automated.
In ~/.ssh/ create a file called config and put the following text inside:
Host *.amazonaws.com StrictHostKeyChecking false
Give the file the right permissions
chmod 600 ~/.ssh/config
Finally, you need to open up ports in the EC2 firewall to allow SSH connections to the server instance that is created by the script. This can be done with the following command from the EC2 command line tools:
ec2-authorize default -p 22
Step 4: Install Curl
The shell script uses a command line utility called curl to determine the external IP address of your computer – hence allowing you exclusively to connect to the proxy server. On Ubuntu, curl can be installed using apt-get.
sudo apt-get install curl
Step 6: Download the script and configure variables
First download the autoproxy archive to somewhere in your home folder and uncompress it. You then need to make the shell script executable
chmod u+x autoproxy.sh
Open the file in a text editor and make changes to the below variables to fit your own setup:
export EC2_HOME=/home/username/folder/ec2
This is the location of the EC2 command line tools which were installed during Step 2.
export EC2_PRIVATE_KEY=~/.ec2/pk-LSOEEFCT2IYW74BHFUIHHSBE63H4GM77.pem
This is the location of your Amazon EC2 Private Key file which you generated when creating your AWS account.
export EC2_CERT=~/.ec2/cert-LSOEEFCT2IYW74BHFUIHHSBE63H4GM77.pem
This is the location of your Amazon EC2 Certificate file which you generated when creating your AWS account.
export JAVA_HOME=/usr/lib/jvm/java-6-sun/jre
Location of Java on your computer.
If you don’t know where Java is installed on your computer you can follow symbolic links in a terminal to find it:
ls -l /usr/bin/java
/usr/bin/java -> /etc/alternatives/java
ls -l /etc/alternatives/java
/etc/alternatives/java -> /usr/lib/jvm/java-6-sun/jre/bin/java
export ssh_key=~/.ssh/id_rsa-gsg-keypair
Location of your Amazon EC2 SSH private RSA key (created in Step 3). This should be located in ~/.ssh and needs 600 permissions.
export autoproxy=/home/username/autoproxy
Location of the autoproxy folder that you downloaded.
Running the script
To run the script, open up a terminal and type the following:
cd /home/*username*/autoproxy/ ./autoproxy.sh
Browser Config
If you chose to use an Amazon Elastic IP Address to access your proxy server, the script should output the address in the terminal once the proxy server is setup. In your browser you simply need to enter the proxy setup section and choose “HTTP Proxy” using the Amazon Elastic IP Address and port 80.
If you chose to use an SSH HTTP Tunnel to connect to access your proxy server, in your browser enter the proxy setup section and choose “SOCKS 5″ with the address as localhost on port 9999.
Disclaimer
Whilst using an Amazon EC2 Instance as a personal proxy server should not be a problem, it appears that Amazon do not approve of use of EC2 instances as public proxy servers available to the masses. For this reason please make sure you limit access to the proxy to your computer only as configured in the script. Use this script at your own risk. I cannot be held responsible for any undesired consequences that result from its use.
Anyway, have a try and see how you get on. Don’t hesitate to get in touch if you have any questions. I’ve spent quite a bit of time on this and I hope that it can help some other people out. I would appreciate any feedback or suggestions 
Tags: amazon, automated, aws, bash, 网络, ec2, 脚本, instance, Linux, proxy, script, server, shell, terminal, Ubuntu, 代理
I also use an amazon EC2 instance as a proxy server. My setup was much simpler… only needed to install FoxyProxy in my browser and cygwin.
The proxy used to work fine and I was able to access most sites without any problems. However recently I stopped being able to reach youtube. Have you had the same problem?
You should create an image so that it’s even easier to setup… Ping me if you do!
Hmm… even if I created my own custom image, the HTTPD conf file would still need updating with the clients own IP address and then an apache restart… or am I missing something?
this looks great!
Could the script be run from a windows box?
probably… but you would need to use Cygwin to get it going as the script is purely for unix environments
struggled a bit with Cygwin so got hold of an ubuntu vmware image and it works great apart from the odd intermitent failure (i think it’s the part that copies over the modified httpd.conf file).
I can get the EC2 server up and running within ubuntu, then connect to the proxy from windows.
Couple of things worth mentioning… I needed to set up the security groups in EC2 for HTTP, HTTPS and SSH to get it to work.
Probably also worth mentioning that when not using the elastic IP you should use the name or ip of the ec2 instance. Simple stuff but would help any AWS / linux novices like me!
Thanks!
Thanks for pointing those extra steps out of me – it seems that the guide isn’t as complete as I had thought. I will update it sometime this week when I have a moment or two to spare!
Ok… finally got round to making these changes today. The guide should be fairly comprehensive now!
你用的是什么中文字体啊？
那个字体叫作 Vera Sans Yuanti，用来显示中文的时候就更好看了！
你如果下载不到的话就用 email 联系我…
Amazing post! I happen to be in the same situation (ie. In China..) , and it would be great to get it to work, but I can’t figure out how to connect to my instance through Windows. putty keeps exiting when i select my key file.
Apparently Putty uses a different type of key file to openssh, you need to convert the SSH key to putty format.
In Ubuntu, install Putty from the repositories and run the following command, changing the paths to your own:
I believe that there are equivalent tools to do the same on windows.
在amazon ec2上没有找到速度测试的地方，上面有3个地点，不知道你当时用的是哪里的服务器？
它这个每小时是你登录以后1分钟也算1小时，还是每次使用的时间累加直到一小时才付款？
如果你自己不身处美国，你用哪个都行，速度的差异不会很大。我用的是 Amazon EC2 (US East – N. Virginia) 这个地区的服务器，速度一直都很快。
应该是每次使用的时间都累积起来，到月底付账… 是按分钟算的。
I’m trying to get this script working and I’m running into a problem. It looks like the EC2 instance you originally used is no longer available: ami-a37b9dca
$ ./autoproxy.sh
Create an Amazon EC2 Server Instance
Client.InvalidManifest: HTTP 404 (Not Found) response for URL http://s3.amazonaws.com:80/iranproxy/iranproxy.manifest.xml: check your manifest is still available in S3.
Wait for instance to load before preceding
Was there anything special about the ami? I’m guessing I might have to build my own. Did you have any special scripting or packages added to the base?
Hi Josh
This isn’t actually my own AMI but I believe it has an apache web server already installed and configured from the word go – which is why I decided to use it. I believe it is a Fedora installation. You can find the original blog post about the AMI on the author’s website.
I have just done a fresh backup today and the script seems to run fine… so I’m not sure why you have experienced the above problem…
Yes, the AMI was no longer available to me either.
I am now using this AMI:
ami-23b6534a
don;t think I had to make any other changes to the script. But let me know if you stilll have issues and I can get my script to you to have a look at.
Thanks helix, I will check that AMI out should the old one stop working for me too (seemed ok last night)
Ok, the script has now been updated to use the ami-23b6534a AMI and everything is working smoothly once again! Thanks again for your input!
First let me say – kudos, your script worked great! Worked right out the bat, minus some configuration misunderstandings on my part.
But well, hulu’s not giving up without a fight… when I try to connect to Hulu using the proxy I get “you are trying to use an anonymous proxy” from them and they won’t let me connect.
Any idea how I can bypass this?
Hi Guy
I’m glad that you managed to get the script up and working OK after some initial issues – please do let me know if you have any suggestions on how I could improve on the existing script, whether it is usability or functionality, thanks!
Unfortunately, Hulu is quite problematic. I don’t use the service myself but after a bit of searching around I came across the following post, where the conclusion in one of the comments was that “Hulu no longer accepts the Amazon IP address range as valid, so you can’t use EC2 to watch Hulu anymore.” That basically sums it up – I don’t think there is any way you will be able to use EC2 to watch Hulu, unless you or someone else can work out a clever workaround.
Sorry, doesn’t look like their is anything I can do to help…
Yeah, they are very quick to plug these “leaks” I wonder how much traffic is trying to get through this way?
At least Pandora works, and even saves some bandwidth
It’s amazing the Internet still has geolocked sites (for free content), I’m hoping this will all go away in a few years.
I’ve also done this on my own. Your approach seems way over complicated though. No need for apache and http proxy etc.
I just start a spot instance. And then run my ssh command and the SOCKS5 proxy is up and going. Simple. Of course, I’m on Ubuntu Linux so ssh is natively available. But if you have ssh or putty on windows it should be that easy too. You still have to set your browser/email/other app to use the proxy.
I have the EC2 perl utility by timkay so the simplest cmd line steps to start are proxy are:
(request a spot instance, I use Ubuntu ami, max price 5 cents, you must have a key setup in AWS control panel along with a default security group with port 22 open)
ec2rsi ami-2d4aa444 -p 0.05 -k MyKey
(check until instance is running)
ec2din
(it tells me the id and DNS name)
ssh -i MyKeyFile -fND 8080
That’s starts the SOCKS proxy. Nothing else needed. Now tell Firefox to use localhost:8080 as SOCKS5 proxy. You should also set the Firefox about:config option for network.proxy.socks_remote_dns to true so that DNS isn’t leaked locally. Only need to do that once though.
(terminate instance when done, ssh will terminate too)
ec2tin
It would be fairly easy to script above but since there’s only a couple commands to go and one to terminate I never bothered.
Sorry, some cmdline parts above were stripped by posting.
The ssh cmd needs the DNS name given by the ec2din output.
The ec2tin needs the id from the ec2din output.
I had them there but foolishly used angle brackets that were stripped out.
Chris
Thanks for posting up your own method – I must agree that it seems a more simple procedure; that is providing that you alone are going to be using the proxy and that all web applications on your computer support the SOCKS protocol.
My own script was designed with a few considerations in mind:
Might I suggest that in the future you provide more constructive criticism when leaving comments? “No need for apache and http proxy etc.”. Who exactly are you speaking for here? Yourself, or everybody else in the world who might have need for a proxy? Whilst your script might well suit your own needs, it certainly doesn’t mean that it is the perfect solution for everyone.