SprayflyLanguages, Linux, Life

Scenario
I am currently living in China and as many of you know, access to the web here is filtered, meaning sites such as YouTube, Flickr, Blogger, Wordpress are often unavailable. Whilst previously I overcame this problem by using TOR and FoxyProxy and setting up rules telling Firefox which URLs should be redirected through the proxy, TOR is not exactly speedy and so although this setup was free, it was not really suitable for smooth viewing of any blocked multimedia websites.

Solutions
I started to look around for other possible proxy solutions but most pay-for proxies are charged on a monthly basis and are fairly expensive. Furthermore, there is always the possibility that the authorities here are already aware of the IP ranges of some commercial proxy servers and so you run the risk of being blocked out yet again. What I really needed was a pay as you go proxy server where you would only be charged when you actually use the server and for the amount of bandwidth used. I then came accross a post on Duane Storey’s blog which described how he created an Amazon EC2 instance which could be used to make a proxy server available that was only to Iranian web users. Duane’s solution involves his own custom Fedora image which has Apache setup to run as a web proxy. This setup was pretty much exactly what I needed – except that I would need to make sure the Proxy server only allowed my own computer to connect rather than making it a public proxy server accessible to Iranians. After a bit of testing, I managed to get something going and was pleased to see that the proxy was very speedy indeed!

Cheap Proxy Server
Amazon EC2 Instances currently cost $0.10 USD per hour to run plus bandwidth charges (which for standard web browsing will be negligent). This gives Amazon EC2 the potential to be an incredibly cheap on-demand proxy server. Finally you only pay for what you use rather than paying an extortionate monthly rate.

Automation
The next thing to do was to automate the server setup process as entering all the necessary commands into a terminal can be cumbersome and time consuming. I used a unix shell script to accomplish the following steps:

1. Create an Amazon EC2 Fedora instance which is configured to run Apache as a proxy server.
2. Connect to Amazon EC2 instance via SSH and modify the Apache configuration file to only allow the client’s computer to connect to the server.
3. Restart Apache to apply changes.
4. (Optional) Assign an Amazon Elastic IP Address to the Instance which can be used to access the HTTP Proxy. The script prints the IP address assigned to the proxy server in the terminal – this IP address can then be pasted into your web browsers proxy configuration settings.
5. (Optional) Create a secure SSH HTTP Tunnel to the proxy server on port 9999. Clients can connect by changing browser network settings to a SOCKS 5 proxy on localhost:9999.
6. The script then waits for keyboard input “stop” upon which the SSH HTTP tunnel will be closed, the Amazon EC2 Instance shutdown and any Amazon Elastic IP addresses released.

The script is designed to work on Ubuntu (Jaunty 9.04) but should also work on other Linux Operating Systems and possibly on Mac OS X too. Before you can get it up and running there is some preliminary preparation which is necessary.

Step 1: Sign up for an account and setup AWS
Before you can use Amazon Web Services, you need to sign up for an account at http://aws.amazon.com/.

This will give you two keys that you will need for accessing your Amazon Web Services: your “access key” and your “secret access key”. Once complete, you then need to sign up for Amazon’s Elastic Compute Cloud (EC2) service. As part of signing up for EC2, you will create an X.509 certificate. You will need to save your private and public keys in ~/.ec2 on your local machine. All this is described in more detail in the AWS “Getting Started Guide” under Setting up an account.

Step 2: Download and install EC2 command line tools
The process of setting up the tools is described under Setting up the Tools in AWS “Getting Started Guide“. Just do what that page says and note down the location of the tools directory.

Step 3: Generate an SSH keypair, disable strict key checking
Follow the instructions here to generate an SSH keypair to be used in combination with your Amazon EC2 Instance. The RSA private key generated needs to be placed in your ~/.ssh folder and should be given 600 permissions.

chmod 600 your_rsa_key

Another important thing for the script to work is to set OpenSSH to disable Strict Host Key Checking for amazon domains only. This has only minor security implications and is vital if you want the whole server setup process to be automated.

In ~/.ssh/ create a file called config and put the following text inside:

Host *.amazonaws.com
StrictHostKeyChecking false

Give the file the right permissions

chmod 600 ~/.ssh/config

Step 4: Install Curl
The shell script uses a command line utility called curl to determine the external IP address of your computer – hence allowing you exclusively to connect to the proxy server. On Ubuntu, curl can be installed using apt-get.

sudo apt-get install curl

Step 6: Download the script and configure variables
First download the autoproxy archive to somewhere in your home folder and uncompress it. You then need to make the shell script executable

chmod u+x autoproxy.sh

Open the file in a text editor and make changes to the below variables to fit your own setup:

export EC2_HOME=/home/username/folder/ec2

This is the location of the EC2 command line tools which were installed during Step 2.

export EC2_PRIVATE_KEY=~/.ec2/pk-LSOEEFCT2IYW74BHFUIHHSBE63H4GM77.pem

This is the location of your Amazon EC2 Private Key file which you generated when creating your AWS account.

export EC2_CERT=~/.ec2/cert-LSOEEFCT2IYW74BHFUIHHSBE63H4GM77.pem

This is the location of your Amazon EC2 Certificate file which you generated when creating your AWS account.

export JAVA_HOME=/usr/lib/jvm/java-6-sun/jre

Location of Java on your computer.

If you don’t know where Java is installed on your computer you can follow symbolic links in a terminal to find it:

ls -l /usr/bin/java

/usr/bin/java -> /etc/alternatives/java

ls -l /etc/alternatives/java

/etc/alternatives/java -> /usr/lib/jvm/java-6-sun/jre/bin/java

export ssh_key=~/.ssh/id_rsa-gsg-keypair

Location of your Amazon EC2 SSH private RSA key (created in Step 3). This should be located in ~/.ssh and needs 600 permissions.

export autoproxy=/home/username/autoproxy

Location of the autoproxy folder that you downloaded.

Running the script
To run the script, open up a terminal and type the following:

cd /home/*username*/autoproxy/
./autoproxy.sh

Browser Config
If you chose to use an Amazon Elastic IP Address to access your proxy server, the script should output the address in the terminal once the proxy server is setup. In your browser you simply need to enter the proxy setup section and choose “HTTP Proxy” using the Amazon Elastic IP Address and port 80.

If you chose to use an SSH HTTP Tunnel to connect to access your proxy server, in your browser enter the proxy setup section and choose “SOCKS 5″ with the address as localhost on port 9999.

Disclaimer
Whilst using an Amazon EC2 Instance as a personal proxy server should not be a problem, it appears that Amazon do not approve of use of EC2 instances as public proxy servers available to the masses. For this reason please make sure you limit access to the proxy to your computer only as configured in the script. Use this script at your own risk. I cannot be held responsible for any undesired consequences that result from its use.

Anyway, have a try and see how you get on. Don’t hesitate to get in touch if you have any questions. I’ve spent quite a bit of time on this and I hope that it can help some other people out. I would appreciate any feedback or suggestions

Tags: amazon, automated, aws, bash, 网络, ec2, 脚本, instance, Linux, proxy, script, server, shell, terminal, Ubuntu, 代理

This entry was posted on Sunday, June 28th, 2009 at 8:04 am and is filed under Linux, Ubuntu. You can follow any comments to this entry through the RSS 2.0 feed. You can leave a comment, or trackback from your own site.