Kinh nghiệm sử dụng s3cmd của Amazon SDK

I came across this interesting utility to sync backup files to Amazon S3. With this utility in less than ten minutes I learned how easy it is to upload a local file to remote folder in S3 bucket.

I already had a script in place to backup the site's files and database dump. To make this script complete I needed a script/ utility to sync backup files. And s3cmd came there as a savior. I would like to share in this blog post how easy it was to install, configure and get started to make the potential use of the same. 

Our servers are running Ubuntu 12.04.1 LTS. Luckily this s3cmd package is available in Ubuntu's official repository. In no time, I got this installed in our machine with apt-get command.

The following command did the trick of installing,

 $ sudo apt-get install s3cmd

This command would prompt to install a dependency package python-support. Once the installation is done it can be validated with the below command,

 $ s3cmd

The output returned might be as below,

1 ERROR: /home/user/.s3cfg: No such file or directory
2 ERROR: Configuration file not available.
3 ERROR: Consider using --configure parameter to create one.

Essentially it looks for the presence of config file in home folder .s3cfg. In our case this is not in place, since this is the first time the utility is being used.

As suggested we need to create one by issuing the same command again with --configure argument.

1 $ s3cmd --configure

This will ask for a series of questions. Essentially Access Key, Secret Key and other optional settings like proxy, choice of protocol (http vs https), etc. 

When all the needful inputs are given, the utility smartly prompts to make a test before saving it to configuration file.

 1 New settings:
 2   Access Key: [your access key]
 3   Secret Key: [your securet key]
 4   Encryption password: somepassword
 5   Path to GPG program: /usr/bin/gpg
 6   Use HTTPS protocol: False
 7   HTTP Proxy server name: 
 8   HTTP Proxy server port: 0
 9 
10 Test access with supplied credentials? [Y/n] Y
11 Please wait...
12 Success. Your access key and secret key worked fine :-)
13 
14 Now verifying that encryption works...
15 Success. Encryption and decryption worked fine :-)
16 
17 Save settings? [y/N] y

I found this tiny feature very handy. I noticed that the Security key I had entered in the first attempt was wrong

Finally to achieve the intended goal, that is to sync the needed file to S3, I used the following command,

 $ s3cmd --delete-removed --reduced-redundancy sync my_backup.tar.gz s3://backup.example.com/project_backup/

Interactive output of the command is below,

1 my_backup.tar.gz -> s3://backup.example.com/project_backup//kst_backup/kst_.tar.gz  [1 of 1]
2  63935864 of 63935864   100% in  186s   335.43 kB/s  done
3 Done. Uploaded 63935864 bytes in 186.1 seconds, 335.42 kB/s