How to install and use s3cmd tool in CentOS to manage AWS S3 storage buckets

   S3cmd is a free tool and client used for uploading, retrieving and managing data in Amazon S3 and other cloud storage service providers that use the S3 protocol, such as Google Cloud Storage. It is used by users who are familiar with command line programs. It is also ideal for batch scripts and automated backup to S3.  How do i install s3cmd ?  I will install the tool on a CentOS host using the yum package manager. See example below:

[root@aodba tmp]# yum search s3cmd
Dependencies Resolved

======================================================================================================
 Package                                     Arch                                  Version                                      Repository                                           Size
======================================================================================================
Installing:
 s3cmd                                       noarch                                1.6.0-3.el6                                  epel                                                162 k
Installing for dependencies:
 python-magic                                x86_64                                5.04-21.el6                                  rhel-x86_64-server-6                                 28 k
Updating for dependencies:
 file                                        x86_64                                5.04-21.el6                                  rhel-x86_64-server-6                                 47 k
 file-libs                                   x86_64                                5.04-21.el6                                  rhel-x86_64-server-6                                313 k

Transaction Summary
=====================================================================================================
Install       2 Package(s)
Upgrade       2 Package(s)

Total download size: 551 k
Downloading Packages:
(1/4): file-5.04-21.el6.x86_64.rpm                                                                                                                                 |  47 kB     00:00
(2/4): file-libs-5.04-21.el6.x86_64.rpm                                                                                                                            | 313 kB     00:00
(3/4): python-magic-5.04-21.el6.x86_64.rpm                                                                                                                         |  28 kB     00:01
(4/4): s3cmd-1.6.0-3.el6.noarch.rpm                                                                                                                                | 162 kB     00:00
-------------------------------------------------------------------------------------------------------

Total                                                                                                                                                      79 kB/s | 551 kB     00:06
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Installed:
  s3cmd.noarch 0:1.6.0-3.el6

Dependency Installed:
  python-magic.x86_64 0:5.04-21.el6

Dependency Updated:
  file.x86_64 0:5.04-21.el6                                                                 file-libs.x86_64 0:5.04-21.el6

Complete!
To finalize the s3cmd tool installation you need to create the config file or the .s3cfg file. See example below on how to configure the conf file.
  • you will have to provide it with your AWS Access Key, Secret Key, default region and a password used for encryption.
[root@aodba tmp]# s3cmd --configure

Enter new values or accept defaults in brackets with Enter.
Refer to user manual for detailed description of all options.

Access key and Secret key are your identifiers for Amazon S3. Leave them empty for using the env variables.
Access Key: AAVVRRDDEEXXYY
Secret Key: gus8muG3QQVmKi5Sw1IcsUZcG
Default Region [US]: ap-southeast-2

Encryption password is used to protect your files from reading
by unauthorized persons while in transfer to S3
Encryption password: VerySecretPassword
Path to GPG program [/usr/bin/gpg]:

When using secure HTTPS protocol all communication with Amazon S3
servers is protected from 3rd party eavesdropping. This method is
slower than plain HTTP, and can only be proxied with Python 2.7 or newer
Use HTTPS protocol [Yes]:

New settings:
  Access Key: AAVVRRDDEEXXYY
  Secret Key: gus8muG3QQVmKi5Sw1IcsUZcG
  Default Region: ap-southeast-2
  Encryption password: VerySecretPassword
  Path to GPG program: /usr/bin/gpg
  Use HTTPS protocol: True
  HTTP Proxy server name:
  HTTP Proxy server port: 0

Test access with supplied credentials? [Y/n] Y
Please wait, attempting to list all buckets...
Success. Your access key and secret key worked fine :-)

Now verifying that encryption works...
Success. Encryption and decryption worked fine :-)

Save settings? [y/N] y
Configuration saved to '/root/.s3cfg'
  • a conf file will be created in the /root/.s3cfg location.
Note:
  • make sure it can only be read by the root/owner user since it stored your keys and password in test format in the file.
Test that your s3cmd tool was installed correctly. Check the installed version:
[root@aodba~]# s3cmd --version

s3cmd version 1.6.0
List all available S3 buckets under your account:
[root@aodba~]# s3cmd ls
2015-11-27 05:18  s3://testbkt
2016-01-30 01:03  s3://productionbkt
In the next article we will see how we can use the s3cmd tool to manage our AWS S3 storage buckets. Working with S3CMD tool on AWS S3 storage buckets