How to install and use s3cmd tool in CentOS to manage AWS S3 storage buckets
S3cmd is a free tool and client used for uploading, retrieving and managing data in Amazon S3 and other cloud storage service providers that use the S3 protocol, such as Google Cloud Storage. It is used by users who are familiar with command line programs. It is also ideal for batch scripts and automated backup to S3.
How do i install s3cmd ?
I will install the tool on a CentOS host using the yum package manager.
See example below:
[root@aodba tmp]# yum search s3cmd
Dependencies Resolved
======================================================================================================
Package Arch Version Repository Size
======================================================================================================
Installing:
s3cmd noarch 1.6.0-3.el6 epel 162 k
Installing for dependencies:
python-magic x86_64 5.04-21.el6 rhel-x86_64-server-6 28 k
Updating for dependencies:
file x86_64 5.04-21.el6 rhel-x86_64-server-6 47 k
file-libs x86_64 5.04-21.el6 rhel-x86_64-server-6 313 k
Transaction Summary
=====================================================================================================
Install 2 Package(s)
Upgrade 2 Package(s)
Total download size: 551 k
Downloading Packages:
(1/4): file-5.04-21.el6.x86_64.rpm | 47 kB 00:00
(2/4): file-libs-5.04-21.el6.x86_64.rpm | 313 kB 00:00
(3/4): python-magic-5.04-21.el6.x86_64.rpm | 28 kB 00:01
(4/4): s3cmd-1.6.0-3.el6.noarch.rpm | 162 kB 00:00
-------------------------------------------------------------------------------------------------------
Total 79 kB/s | 551 kB 00:06
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Installed:
s3cmd.noarch 0:1.6.0-3.el6
Dependency Installed:
python-magic.x86_64 0:5.04-21.el6
Dependency Updated:
file.x86_64 0:5.04-21.el6 file-libs.x86_64 0:5.04-21.el6
Complete!
- you will have to provide it with your AWS Access Key, Secret Key, default region and a password used for encryption.
[root@aodba tmp]# s3cmd --configure
Enter new values or accept defaults in brackets with Enter.
Refer to user manual for detailed description of all options.
Access key and Secret key are your identifiers for Amazon S3. Leave them empty for using the env variables.
Access Key: AAVVRRDDEEXXYY
Secret Key: gus8muG3QQVmKi5Sw1IcsUZcG
Default Region [US]: ap-southeast-2
Encryption password is used to protect your files from reading
by unauthorized persons while in transfer to S3
Encryption password: VerySecretPassword
Path to GPG program [/usr/bin/gpg]:
When using secure HTTPS protocol all communication with Amazon S3
servers is protected from 3rd party eavesdropping. This method is
slower than plain HTTP, and can only be proxied with Python 2.7 or newer
Use HTTPS protocol [Yes]:
New settings:
Access Key: AAVVRRDDEEXXYY
Secret Key: gus8muG3QQVmKi5Sw1IcsUZcG
Default Region: ap-southeast-2
Encryption password: VerySecretPassword
Path to GPG program: /usr/bin/gpg
Use HTTPS protocol: True
HTTP Proxy server name:
HTTP Proxy server port: 0
Test access with supplied credentials? [Y/n] Y
Please wait, attempting to list all buckets...
Success. Your access key and secret key worked fine :-)
Now verifying that encryption works...
Success. Encryption and decryption worked fine :-)
Save settings? [y/N] y
Configuration saved to '/root/.s3cfg'- a conf file will be created in the /root/.s3cfg location.
- make sure it can only be read by the root/owner user since it stored your keys and password in test format in the file.
[root@aodba~]# s3cmd --version
s3cmd version 1.6.0[root@aodba~]# s3cmd ls
2015-11-27 05:18 s3://testbkt
2016-01-30 01:03 s3://productionbkt