Understanding and Reporting on EC2 EBS Volume/Prices you have in your AWS account

   Many times you get lost in craziness of building EC2 instances and new systems wheel working in an very agile IT environment.

    With the new era of cloud commuting and the open doors to "unlimited" resources on the fly and with limited effort in configuration it all. You get drag into the spending pit, this where you separate the boys from the real nerds :).  Having the capability to overlook the entire infrastructure is quite hard when your company does not want to buy tools(Cloud Management Tools) to do it for you.

      So in such cases you need to come-up with nerdy scripts and sweat a bit to build your own tools to monitor and manager the Cloud infrastructure.   Lucky for us Amazon AWS has a bunch of tools to enable administration but they are all complicated and you need to learn them(not ideal for somebody that already is overworked).

 So getting to the point in this tutorial we will see how we can use aws ec2 client to create EC2 Usage Reports

I assume you already have the aws ec2 client installed - in case you do not have it follow this tutorial on how to do it in 1 minute - How to install and configure the AWS client

Is important to understand the AWS EC2 EBS volume types, this way you will have a better look over what you have and what you really use.

This a map of EBS types and their coresponding prices in Asia Pacific area:

Amazon EBS Provisioned IOPS SSD (io1) volumes

• $0.138 per GB-month of provisioned storage
• $0.072 per provisioned IOPS-month

Amazon EBS Throughput Optimized HDD (st1) volumes

• $0.054 per GB-month of provisioned storage

Amazon EBS Cold HDD (sc1) volumes

• $0.03 per GB-month of provisioned storage

Amazon EBS Snapshots to Amazon S3

• $0.055 per GB-month of data stored

More on the rest of the Areas see link here - AWS EBS Prices 

How do i get charged by Amazon for my Volume ?

This is not very transparent and many get confused. We all know that Amazon will provide you EBS volumes on monthly rate base. eg: 1 Gb = 0.10 $ cents; But what if i i need 1 Tb of data but is only for a single day out of a month ? Do i pay for the full month ? The answer is no ! You will be charged for the factor of quantity and time(hourly based), so the final cost will be the weighted average of usage over the month. -i hope this makes sense. Example 1: Full month use of 100 Gb : 100 *  0.12 = 12 $ - this is for a full month usage. 100 Gb for a single day(24H) will pay: 12(total sum) / 30(days) = 0.40 $    Most used EBS volume type is GP2 type, so is around 10 cents depending on where you are located. This 10 cents can ad up to big amounts as you are getting charged for the full size of the volume and not for what ever you are using.

  Example 2:

You add a volume of 100 Gb but only have 10 Gb on it ! You are going to pay for the 100 Gb.(EBS volume are charged for allocation amount and not usage amount )

So let us go over some reports i normally use for my daily Cloud administration

Count how Many GP2 Volume types you have

[root@aodba]# aws ec2 describe-volumes --output text| grep gp2 | wc -l

53

Sum the total amount of GB you have as GP2 volume type

[root@aodba]# aws ec2 describe-volumes --output text | grep VOLUMES | grep gp2 | awk {'print $6'}| awk '{s+=$1} END {print s" Gb"}'

2981 Gb

Sum the total amount of GB and Calculate the spent $ value you have as GP2 volume type

• we will assume you are getting the rate of 0.12 $ dollars per Gb per month

[root@aodba]# aws ec2 describe-volumes --output text | grep VOLUMES | grep gp2 | awk {'print $6'}| awk '{s+=$1} END {print s * 0.12 " $/month"}'

357.72 $/month

Group EBS volume types and calculate their usage

• this is tricky script as you need to group and calculate all types of volume size.
• also you have volumes that might not have any snapshot on them so the output that you feed to awk must be processed differently .

I have put the script inside a executable file called grpvols.sh

[root@paodba]# cat grpvols.sh
aws ec2 describe-volumes --output text | grep VOLUMES | grep snap | grep io1  /tmp/io1

aws ec2 describe-volumes --output text | grep VOLUMES | grep snap | grep gp2 | awk '{print $6" "$9" "$10}'  /tmp/gp2 

aws ec2 describe-volumes --output text | grep VOLUMES | grep snap | grep standard  /tmp/standard 

aws ec2 describe-volumes --output text | grep VOLUMES | grep -v snap | grep io1 | awk '{print $6" "$8" "$9}'  /tmp/io1 

aws ec2 describe-volumes --output text | grep VOLUMES | grep -v snap | grep standard | awk '{print $5" "$7" "$8}'  /tmp/standard 

aws ec2 describe-volumes --output text | grep VOLUMES | grep -v snap | grep gp2 | awk '{print $6" "$8" "$9}'  /tmp/gp2

cat /tmp/io1  /tmp/final
cat /tmp/standard  /tmp/final
cat /tmp/gp2  /tmp/final
cat final | awk '{print $3 " "$1}' | awk '{
    arr[$1]+=$2
   }
   END {
     for (key in arr) printf("%-10s %s\n", key, arr[key]" Gb")
   }'   | sort +1n -1


-- Output of the Script Execution

[root@prodvert01 tmp]# ./grpvols.sh
gp2        2981 Gb
io1        10 Gb
standard   100 Gb

This will sum the size of your volumes based on the volume type. As you can see we have 2981 Gb of gp2 volume type and 10 Gb of io1 type and also 100 Gb of standard volume type. I hope this was useful and if any has an opinion please fell free to drop me a comment.