How to Clean up Amazon unused EBS Volumes and lower Costs

    Moving to Cloud is great and offers you huge flexibility, Startups became easy to initiate and we no longer need to invest huge amounts of money in something that might not work.

All this is great but tracking AWS costs is complicated, especially when you have several major business units, distributed teams, and dozens of projects.

In this tutorial we will focus on how we can track and remove unused volumes from our AWS infrastructure.

So what is the motivation for this ?     Organizing EBS volumes and deleting detached volumes on a regular basis to can help decrease AWS spend. See how easy it is to identify, tag, snapshot, detach, and delete those unused EBS volumes.

First we need to understand Volume state types:

  1. creating     - The volume is getting created and is not yet ready.
  2. available   -  The volume is available for use.
  3. in-use        - The volume is in use/attached.
  4. deleting     - The volume is getting deleted.
  5. error          -  The volume is in error state(pray that you have snapshot of it).
We are going to look for volume with the state of available since this volumes are ready to be used but are not used/attached to any of our instances. I assume you have already have installed you AWS EC2 client, if not you can follow this easy tutorial here - Install AWS EC2 client. Things to consider You have AWS infrastructure spread across multiple Regions.   Using the AWS console can be very time consuming and not very efficient when dealing with multiple Regions and multiple projects with 10 + maybe 100 + instances that carry 2-3 volumes and also have snapshots generated. not to mention when you have more than one administrator and your business is in full developing mode.

Now let us get to work 

 - i am going to walk you thru the process i use to manage my unused volumes/snapshots and how i clean them up across all regions.

List all available regions:

[root@aodba]# for machine in aws ec2 describe-regions --output text|awk {'print $3'}; do

 echo $machine
 done
ap-south-1
eu-west-1
ap-southeast-1
ap-southeast-2
eu-central-1
ap-northeast-2
ap-northeast-1
us-east-1
sa-east-1
us-west-1
us-west-2
  • this script will list all available regions in AWS.
  • we will use this output to loop thru all the regions and scatter for unused volumes.
List all volume in a region: 
[root@aodba]#for volumes in aws ec2 describe-volumes --region us-east-1 --output text| grep available | awk '{print $9}' | grep vol| tr '\n' ' '

  do 
    echo  "us-east-1  "$volumes
done
us-east-1  vol-e2fb984f
us-east-1  vol-84fd9e29
  • this script will loop thru the volumes in your region us-east-1 and find all volumes with status available.
Ok , so we have two script that are both looping to give us the volumes with status available, now we need to combine them and create a nested loop.   
[root@aodba]#for regions in aws ec2 describe-regions --output text|awk {'print $3'}

do
   for volumes in aws ec2 describe-volumes --region $regions --output text| grep available | awk '{print $9}' | grep vol| tr '\n' ' '

    do 
	echo $regions" "$volumes
	done
done

ap-southeast-2 vol-120039d9
ap-southeast-2 vol-100039db
ap-southeast-2 vol-4f033a84
ap-southeast-2 vol-4e033a85
us-east-1  vol-e2fb984f
us-east-1  vol-84fd9e29
  • you can see that now our script went and looped thru all regions and thru all volumes in each region and picked up all volumes with state available 
So what now ? This tutorial was about cleaning right ?      Using this script now we can implement an automated clean up task by adding a new functionality to this script.

Before you run/execute anything we will generate the command that will be executed and examine it:

  •  for the purpose of this tutorial and to keep it clean i have passed a specific region to my script in this case us-region-1.
  • i have also add ed the delete-volume command with the volume-id option
[root@aodba]# 
for regions in us-east-1
   do
for volumes in aws ec2 describe-volumes --region $regions --output text| grep available | awk '{print $9}' | grep vol| tr '\n' ' '

   do
 echo "aws ec2 delete-volume --region "$regions " --volume-id " $volumes
   done
done
--output

aws ec2 delete-volume --region us-east-1  --volume-id  vol-e2fb984f

aws ec2 delete-volume --region us-east-1  --volume-id  vol-84fd9e29
  • we can see that the command was generated using the echo command and the syntax seems ok.
Executing the delete-volume command:
  • now this is up to you, if you wanna run it using the echo option, copy+paste or let the script do all the work as i will demonstrate here.
[root@aodba]# 
for regions in us-east-1
 do
   for volumes in aws ec2 describe-volumes --region $regions --output text| grep available | awk '{print $9}' | grep vol| tr '\n' ' '

     do
 aws ec2 delete-volume --region $regions --volume-id $volumes

   done
 done
  • this will run the command that he generates dynamically as demonstrated in previous steps(Note the available state of the volume) .
Now we will try to list the volumes again and see if we have any volumes with state available/no in use. 
[root@aodba]# aws ec2 describe-volumes --region us-east-1 --output text| grep available | awk '{print $9}' | grep vol
  • we can see nothing is returned.

So the final Script to clear all volumes with state available across all regions is bellow

Use this script with care and test, test, test, test again and again on your DEV enviroment or some dummy environment. I will not be responsible for any damage that this script might cause. 
for regions in aws ec2 describe-regions --output text|awk {'print $3'}

do
   for volumes in aws ec2 describe-volumes --region $regions --output text| grep available | awk '{print $9}' | grep vol| tr '\n' ' '

    do 
	aws ec2 delete-volume --region $regions --volume-id $volumes

	done
done
hope this was useful...

Copyright 2020. All Rights Reserved