According to the Shared Responsibility Model AWS is responsible for “Management of the Cloud”, while a customer is responsible for Security and Operations in the Cloud + attention to costs as AWS provides virtually unlimited capacity on the “pay as you go” model.
There are native AWS tools like AWS Trusted Advisor, AWS Cost Explorer, and AWS Security Hub, which show you Insights about the efficiency of your cloud usage, but sometimes you can feel a lack of visibility, especially when you have dozens/hundreds of AWS accounts in your organization.
There are many tools on the market, which can help with aggregation, visualization, and analysis of Security (Palo Alto Prisma, Orca, Wiz) and Costs (Anodot, CloudHealth, etc.), they are interesting, but might not be affordable for all organizations.
I have accidentally found a pretty interesting AWS solution, which uses the abovementioned AWS services as data sources and Amazon QuickSight for visualization. It’s called Cloud Intelligence Dashboards Framework and looks like this in general:
The resource creation is fully automated using CloudFormtaion templates, but some prerequisites should be done manually, depending on the number of Dashboards you are going to deploy. In this post let’s quickly look at the following:
- CUDOS Dashboard
- Cost Intelligence Dashboard
- KPI and Modernization Dashboard
- Trusted Advisor Organizational (TAO) Dashboard
- Compute Optimizer Dashboard
Deployment prerequisites
It’s better to follow the official deployment procedure, but I will just mention what you need to do to get Dashboards and the main part of this article will be about visualization.
Enable Amazon Quicksight
First of all, choose a central AWS account, which will aggregate all data for visualization. In the solution’s terms, it’s a “Data Collection Account”, in my case, it’s just an Organization Management Account for simplicity.
After pressing the Sign up button you will be presented with 2 options, please ensure you select the Enterprise Edition during this step.
Amazon QuickSight Q feature has an additional monthly cost and is not needed for CID
Select Continue and you will be presented with an option to add Paginated Reports. This is not required to deploy these dashboards, so you can safely choose No, Maybe Later.
Amazon QuickSight Paginated Reports feature has an additional monthly cost and is not needed for CID
So we can see an initial QuickSight page:
Enable AWS Compute Optimizer
This is needed only if you are going to deploy the “Compute Optimizer Dashboard”.
Enable AWS Compute Optimizer
This is needed only if you are going to deploy the “Compute Optimizer Dashboard”.
Trusted Advisor Organizational View
This is needed only if you are going to deploy the “Trusted Advisor Organizational (TAO) Dashboard”.
Make sure all concerned accounts have a Business, On-Ramp, or Enterprise Support Plan.
If you have a Basic or Developer Support plan, you can use the Trusted Advisor console to access all checks in the Service Limits category and six checks in the Security category.
For this case, I choose the Business Support plan to get as many findings as possible.
Enable trusted access for your organization:
CUDOS Dashboard
It provides you with high-level details and operational insights with the ability to drill down to resource-level granularity. In the CUDOS dashboard, you can find auto-generated cost optimization recommendations and actionable insights that can be used by your FinOps practitioners, Product Owners, and Engineering teams out of the box. It allows you to quickly identify spikes and uncover uncertainties in your AWS usage by highlighting particular resources that can be optimized.
Deployment
The deployment procedure is pretty straightforward. There are 3 main:
- Deploy a bucket for aggregated CUR in the Data Collection Account
- Deploy CUR, bucket, and a replication policy in Source Accounts (can be one or many Sources).
- Deploy Cloud Intelligence Dashboards (CID) Stack in Data Collection Account
Results
And the most interesting part is what it shows:
- Executive: Billing Summary
- Executive: RI/SP Summary
- Executive: MoM Trends
- Compute
- Storage
- Amazon S3
- Databases
- Amazon DynamoDB
- Messaging and Streaming
- Data Transfer & Networking
- AI/ML
- Monitoring & Observability
- Analytics
- End User Computing
- GameTech & Media
- TAGsplorer
- OPTICS Explorer
A lot of things, but my purpose is not to show everything, just to pay attention to how powerful it is and how valuable it can be for a wide variety of organization
Different trends and cost changes:
Quite handy thing is to see how efficient your Saving Plans:
Storage statistics:
Data transfer statistics:
The monitoring part includes CloudWatch, CloudTrail, and AWS Config:
Cost Intelligence Dashboard
It is a customizable and accessible dashboard to help create the foundation of your own cost management and optimization (FinOps) tool. Executives, directors, and other individuals within the CFO’s line of business or who manage cloud financials for an organization will find the Cloud Intelligence Dashboard easy to use and relevant to their use cases. Little to no technical knowledge or understanding of AWS Services is required
Results
- Billing Summary
- Cost Summary
- Compute Summary
- Storage Summary
- RI/SP Summary
- Expiring RI/SP Tracker
- OPTICS Explorer
- MoM Pivot
KPI and Modernization Dashboard
Potential Savings- Graviton vs. AMD vs. Intel
EBS GP3 Potential savings
Compute Optimizer Dashboard
Helps your organization to visualize and trace right sizing recommendations from AWS Compute Optimizer. These recommendations will help you identify Cost savings opportunities for over-provisioned resources and also see the Operational risk from under-provisioned ones
My dashboard is not so representative due to the absence of data:
Let’s refer to the official demo dashboard:
Check how optimized EC2 instances are:
Recommended instance family changes:
EBS optimization:
Lambda optimization options:
Trusted Advisor Organizational (TAO) Dashboard
Provides you visibility for all cost optimization opportunities and auto-identified idle resources together with highlighted by AWS Trusted Advisor risks and flagged resources across Security, Reliability, and Performance pillars. TAO provides historical trends allowing you to track the results of optimizations
Statistics by status and by category:
Monitoring of IAM access keys rotation:
IAM password policy by account:
Unrestricted Security Groups:
Outdated Lambda runtimes:
Monthly savings opportunities:
Unassociated elastic IPs (they are always forgotten somewhere):
Service limits monitoring. In my experience, it is always met unexpectedly and prevents quick deployment, when you need to wait while AWS is approving limit increase:
Pricing
As usual, it is “pay as you go” and depends on the number of features you enable and the volume of data, generated by your accounts.
AWS says that it’s about 100-200$ per month.
There is also a free trial for 30 days for 4 users of QuickSight, so the first month’s overall cost may be less if a trial period is still available. So you are ready to try it without risks.
Conclusion
The Cloud Intelligence Dashboards is an open-source framework, lovingly cultivated and maintained by a group of customer-obsessed AWSers, that provides customers actionable insights and optimization opportunities at a scale of organization. Supported by the Well-Architected framework, the dashboards can be deployed by any customer using a CloudFormation template or a command-line tool in their environment. These dashboards help customers drive financial accountability, optimize cost, track usage goals, implement best practices for governance, and achieve operational excellence across all Well Architected pillars.
In this post, we quickly looked at what dashboards look like and how much valuable information they bring to your organization.
The solution is actively supported by AWS:
https://github.com/aws-samples/aws-cudos-framework-deployment
https://github.com/awslabs/cid-framework
and can compete on price with other well-known 3d party solutions
