Cross-account monitoring with Amazon CloudWatch

Problem statement

Using multiple AWS accounts is one of guidelines to set up a well-architected environment. By using multiple accounts, you can best support your security goals and business processes. In previous posts we already covered multi-account logging, backups and security services integrations. In this post we will look at multi-account monitoring with Amazon CloudWatch, because it’s important to provide a support team with ability to see metrics and dashboards from different accounts and regions in one place, for example in the dedicated “monitoring” account. Moreover we need to figure out how to configure cross-account access for dozens and hundreds of accounts and automatically add a new account for monitoring right after enrollment. Cross-account management is a best practice in ASW and Automat-IT, the post is based on knowledge gained in many implementations for our customers.

Solution design

A high-level diagram of the proposed solution is below:

Every member account will have an IAM role for cross-account access that allows the “monitoring” account to assume it. Monitoring team has access only to CloudWatch in this account. The solution has to solve the following:

• Complete configuration on 50+ current accounts

• Every new account in the organization must automatically deploy the required IAM role.

• Needs to be managed via Git repository.

Solution implementation

The recommended way for deploying resources across different AWS accounts and regions is using CloudFormation StackSets. There is a solution called “Customizations for AWS Control Tower”, which give us exactly what we need:

• Configuration can be stored as code in Git repo.

• Deployment can be started by a commit or event of AWS account enrollment.

• Resources can be deployed across different accounts and regions within a Landing Zone.

The diagram below presents the architecture of the “Customizations for AWS Control Tower”:

We need to deploy the IAM role for cross-account access for Amazon CloudWatch. The following CloudFormation template will be used:

---
AWSTemplateFormatVersion: '2010-09-09'
Description: EnablesCloudWatchincentralmonitoringaccountstoassumepermissionstoviewCloudWatchdatainthecurrentaccount

Parameters:
  MonitoringAccountIds:
    Description: Allowsoneormoremonitoringaccountstoviewyourdata.EnterAWSaccountids,12numericdigitsincomma-separatedlist
    Type: CommaDelimitedList

  Policy:
    Description: ThelevelofaccesstogivetotheMonitoringaccounts
    Type: String
    Default: CloudWatch-and-AutomaticDashboards
    AllowedValues:
      - CloudWatch-and-AutomaticDashboards
      - CloudWatch-and-ServiceLens
      - CloudWatch-AutomaticDashboards-and-ServiceLens
      - CloudWatch-core-permissions
      - View-Access-for-all-services

Conditions:
  DoFullReadOnly: !Equals [ !RefPolicy, View-Access-for-all-services ]
  DoAutomaticDashboards: !Equals [ !RefPolicy, CloudWatch-and-AutomaticDashboards ]
  DoServiceLens: !Equals [ !RefPolicy, CloudWatch-and-ServiceLens ]
  DoServiceLensAndAutomaticDashboards: !Equals [ !RefPolicy, CloudWatch-AutomaticDashboards-and-ServiceLens ]
  DoCWReadOnly: !Equals [ !RefPolicy, CloudWatch-core-permissions ]

Resources:
  CWCrossAccountSharingRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: CloudWatch-CrossAccountSharingRole
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              AWS: !Split
                - ','
                - !Sub
                  - 'arn:aws:iam::${inner}:root'
                  - inner: !Join
                      - ':root,arn:aws:iam::'
                      - Ref: MonitoringAccountIds
            Action:
              - sts:AssumeRole
      Path: "/"
      ManagedPolicyArns: !If
        - DoFullReadOnly
        -
          - arn:aws:iam::aws:policy/CloudWatchReadOnlyAccess
          - arn:aws:iam::aws:policy/CloudWatchAutomaticDashboardsAccess
          - arn:aws:iam::aws:policy/job-function/ViewOnlyAccess
          - arn:aws:iam::aws:policy/AWSXrayReadOnlyAccess
        - !If
          - DoAutomaticDashboards
          -
            - arn:aws:iam::aws:policy/CloudWatchReadOnlyAccess
            - arn:aws:iam::aws:policy/CloudWatchAutomaticDashboardsAccess
          - !If
            - DoServiceLens
            -
              - arn:aws:iam::aws:policy/CloudWatchReadOnlyAccess
              - arn:aws:iam::aws:policy/AWSXrayReadOnlyAccess
            - !If
              - DoServiceLensAndAutomaticDashboards
              -
                - arn:aws:iam::aws:policy/CloudWatchReadOnlyAccess
                - arn:aws:iam::aws:policy/CloudWatchAutomaticDashboardsAccess
                - arn:aws:iam::aws:policy/AWSXrayReadOnlyAccess
              -
                - arn:aws:iam::aws:policy/CloudWatchReadOnlyAccess

This template provides us exactly the role which we can see in CloudWatch settings page, if we configure cross-account monitoring manually:

The simplest configuration in the git repository look like this:

The “templates” folder consists of the above mentioned CloudFormation template. The “manifest.yaml” file looks like this:

---
region: us-east-1
version: 2021-03-15

# Control Tower Custom Resources (Service Control Policies or CloudFormation)
resources:

# A role for Cloudwatch, needed for cross-account/cross-region access to CloudWatch metrics.
  - name: MonitoringRole
    resource_file: templates/monitoring-role.yaml
    deploy_method: stack_set
    parameters:
      - parameter_key: MonitoringAccountIds
        parameter_value: "1**********4"
      - parameter_key: Policy
        parameter_value: "View-Access-for-all-services"
    deployment_targets:
       organizational_units:
         - Prod
         - Dev
         - Stage

The CodePipeline looks as following:

• The first stage is checking out the configuration code.

• The second stage is a validation of syntax and security scan, which disallow overly permissive policies, security groups, disabled encryption for databases, etc.

• The third stage deploys service control policies if you provide them.

• The last stage deploys CloudFormation templates via CloudFormation StackSets.

Once the pipeline is deployed, we can see all our AWS accounts as stack instances: