What is GitOps
GitOps is an operational framework that uses DevOps best practices for application development, such as version control, collaboration, compliance, and CI/CD, and applies them to infrastructure automation.
In the GitOps approach, we use a Git repository as the single source of truth for infrastructure definitions. Git is an open-source version control system that tracks code management changes, and a Git repository is a .git folder in a project that tracks all changes made to files in a project over time. Infrastructure as code (IaC) is the practice of keeping all infrastructure configurations stored as code. The actual desired state (e.g., number of replicas or pods) may or may not be stored as code (e.g., number of replicas).
AWS introduced a new feature for CloudFormation called Git sync at the end of 2023. This feature enables customers to synchronize their stacks from a CloudFormation template stored in a remote Git repository.
Link AWS to GitHub repository
We can see the following message at the start:
To provision resources with your Git provider, AWS uses CodeStar Connections APIs
Let’s try CodeStart to connect our GitHub repository:
In the end we can see the following:
On July 31, 2024, AWS will end support for AWS CodeStart projects. Only existing CodeStart users can currently create projects. If you’re a new CodeStart user, consider Amazon CodeCatalyst
Let’s try Amazon CodeCatalyst. First of all, we need to link our AWS Account
Make sure your new CodeCatalyst space was verified:
Create a new project:
Link your repository
Authorize Amazon CodeCatalyst in your GitHub:
You can grant permission only to the specific Repository:
The GitHub repository has been linked:
Using AWS CloudFormation Git sync
Now, you can select your Git repository in the CloudFormation console. You can set a target branch as well:
A stack deployment file is a JSON or YAML standard-formatted file that contains parameters and values for managing your CloudFormation stack. It is monitored for changes. When changes to the file are committed to the repository, the associated stack is automatically updated.
The stack deployment file contains a key-value pair and two dictionaries:
- template-file-path
This is the full repository path for the CloudFormation template file. The template file declares the resources for the CloudFormation stack associated with this deployment file. - parameters
The parameters dictionary contains key-value pairs that configure the resources in the stack. A stack deployment file can have up to 50 parameters. - tags
The tags dictionary contains optional key-value pairs that you can use to identify and categorize resources in the stack. A stack deployment file can have up to 50 tags.
Here is my deployment-file.yaml
template-file-path: EC2InstanceWithSecurityGroupSample.yaml
parameters:
SubnetId: subnet-0f07863381aaf5c4b
VpcId: vpc-091433f2ce1cdccd2
tags:
cost-center: '123456'
org: 'AWS'
In this demo, we will create a simple EC2 instance, so subnet and VPC IDs are provided as input parameters (EC2InstanceWithSecurityGroupSample.yaml):
AWSTemplateFormatVersion: '2010-09-09'
Metadata:
License: Apache-2.0
Description: 'AWS CloudFormation Sample Template'
Parameters:
InstanceType:
Description: WebServer EC2 instance type
Type: String
Default: t3.small
AllowedValues: [t3.nano, t3.micro, t3.small, t3.medium, t3.large, t3.xlarge, t3.2xlarge,
m4.large, m4.xlarge, m4.2xlarge, m4.4xlarge, m4.10xlarge,
m5.large]
ConstraintDescription: must be a valid EC2 instance type.
SSHLocation:
Description: The IP address range that can be used to SSH to the EC2 instances
Type: String
MinLength: 9
MaxLength: 18
Default: 0.0.0.0/0
AllowedPattern: (\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})/(\d{1,2})
ConstraintDescription: must be a valid IP CIDR range of the form x.x.x.x/x.
LatestAmiId:
Type: 'AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>'
Default: '/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2'
SubnetId:
Type: String
VpcId:
Type: String
Resources:
EC2Instance:
Type: AWS::EC2::Instance
Properties:
InstanceType: !Ref 'InstanceType'
SecurityGroupIds: [ !GetAtt InstanceSecurityGroup.GroupId ]
ImageId: !Ref 'LatestAmiId'
SubnetId: !Ref 'SubnetId'
InstanceSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: Enable SSH access via port 22
VpcId: !Ref 'VpcId'
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 22
ToPort: 22
CidrIp: !Ref 'SSHLocation'
The template file is set in the dedicated line. We also need a specific IAM role:
The following permissions are required for the role:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "SyncToCloudFormation",
"Effect": "Allow",
"Action": [
"cloudformation:CreateChangeSet",
"cloudformation:DeleteChangeSet",
"cloudformation:DescribeChangeSet",
"cloudformation:DescribeStackEvents",
"cloudformation:DescribeStacks",
"cloudformation:ExecuteChangeSet",
"cloudformation:GetTemplate",
"cloudformation:ListChangeSets",
"cloudformation:ListStacks",
"cloudformation:ValidateTemplate"
],
"Resource": "*"
},
{
"Sid": "PolicyForManagedRules",
"Effect": "Allow",
"Action": [
"events:PutRule",
"events:PutTargets"
],
"Resource": "*",
"Condition": {
"StringEquals": {
"events:ManagedBy": [
"cloudformation.sync.codeconnections.amazonaws.com"
]
}
}
},
{
"Sid": "PolicyForDescribingRule",
"Effect": "Allow",
"Action": "events:DescribeRule",
"Resource": "*"
}
]
}
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "CfnGitSyncTrustPolicy",
"Effect": "Allow",
"Principal": {
"Service": "cloudformation.sync.codeconnections.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
Now we can proceed with the CloudFormation stack:
CloudFormation creates a new pull-request to complete the integration:
A pull request is automatically created in your Git repository when you choose Submit. You must merge this pull request into your Git repository to complete the process. When you merge the pull request, the stack is created and Git sync monitors the CloudFormation template and stack deployment file for changes to update the stack.
Once the sync is completed, we can see the EC2 instance is created:
The template itself is shown in the CloudFormation console:
EC2 instance is running:
Infrastructure code changes
Let’s add a simple change to the CloudFormation stack. Add a new port into the Inbound rule of the Security Group:
Create a pull request and merge it:
Stack update is executed automatically:
We can see that new code appeared in the CloudFormation console:
Stack update completed:
A new rule appeared in the Security Group:
We can also check the details of the CloudFormation Change Set:
ChangeSet status returned to the GitHub:
Add breaking changes
In the new pull request, I add a wrong thing intentionally (such AMI ID does not exist):
ChangeSet is created after the commit:
Execution failed
Our EC2 instance is still up and running because CloudFormation did not touch it. So we can revert the breaking code changes:
New sync is successful:
Conclusion
With Git sync, you can manage your CloudFormation stacks with source control by configuring AWS CloudFormation to monitor a Git repository. When you commit changes to the template or the deployment file, CloudFormation automatically updates the stack. This way, you can use pull requests and version tracking to configure, deploy, and update your CloudFormation stacks from a centralized location.
