Former2 - AWS Infrastructure to CloudFormation

with a special guest: Ian McKay, Former2 Developer

While working on AWS services, I always wondered why there is no tool that I can use to

convert existing services/configurations to CloudFormation templates. It would be so practical to

have this as an option on any given AWS service console, so I asked AWS Awesome support

and our TAM several times, but unfortunately I always received the reply stating that this is a

great idea and that was the end of it.

When I finally found such a tool, my joy knew no bounds. Behold “Former2”.

I was so impressed by Ian’s solution that I contacted him by email and eventually grabbed him

for a chat about his project:

Tell me a little bit about yourself:

I am working in Sydney as the Cloud Lead at Kablamo, and also as an AWS Hero. I enjoy

speaking at meetups, co-hosting podcasts, and writing my thoughts about AWS on my blog at

onecloudplease.com.

What can you tell me about the project?

Former2 allows you to generate Infrastructure as Code outputs from your existing resources

within your AWS account (that's 124 services, mind you (-; ).

By making the relevant calls using the AWS JavaScript SDK, Former2 will scan across your

infrastructure and generate an output to your liking: CloudFormation, Terraform, CDK Cfn

Primitives/Terraform, and other languages such as Python, Java, C#, & Pulumi - TypeScript, and

finally, even as a draw.io diagram.

I worked previously on a couple of other projects, but really took this project to heart once I saw

its usability. Former2 is actually a fork of an old project used by AWS back in the day, about 10

years ago, so I took it, and continued with it as an open source project.

It can be used via a web interface as well as a command line tool, or you can even host your

own http server.

Former2 does not create any resources within your AWS account and will keep your AWS

credentials locally (in memory) without sending it to the cloud or somewhere else.

You can exclude services, sort outputted data or filter by searching for a specific application.

Former2 takes advantage of the AWS SDK for JavaScript in the browser. It makes Get/List calls

to determine the resources within your AWS account and maps those SDK responses to the

various template formats. Former2 derives its name from the previous CloudFormer tool.

Former2 has evolved to fully support all AWS Services. Both inexperienced and experienced

AWS developers can use Former2 to quickly construct Infrastructure as Code templates for their

companies or customers. You can also add an option to add a deletion policy to your CF

(delete/retain)

What does it cost?

Since it is an open source tool, it will not incur any charges beside the API calls to AWS.

Why open source?

Besides being an awesome concept, as an open source software, anyone can contribute to

Former2, meaning enhancements or bug fixes can be shared by the community. Former2 has

more than 30 pull requests and more than 100 issues raised, which has significantly contributed

to shaping the project. Many of these contributions are raising or fixing bugs that I otherwise

wouldn’t find due to the sheer scale of the AWS products and services. I thank everyone who

has contributed directly or indirectly to the project.

Building trust is also important. Handling AWS credentials is a high-risk activity, and Former2 is

designed to never send these credentials to an external server that isn’t an AWS API endpoint.

Being fully transparent about the code you are running is essential to show that you understand

the sensitivity of the data.

Well, I thank you Ian, for your time, your responsiveness on GitHub and for creating this

awesome solution!

--

Before we start, a couple of words about CloudFormation:

AWS CloudFormation is an Infrastructure as Code (IaC) service, which means you can use text

files (JSON or YAML) to define a set of resources to be deployed without worrying about

semantics of how they are deployed. Using AWS CloudFormation truly helps with consistency

and repeatability.

More information can be found at this location: https://aws.amazon.com/cloudformation/

So what is Former2, actually?

Former2 has a couple of usages, depending on how you look at it:

● It can be a replication tool that you can create or move resources from one environment

to another.

● It can be a backup tool, but not in its traditional sense.

While Former2 provides a current snapshot of your AWS infrastructure, when redeploying in

many services and configurations it may not be relevant the second time. Security groups, local

IPs, AMI IDs or any resources created by Stackset are randomly assigned, and as such, it is not

likely you will get the same ones when restoring. Former2 actually provides you with a list of

resources you had and how they interacted with other services/resources. This precious

information can be a godsend for those who lost access to their account, or if the account

configuration is destroyed by a rough actor, especially if you don’t really know what you had

inside your account.

So, to recap: Former2 provides you with a backup/list of your infrastructure-service connections

and configuration, but not a backup of your data (the content of the EC2 server, for example).

--

While debugging some issues I noticed on Former2, I worked with Ian and we managed to solve

those issues (different behavior on edge scenarios) by communicating via his GitHub account.

This is the power of open source.

While working on the Former2 CLI, I noticed that it takes the AWS user identity from the

~/.aws/credentials file. This means that if you have one user or a couple of local users, you can

use it without any issue.

But what if you are using assume role on the ~/.aws/config file? What if you are getting your

accounts from your organization? With this setup you are stuck, and this great adventure comes

to a screeching halt before you even had a chance to check out this solution.

Fret not, young grasshopper - we have the power of the code.

I looked into the Former2 source code (NodeJs) and found that you can actually use 3 different

options:

1. Using the default configuration in the ‘AWS_SHARED_CREDENTIALS_FILE’ environment

variable.

2. Add a custom profile using the ‘AWS_PROFILE’ environment variable.

3. Directly load the AWS.SharedIniFileCredentials provider.

var creds = new AWS.SharedIniFileCredentials({profile: 'myprofile'});
AWS.config.credentials = creds;

More details can be found at

https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/SharedIniFileCredentials.html

My use case was a bit more complicated: I had a couple hundred accounts that obviously

cannot and will not be configured with an access key and secret key, so I needed to get

temporary credentials for each account. Next thing was to understand where and how to store

them:

● Where was obvious - on the ~/.aws/credentials file.

● How needed more brainstorming. This is the main file, undoubtedly one of the most

important files, which I can’t use to store hundreds of temporary credentials. It will get real messy, real fast.

Let’s not forget that the default duration of IAM role session is 900 seconds (15 minutes), and