Home » Tech Tips » AWS CloudFormation Using Command Line Interface (CLI)

AWS CloudFormation Using Command Line Interface (CLI)

by

Despite my ambivalent feeling about CloudFormation I use it a lot, but managing stacks through the Console is a pain. Fortunately, this service enjoys the same CLI support most other ones do, so it is just a matter of scripting to make it more developer-friendly.

There are several projects that provide a better experience from the terminal, but for simple needs I prefer simple tools. Here you can find a couple of what I use regularly for various common tasks.

What Is the CloudFormation Command Line Interface?

The CloudFormation Command Line Interface (CLI) is an open-source tool that enables you to develop and test AWS and third-party resources, and register them for use in AWS CloudFormation. The CloudFormation CLI provides a consistent way to model and provision both AWS and third-party resources through CloudFormation.

Note: In all the examples I use $STACK_NAME variable as the current stack name. You can set it by defining it as follows:

export STACK_NAME=stack1

Regions

Since CloudFormation is region-specific, each region provides an isolated environment. To change the region, either specify it for the aws command:

aws --region us-west-1 cloudformation ...

or define the AWS_DEFAULT_REGION environment variable:

export AWS_DEFAULT_REGION="us-west-1"

Deploy

The most used use-case is to deploy a template file to a new stack or update an existing one. You can use the deploy command for these tasks:

aws cloudformation deploy \
	--template-file cloudformation.yml \
	--capabilities CAPABILITY_IAM \
	--stack-name $STACK_NAME

 

It needs these parameters at the minimum:

  • template-file
  • stack-name

In this case, I defined cloudformation.yml as the template and $STACK_NAME as the name of the new stack.

Capabilities

You may notice the --capabilities argument of the above command. This is used to explicitly acknowledge that the stack creates IAM resources. There are 2 possible values: CAPABILITY_IAM and CAPABILITY_NAMED_IAM. The latter is required when you define a name for an IAM resource, for example, the RoleName:

LambdaExecutionRole:
  Type: AWS::IAM::Role
  Properties:
    RoleName: my_role

Parameters

When a stack requires parameters, you can supply them like this:

--parameter-overrides "PARAMETER1=value1" "PARAMETER2=value2"

Changesets

Under the hood, the deploy command does 2 things: it creates a changeset that describes what to do, then executes it. To skip the second one, use this flag:

--no-execute-changeset

Instead of a full deploy cycle, it stops when the changeset is created:

Waiting for changeset to be created..
Changeset created successfully. Run the following command to review changes:
aws cloudformation describe-change-set --change-set-name <arn>

After the changeset is ready, view it with the command:

aws cloudformation describe-change-set \
	--change-set-name <arn>

It gives back a lot of information, and usually, it’s enough to see what resources are changed. To filter for this, define a query:

aws cloudformation describe-change-set \
	--change-set-name <arn> \
	--query "Changes[].ResourceChange"

It gives back a smaller JSON describing how each resource will be affected when the changeset is executed:

[
    {
        "Action": "Remove",
        "LogicalResourceId": "LambdaApiGatewayInvoke",
        "PhysicalResourceId": "stack1-LambdaApiGatewayInvoke-1H6XSIBWPT9MB",
        "ResourceType": "AWS::Lambda::Permission",
        "Scope": [],
        "Details": []
    },
    {
        "Action": "Add",
        "LogicalResourceId": "LambdaApiGatewayInvoke2",
        "ResourceType": "AWS::Lambda::Permission",
        "Scope": [],
        "Details": []
    }
]

Review changes before execution

It is a good practice to review what will be changed before actually executing them, especially when updating a critical stack. Let’s construct such script!

Related:  AWS Services Every Developer Should Know About

It needs to perform 4 steps:

  • Create a new changeset (deploy --no-execute-changeset)
  • Get the changeset Arn (list-change-sets)
  • Describe the changeset (describe-change-set)
  • Cleanup (delete-change-set)

The 2 missing steps are (2) and (4).

To get the last changeset’s Arn for a given stack:

aws cloudformation list-change-sets \
	--stack-name $STACK_NAME | \
	jq -r '.Summaries |
		sort_by(.CreationTime) |
		.[-1].ChangeSetId
	'

And to delete a changeset:

aws cloudformation delete-change-set \
	--change-set-name $CHANGESET

With the above building blocks, this rather complex script can check what a template would change:

CHANGESET=$( \
		aws cloudformation deploy \
			--template-file cloudformation.yml \
			--capabilities CAPABILITY_IAM \
			--stack-name $STACK_NAME \
			--no-execute-changeset > /dev/null && \
		aws cloudformation list-change-sets \
			--stack-name $STACK_NAME | \
		jq -r '.Summaries |
			sort_by(.CreationTime) |
			.[-1].ChangeSetId') && \
aws cloudformation describe-change-set \
	--change-set-name $CHANGESET \
	--query "Changes[].ResourceChange" && \
aws cloudformation delete-change-set \
	--change-set-name $CHANGESET

Events

To get the events related to a given stack, use this command:

aws cloudformation describe-stack-events \
	--stack-name $STACK_NAME

It gives back a large JSON, detailing what happened to each resource:

{
	"StackEvents": [
		"StackId": "arn:aws:cloudformation:us-west-1:757253985935:stack/stack1/22059b00-6a88-11e9-b3cd-50d5caff88fd",
		"EventId": "3b11a760-6a88-11e9-ba0c-02e6681d586c",
		"StackName": "stack1",
		"LogicalResourceId": "stack1",
		"PhysicalResourceId": "arn:aws:cloudformation:us-west-1:757253985935:stack/stack1/22059b00-6a88-11e9-b3cd-50d5caff88fd",
		"ResourceType": "AWS::CloudFormation::Stack",
		"Timestamp": "2019-04-29T14:08:15.025Z",
		"ResourceStatus": "CREATE_COMPLETE"
	},
	{
		"StackId": "arn:aws:cloudformation:us-west-1:757253985935:stack/stack1/22059b00-6a88-11e9-b3cd-50d5caff88fd",
		"EventId": "LambdaApiGatewayInvoke-CREATE_COMPLETE-2019-04-29T14:08:12.548Z",
		"StackName": "stack1",
		"LogicalResourceId": "LambdaApiGatewayInvoke",
		"PhysicalResourceId": "stack1-LambdaApiGatewayInvoke-1H6XSIBWPT9MB",
		"ResourceType": "AWS::Lambda::Permission",
		"Timestamp": "2019-04-29T14:08:12.548Z",
		"ResourceStatus": "CREATE_COMPLETE",
		"ResourceProperties": "{\"FunctionName\":\"arn:aws:lambda:us-west-1:757253985935:function:stack1-LambdaFunction-FLV1VGO5J08S\",\"Action\":\"lambda:InvokeFunction\",\"SourceArn\":\"arn:aws:execute-api:us-west-1:757253985935:v2510wqve1/*/*/*\",\"Principal\":\"apigateway.amazonaws.com\"}"
	},
...
}

This contains all the informations, but it is hard to get an overview what happened to what. To extract the valuable data and format it to a columnar format, use this snippet:

aws cloudformation describe-stack-events \
	--stack-name $STACK_NAME | \
jq -r '.StackEvents[] |
	"\(.Timestamp | sub("\\.[0-9]+Z$"; "Z") | fromdate | strftime("%H:%M:%S") ) \(.LogicalResourceId) \(.ResourceType) \(.ResourceStatus)"
' | column -t

This shows a table with less information but that is easier to get a glimpse of the events:

14:08:15  stack1                  AWS::CloudFormation::Stack   CREATE_COMPLETE
14:08:12  LambdaApiGatewayInvoke  AWS::Lambda::Permission      CREATE_COMPLETE
14:08:07  ApiGatewayDeployment    AWS::ApiGateway::Deployment  CREATE_COMPLETE
14:08:07  ApiGatewayDeployment    AWS::ApiGateway::Deployment  CREATE_IN_PROGRESS

Note: timezone is not supported by jq at the moment, so it shows the time in UTC.

Related:  AWS - IAM Role Use Cases

Watch events

To see how things unfold near real-time, use watch:

watch "aws cloudformation \
	describe-stack-events \
		--stack-name $STACK_NAME | \
	jq -r '.StackEvents[] |
		\"\\(.Timestamp | sub(\"\\\\.[0-9]+Z$\"; \"Z\") | fromdate | strftime(\"%H:%M:%S\") ) \\(.LogicalResourceId) \\(.ResourceType) \\(.ResourceStatus)\"
	' | column -t"

Deploy and watch the events

When I deploy something using the AWS Console, I just press refresh every other second to see how the resources are appearing at my behest. The beauty of the CLI is that I don’t need to manually click.

To deploy a stack and get the events refreshed automatically, use:

(aws cloudformation deploy \
	--template-file cloudformation.yml \
	--capabilities CAPABILITY_IAM \
	--stack-name $STACK_NAME > /dev/null & \
) && watch "aws cloudformation describe-stack-events \
	--stack-name $STACK_NAME | \
		jq -r '.StackEvents[] |
			\"\\(.Timestamp | sub(\"\\\\.[0-9]+Z$\"; \"Z\") | fromdate | strftime(\"%H:%M:%S\") ) \\(.LogicalResourceId) \\(.ResourceType) \\(.ResourceStatus)\"
		' | column -t"

List

To get a list of the stacks in the current region, use:

aws cloudformation list-stacks | jq -r '.StackSummaries[] |
	select(.StackStatus != "DELETE_COMPLETE") |
	"\(.StackName) \(.LastUpdatedTime) \(.StackStatus)"
' | column -t

Deleted stacks are shown for some time so we need to filter them out.

stack2  2019-04-29T14:32:00.614Z  CREATE_COMPLETE
stack1  2019-04-29T14:07:39.159Z  CREATE_COMPLETE

As usual, this can be watched, which can be useful when you deploy/delete multiple stacks:

watch "aws cloudformation list-stacks | \
	jq -r '.StackSummaries[] |
		select(.StackStatus != \"DELETE_COMPLETE\") | \"\(.StackName) \(.LastUpdatedTime) \(.StackStatus)\"
	' | column -t"

Delete

Stack deletion is easy with this script:

aws cloudformation delete-stack --stack-name $STACK_NAME

 

Related:  Top 10 Ansible Modules for Linux System Automation

List resources

To list the resources managed in a stack, use:

aws cloudformation list-stack-resources \
	--stack-name $STACK_NAME | \
	jq -r '.StackResourceSummaries[] |
		"\(.LogicalResourceId) \(.ResourceType) \(.ResourceStatus)"
	' | column -t

This prints in a columnar format, extracted from the JSON the AWS CLI returns:

ApiGateway              AWS::ApiGateway::RestApi     CREATE_COMPLETE
ApiGatewayDeployment    AWS::ApiGateway::Deployment  CREATE_COMPLETE
ApiGatewayMethod        AWS::ApiGateway::Method      CREATE_COMPLETE
LambdaApiGatewayInvoke  AWS::Lambda::Permission      CREATE_COMPLETE
LambdaExecutionRole     AWS::IAM::Role               CREATE_COMPLETE
LambdaFunction          AWS::Lambda::Function        CREATE_COMPLETE
ProxyResource           AWS::ApiGateway::Resource    CREATE_COMPLETE

As usual, it can be watched:

watch "aws cloudformation list-stack-resources \
	--stack-name $STACK_NAME | \
	jq -r '.StackResourceSummaries[] |
		\"\(.LogicalResourceId) \(.ResourceType) \(.ResourceStatus)\"
	' | column -t"

Deploy and watch resources

To watch the resources during a deploy, you can combine the above script with the deployment:

(aws cloudformation deploy \
	--template-file cloudformation.yml \
	--capabilities CAPABILITY_IAM \
	--stack-name $STACK_NAME > /dev/null & \
) && watch "aws cloudformation list-stack-resources \
	--stack-name $STACK_NAME | \
	jq -r '.StackResourceSummaries[] |
		\"\(.LogicalResourceId) \(.ResourceType) \(.ResourceStatus)\"
	' | column -t"

Outputs

When you want to use the outputs of a stack, the AWS CLI returns them as an array of objects. To extract an output based on its name, use jq’s select:

aws cloudformation describe-stacks \
	--stack-name $STACK_NAME | \
	jq -r '.Stacks[0].Outputs[] |
		select(.OutputKey == "URL") |
		.OutputValue
	'

 

Related Posts

Leave a Comment