Richard Crowley’s blog

Blueprints in the new AWS CloudFormation

Cross-posted from the DevStructure Blog

Just last week, Amazon Web Services rolled out the first production release of the Amazon Linux AMI and with it some powerful new features for CloudFormation.  There’s a lot there (like updating stacks and IAM integration) but we at DevStructure are most excited about Application Bootstrapping.

The new Amazon Linux AMI comes with the aws-cfn-bootstrap package, which can unpack tarballs, place configuration files, install packages, and restart services at provision time.  Sound familiar?  That’s right, this new section of the CloudFormation template language uses the blueprint(5) file format!

Amazon’s PDF whitepaper, Bootstrapping Applications via AWS CloudFormation, walks through building a complete stack template by hand.  To summarize, you’ll need to create an IAM resource and declare your EC2 instances like this:

"Resources": {
  "DemoInstance": {
    "Metadata": {
      "AWS::CloudFormation::Init": {
        "config": THIS IS WHERE THE BLUEPRINT GOES!
      }
    },
    "Properties": {
	    "ImageId": {"Fn::FindInMap": [
	      "AWSRegionArch2AMI",
	      {"Ref": "AWS::Region"},
	      {"Fn::FindInMap": [
	        "AWSInstanceType2Arch",
	        {"Ref": "InstanceType"},
	        "Arch"
	      ]}
	    ]},
      "InstanceType": {"Ref": "InstanceType"},
      "KeyName": {"Ref": "KeyName"},
      "SecurityGroups": [{"Ref": "DemoSecurityGroup"}],
      "UserData" : {"Fn::Base64" : {"Fn::Join" : ["", [
        "#!/bin/sh\n",
        "/opt/aws/bin/cfn-init",
        " -s '", {"Ref" : "AWS::StackName"}, "'",
        " -r 'DemoInstance'",
        " --region '", { "Ref" : "AWS::Region" }, "'",
        " --access-key '", {"Ref": "DemoKey"}, "'",
        " --secret-key '", {"Fn::GetAtt": ["DemoKey", "SecretAccessKey"]}, "'",
        "\n",
        "/opt/aws/bin/cfn-signal",
        " -e $?",
        " '", {"Ref" : "DemoWaitConditionHandle"}, "'",
        "\n"
      ]]}}
    },
    "Type": "AWS::EC2::Instance"
  }
}

The user-data calls cfn-init with the newly-generated IAM credentials to fetch and process the metadata, and cfn-signal to report success or failure via a WaitCondition.

Packages managed by Yum, Python’s easy_install, and RubyGems plus files and services all work natively within CloudFormation.  Source tarballs will work if you upload them someplace and provide the fully-qualified URL.

Going the other direction, the metadata from an existing CloudFormation template can be loaded into Blueprint by copying out the "config" document fragment and passing it on standard input to blueprint-create(1).

Our thanks to the Reto Kramer, Chris Whitaker, and Adam Thomas for making it even easier to deploy blueprints to Amazon EC2.

Today, we’re releasing Blueprint 3.1, which includes a number of fixes and improvements but most importantly, understands all of Amazon’s extensions to the blueprint(5) format, allowing seamless transition to and from AWS CloudFormation.  Get 3.1 from GitHub, DevStructure’s Debian archive, or from PyPI.