Cogito - Adding “I think” to “IAM”

Today we are open-sourcing Cogito, an abstraction of AWS IAM syntax. IAM (Identity and Access Management) is AWS’s module that dictates the ability of various users and resources to mutate other resources within the AWS ecosystem. Permissions are described through a JSON-structured policy document that lives within the AWS console.

In AWS accounts with many microservices like ours, IAM policies quickly become difficult to maintain. Ensuring a consistent system while balancing security checks with ease-of-use can lead to such a headache that users avoid dealing with it by tending toward allowing blanket open permissions. In addition, the structure of the JSON policies was difficult to remember and work with in larger tooling.

We wanted a tool that could abstract away some of this pain, as well as provide us with a starting point from which to move forward with better practice around our IAM policies. We wanted an intuitive way to describe policies without having to remember a complicated JSON structure, as well as the ability to check our policies into source control.

The solution

The solution we built for all of these problems was to design a new, intuitive syntax that we could maintain in our own repositories with a minimal learning curve. This was the basis of libcogito. libcogito is a small C library that allows translation between the AWS-specified JSON policy document syntax and Cogito’s own syntax. For example, with the built-in AmazonS3ReadOnlyAccess policy, you get:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:Get*",
        "s3:List*"
      ],
      "Resource": "*"
    }
  ]
}

In Cogito’s syntax, this becomes:

ALLOW s3:Get*, s3:List* ON *;

For this small policy you can already see a large reduction in size - for more expansive policies the benefits are even greater. Through Cogito you end up writing human-readable statements that are easier to maintain and understand. You can even validate the syntax with the library on your own system, as opposed to having to hit an AWS endpoint to validate.

Integration

Syntax highlighting

Once we built libcogito, we turned our attention to integrating it into our workflow. We did this in a couple of ways. The first was to build support into our editors for easier management. As it turns out, building .tmbundles is relatively straightforward, and so we built cogito.tmbundle for syntax highlighting.

Humidifier

We also integrated Cogito into humidifier, our open-source tool for managing CloudFormation resources. We’ve been successfully managing our CloudFormation resources for over a year now with the help of humidifier. However, in order to write managed policies with humidifier, you still ended up having to write the JSON and then dumping that into one of the properties. With Cogito, we were able to remedy this by building cogito-rb, a ruby gem that wraps libcogito and allows us to do things like:

Humidifier::IAM::ManagedPolicy.new(
  managed_policy_name: 'TestPolicy',
  policy_document: {
    'Version' => '2012-10-17',
    'Statement' =>
      JSON.parse(Cogito.to_json(File.read('TestPolicy.iam')))
  }
)

This code could then be checked in alongside a TestPolicy.iam file, allowing easy maintenance and readability. For more examples of how to use Cogito with Ruby, check out the cogito-rb repository and its associated documentation. This gem is released and available on rubygems.

CloudFormation

Finally, we integrated through CloudFormation syntax itself, as a custom resource. Custom resources enable stacks to hit external APIs when CloudFormation stacks are created or updated. In our case, this allowed us to build a lambda that uses cogito-py and an AWS linux compiled version of libcogito to perform the translation.

All of that is to say, we can write IAM syntax in our CloudFormation stacks. An example stack would look like:

{
  "Resources": {
    "TestPolicyCogitoResource": {
      "Type": "Custom::CogitoResource",
      "Version": "1.0",
      "Properties": {
        "ServiceToken": "arn:aws:lambda:us-east-1:000123456789:function:cogito-dev-cogito",
        "Policy": "ALLOW s3:Get*, s3:List* ON *;"
      }
    },
    "TestPolicy": {
      "Type": "AWS::IAM::ManagedPolicy",
      "Properties": {
        "PolicyDocument": {
          "Fn::GetAtt": [
            "TestPolicyCogitoResource",
            "PolicyDocument"
          ]
        }
      }
    }
  }
}

When this stack gets deployed, a new managed policy will be created with the Cogito-translated policy document containing:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:Get*",
        "s3:List*"
      ],
      "Resource": "*"
    }
  ]
}

An example of how to deploy this AWS Lambda and more extensive examples can be found in our cogito-resource repository. For more examples of how to use Cogito with python, check out the cogito-py repository and its associated documentation. This package is released and available on pypi.

Backup & restore

Using Cogito, we were able to build even more tools into our workflow. For example, the following snippets allow us to dump all of our existing IAM policies to a zip file, and then restore from that zip file whenever we want:

Wrapping up

Overall, we’ve had great success using Cogito within our workflow. The various integration points make it easy to understand and maintain, and it generally works well as means of solving our various pain points with AWS IAM syntax. If you also have headaches working with a large amount of policies in your system, feel free to install cogito in any of its various forms. When you do please share your experience, approach, and any feedback in a gist, on a blog, or in the comments.