Tag Archives: aws

Get your EKS pod to assume an IAM role using IRSA

IRSA, or ‘IAM Roles for Service Accounts’ is a new AWS mechanism that lets pods running in your EKS cluster automatically assume an IAM role when using other AWS resources.

For example, we might have a DynamoDB with a read-write role associated, and we want a payment processing pod in our cluster to write to it. IRSA should make this pretty transparent.

First off, follow this tutorial. The rest of this post is basically an errata sheet for the missing information and steps to get it working.

Our approach

We’re going to tackle this in two sections:

  • Get the s3-echoer example app working in our cluster – this proves the cluster setup is correct
  • Amend our application to correctly use IRSA

Getting the cluster configuration right

If you’re using Terraform…

Two possible screw-ups here depending on which tutorial you’re following – I went with this one, which needs some tweaks:

OpenID Provider CA thumprints missing

Using Terraform to spin out your cluster? This bug will currently leave the OpenID Connect provider in an invalid state, because no certificate thumbprints are added. You’ll need to either hack some Bash into your build, or hard-code the CA thumb in your .tf file. The hacking method is fine if you’re only ever targeting one AWS region. Without this step, you will find that the s3-echoer portion of the original tutorial fails.

Wrong role assumption policy

The tutorial adds a policy to your service account role that limits who can assume the role to your aws-node pods running in your cluster. Since your code will be running in your own pods using your own service account, you need to use your own namespace and service account name instead. For example, if our deployment were under a service user called ‘payments-service-user’ running in the default namespace, we’d want:

data "aws_iam_policy_document" "example_assume_role_policy" {
  statement {
    actions = ["sts:AssumeRoleWithWebIdentity"]
    effect  = "Allow"

    condition {
      test     = "StringEquals"
      variable = "${replace(aws_iam_openid_connect_provider.example.url, "https://", "")}:sub"
      values   = ["system:serviceaccount:default:payments-service-user"]
    }

    principals {
      identifiers = ["${aws_iam_openid_connect_provider.example.arn}"]
      type        = "Federated"
    }
  }
}

You should now be at a point that you can get the s3-echoer to run.

You can check if your own cluster’s service account is setup OK by Bash’ing into any pod and running the env command. We need the following three environment variables set:

  • AWS_DEFAULT_REGION
  • AWS_ROLE_ARN
  • AWS_WEB_IDENTITY_TOKEN_FILE

Amending our application

While the SDKs just work with other AWS hosting models, you need to do some manual work to get EKS IAM role assumption to work. This isn’t consistent across SDK languages – the Go SDK seems to fall into the Just Works category, but others like the Java SDK need a tweak.

In Java, in particular, you need to add an instance of STSAssumeRoleWithWebIdentitySessionCredentialsProvider to a credentials chain, and pass that custom chain to your SDK init code via the withCredentials builder method.

This class doesn’t automatically come as part of the credentials chain. Nor does it automatically initialise itself from environment variables the same way other providers do.

You’ll have to pass in the web identity token file, region name and role ARN to get it running.