Description

When creating an EKS cluster with Bottlerocket AMI nodes using the EKS module and managed node groups, I observed that two launch templates are being created. Although the EKS node group correctly references the template with the tag eks:cluster-name set to swi-dev-3-bottlerocket, the auto scaling group ends up referencing a launch template with eks:nodegroup-name as fyn-nodegroup-20250407060414008800000003. As a result, the EKS nodes fail to join the cluster and do not appear in the EKS nodes section.

I am including the Terraform variables and the relevant plan output for context.

Terraform Variables

eks_managed_node_group_defaults = {
  ami_type                       = "BOTTLEROCKET_x86_64"
  ami_release_version            = "latest"
  ami_id                         = "ami-006b2b365bdf44e34"
  use_latest_ami_release_version = true
  create_iam_role                = false
  iam_role_arn                   = "arn:aws:iam::<account_id>:role/eks-node-role-standard"
  capacity_type                  = "ON_DEMAND"
  disk_size                      = 100
  enable_monitoring              = false
  update_config = {
    max_unavailable_percentage = 50
  }
  timeouts = {
    create = "15m"
    update = "40m"
    delete = "40m"
  }
  vpc_security_group_ids         = ["sg-0e2115e2c49455274"]

  # Enable Bootstrap User Data
  enable_bootstrap_user_data     = true  

  # Enable Custom Launch Template
  use_custom_launch_template     = true
  create_launch_template         = true

  # Reference the custom launch template
  launch_template_name           = "bottlerocket-eks-template"
  launch_template_version        = "$Latest"

  # Bootstrap Scripts
  pre_bootstrap_user_data = <<-EOT
    #!/bin/bash
    echo "Running pre-bootstrap"
  EOT
  post_bootstrap_user_data = <<-EOT
    #!/bin/bash
    echo "Running post-bootstrap..."
  EOT
  bootstrap_extra_args = <<-EOT
  [settings.kubernetes.kubelet-extra-args]
  "--max-pods" = "7"
  EOT  
  ebs_optimized                  = true
  block_device_mappings = [
    {
      device_name = "/dev/xvda"
      ebs = {
        volume_size           = 50
        volume_type           = "gp3"
        encrypted             = true
        kms_key_id            = "arn:aws:kms:ap-south-1:<account_id>:key/d40a66e2-f87d-4ac2-8a5b-27ff2763ef69sdfsdfdsfds"
        delete_on_termination = true
      }
    }
  ]
}

eks_managed_node_groups = {
  dna-nodegroup = {
    instance_types = ["m5a.large"]
    disk_size      = 35
    min_size       = 1
    max_size       = 6
    desired_size   = 1
    taints         = []
    labels         = {
      env  = "nodegroup"
      role = "default"
    }
  }
}

Terraform Plan Output

After running a Terraform plan, the following resources are scheduled for creation:

1. EKS Node Group Resource:

   # module.eks.module.eks_managed_node_group["dna-nodegroup"].aws_eks_node_group.this[0] will be created
   + resource "aws_eks_node_group" "this" {
       + ami_type               = "BOTTLEROCKET_x86_64"
       + capacity_type          = "ON_DEMAND"
       + instance_types         = [
           + "m5a.large",
         ]
       + labels                 = {
           + "env"  = "nodegroup"
           + "role" = "default"
         }
       + node_role_arn          = "arn:aws:iam::<account_id>:role/eks-node-role-standard"
       + release_version        = "1.35.0-af533f46"
       + subnet_ids             = [
           + "subnet-0671e6409e4b04333",
           + "subnet-073d4f6dde462b444",
         ]
       + tags                   = {
           + "Environment" = "dev-3-bottlerocket"
           + "Name"        = "dna-nodegroup"
           + "Terraform"   = "true"
         }
       + version                = "1.30"
   
       + launch_template {
           + version = "$Latest"
         }
       ...
     }
   

2. Launch Template Resource:

   # module.eks.module.eks_managed_node_group["dna-nodegroup"].aws_launch_template.this[0] will be created
   + resource "aws_launch_template" "this" {
       + description            = "Custom launch template for dna-nodegroup EKS managed node group"
       + ebs_optimized          = "true"
       + name_prefix            = "bottlerocket-eks-template-"
       + tags                   = {
           + "Environment" = "dev-3-bottlerocket"
           + "Terraform"   = "true"
         }
       + update_default_version = true
       + vpc_security_group_ids = [
           + "sg-0e2115e2c49455274",
         ]
       + block_device_mappings {
           + device_name = "/dev/xvda"
           + ebs {
               + delete_on_termination = "true"
               + encrypted             = "true"
               + kms_key_id            = "arn:aws:kms:ap-south-1:<account_id>:key/d40a66e2-f87dsfsdfdsfdsfsdfsdfsd"
               + volume_size           = 50
               + volume_type           = "gp3"
             }
         }
         ...
     }
   

After applying, running:

aws ec2 describe-launch-templates \
  --filters "Name=tag:eks:cluster-name,Values=swi-dev-3-bottlerocket" \
  --output table

returns a launch template with:

--------------------------------------------------------------------------------------------------------------
|                                           DescribeLaunchTemplates                                          |
+------------------------------------------------------------------------------------------------------------+
||                                              LaunchTemplates                                             ||
|+----------------------+-----------------------------------------------------------------------------------+|
||  CreateTime          |  2025-04-07T06:04:22+00:00                                                        ||
||  CreatedBy           |  arn:aws:sts::<account_id>:assumed-role/AWSServiceRoleForAmazonEKSNodegroup/EKS   ||
||  DefaultVersionNumber|  1                                                                                ||
||  LatestVersionNumber |  1                                                                                ||
||  LaunchTemplateId    |  lt-00858c9177ce54cc7                                                             ||
||  LaunchTemplateName  |  eks-f4cb076c-b9cd-00b8-940e-8af6eeabca30                                         ||
|+----------------------+-----------------------------------------------------------------------------------+|
|||                                                  Tags                                                  |||
||+--------------------------------+-----------------------------------------------------------------------+|
|||               Key              |                                 Value                                 ||
||+--------------------------------+-----------------------------------------------------------------------+|
|||  eks:cluster-name              |  swi-dev-3-bottlerocket                                     ||
|||  eks:nodegroup-name            |  fyn-nodegroup-20250407060414008800000003                              ||

Expected Behavior

Both the EKS node group and the associated auto scaling group should reference the same launch template. Specifically, the launch template used should consistently have eks:cluster-name set to swi-dev-3-bottlerocket to ensure that the nodes join the cluster as expected.

Actual Behavior

• Two different launch templates are created.
• The EKS node group correctly references the launch template with eks:cluster-name as swi-dev-3-bottlerocket.
• The auto scaling group, however, references a launch template with eks:nodegroup-name as fyn-nodegroup-20250407060414008800000003.
• This mismatch prevents the nodes from joining the EKS cluster.

Module & Environment Information

• Module Version: 20.33.1
• Terraform Version: v1.11.1
• AWS Provider Version: 5.84.0

Steps to Reproduce

1. Use the provided Terraform variables to configure the EKS module with Bottlerocket AMI nodes.
2. Run terraform plan and note that two launch templates are scheduled for creation.
3. Apply the configuration.
4. Verify using the AWS CLI (aws ec2 describe-launch-templates --filters "Name=tag:eks:cluster-name,Values=swi-dev-3-bottlerocket") that the auto scaling group is referencing a launch template with eks:nodegroup-name set to fyn-nodegroup-20250407060414008800000003 instead of the expected swi-dev-3-bottlerocket.
5. Check the EKS console and observe that the nodes do not join the cluster.

Additional Context

This issue appears to be related to the custom launch template configuration within the EKS module. The mismatch between the launch template referenced by the EKS node group and the one used by the auto scaling group results in nodes not being recognized by the EKS cluster. Any insights or fixes to ensure that both resources consistently use the same launch template would be greatly appreciated.

provider.tf

`terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "5.84.0"
}
kubectl = {
source = "gavinbunney/kubectl"
version = "1.19.0"
}
}
}

provider "aws" {
region = "ap-south-1"
}

provider "kubernetes" {
config_path = "~/.kube/config"
host = data.aws_eks_cluster.cluster.endpoint
token = data.aws_eks_cluster_auth.cluster.token
cluster_ca_certificate = base64decode(data.aws_eks_cluster.cluster.certificate_authority[0].data)
}

provider "kubectl" {
host = data.aws_eks_cluster.cluster.endpoint
token = data.aws_eks_cluster_auth.cluster.token
cluster_ca_certificate = base64decode(data.aws_eks_cluster.cluster.certificate_authority[0].data)

load_config_file = false # Avoids conflicts with kubectl's default config

}

Fetch EKS cluster details

data "aws_eks_cluster" "cluster" {
depends_on = [module.eks] # Ensure the module is created before fetching data
name = var.cluster_name
}

Get EKS authentication token

data "aws_eks_cluster_auth" "cluster" {
depends_on = [module.eks] # Ensure dependency on EKS module
name = var.cluster_name
}
`