📖 Table of Contents

1. 🌍 Localstack Terraform Test Runner 🚀
2. 🎯 Purpose
3. 🔧 Installation
4. 🏃‍ How to Run
5. 🔍 How to Run Test Cases
6. 🔢 Default Environment Variables for Terraform Tests
7. ⚙️ Options
8. 🔐 Services

---

🌍 Localstack Terraform Test Runner 🚀

This utility serves as a test runner designed specifically for Localstack and Terraform. By leveraging it, users can execute test cases from the Hashicorp Terraform provider AWS against a Localstack Instance.

🎯 Purpose

The primary objective behind this project is to segregate test cases from the Localstack repo and execute them against Localstack. This helps in obtaining parity metrics.

---

🔧 Installation

1. 📦 Clone the repository (including submodules):

git clone git@github.com:localstack/localstack-terraform-test.git --recurse-submodules

2. 🔀 Ensure you're on the latest version of the submodules:

git submodule update --init --recursive

3. 🚀 Install dependencies:

make install

---

🏃‍♂️ How to Run

• 🔑 (Pro-image only) Set the LOCALSTACK_API_KEY environment variable.
• Apply the patch to the Terraform provider AWS:

python -m terraform_pytest.main patch

⚠️ Note: The above operation isn't idempotent. Ensure you apply the patch only once.

• Construct a testing binary for the Golang module:

python -m terraform_pytest.main build -s s3

• Now you're all set to utilize the python -m pytest commands to list and execute test cases derived from Golang.

---

🔍 How to Run Test Cases

• 📋 List all test cases from a specific service:

python -m pytest terraform-provider-aws/internal/service/<service> --collect-only -q

• 🚀 Execute a particular test case:

python -m pytest terraform-provider-aws/internal/service/<service>/<test-file> -k <test-case-name> --ls-start

or

python -m pytest terraform-provider-aws/internal/service/<service>/<test-file>::<test-case-name> --ls-start

• You can prepend additional environment variables to the command. For instance:

AWS_ALTERNATE_REGION='us-west-2' python -m pytest terraform-provider-aws/internal/service/<service>/<test-file>::<test-case-name> --ls-start

To run custom terratest test cases, you can run:

python -m terraform_pytest.main terratest-tests

---

🕵️ How to Add new terratest tests

1. Go into the terratest folder (which contains a regular go module)
2. Go into the directory where your service of interest is (e.g. ec2), or create it if it doesn't exist
3. Create a golang test file with a naming like {your_use_case_of_interest}_test.go replacing {your_use_case_of_interest} with something meaningful.
4. Create an associated directory that will host the required Terraform files (e.g. {your-use-case-of-interest}/), and put your main.tf, variables.tf and whatever you require for your test scenario.
5. Add actual testing logic to your golang file, which can be as simple as the following (and can involve logic as complex as you wish, interacting with the components created by Terraform):

package test

import (
	"github.com/gruntwork-io/terratest/modules/terraform"
	"github.com/localstack/localstack-terraform-test/terratest"
	"testing"
)

func TestUseCaseOfInterest(t *testing.T) {
	terratest.SetUpFakeAWSCredentials(t)
	awsRegion := "us-east-1"
	terraformOptions := &terraform.Options{
		TerraformDir: "./use-case-of-interest/",
		Vars: map[string]interface{}{
			"region": awsRegion,
		},
	}
	defer terraform.Destroy(t, terraformOptions)
	terraform.InitAndApply(t, terraformOptions)
}

🔢 Default Environment Variables for Terraform Tests

Variable	Default Value
TF_ACC	1
AWS_ACCESS_KEY_ID	test
AWS_SECRET_ACCESS_KEY	test
AWS_DEFAULT_REGION	us-west-1
AWS_ALTERNATE_ACCESS_KEY_ID	test
AWS_ALTERNATE_SECRET_ACCESS_KEY	test
AWS_ALTERNATE_REGION	us-east-2
AWS_THIRD_SECRET_ACCESS_KEY	test
AWS_THIRD_ACCESS_KEY_ID	test
AWS_THIRD_REGION	eu-west-1

---

⚙️ Options

• --ls-start: Initializes the Localstack instance before test case execution. It triggers the CLI:

localstack start -d

• --gather-metrics: Gathers raw test metrics for a specific run. But first, make sure you manually install the extension:

localstack extensions init
localstack extensions install "git+https://github.com/localstack/localstack-moto-test-coverage/#egg=collect-raw-metric-data-extension&subdirectory=collect-raw-metric-data-extension"

Remember to set the SERVICE environment variable for naming the metric file.

---

🔐 Services

Executing this test suite is a time-intensive process. To cater to this, the following mechanisms are in place:

Mechanism	Description
Blacklisting	Services devoid of tests are blacklisted to avoid needless execution.
Ignored	Services might have test cases, but if they all fail leading to timeouts, they're marked as non-functional and bypassed. Refer to terraform_pytest/utils.py.
Partitioning	Some services are extensive and get divided into partitions. Each partition holds a unique subset of tests for that particular service.