bug: Secrets Manager is not properly using --rotation-lambda-arn and --rotation-rules #12144
Description
Is there an existing issue for this?
- I have searched the existing issues
Current Behavior
(There were similar previous issues--but they were all closed as fixed.)
I create a secret key in Secrets Manager, set up a lambda to trigger on secret rotation, including all the appropriate IAM, etc. I can manually invoke the lambda and it works fine. My start up script includes as its last command:
aws secretsmanager rotate-secret \
--region $AWS_DEFAULT_REGION \
--endpoint-url $AWS_ENDPOINT_URL \
--secret-id MySecretKey \
--rotation-lambda-arn $ROTATION_LAMBDA_ARN \
--rotation-rules AutomaticallyAfterDays=30
This completes successfully with a 200 in the logs and appropriate/expected output in the lambda's logs.
However... if, after this script is done, I immediately issue this command:
aws secretsmanager rotate-secret \
--region $AWS_DEFAULT_REGION \
--endpoint-url $AWS_ENDPOINT_URL \
--secret-id MySecretKey
Then I see this in the logs:
2025-01-20T04:06:19.586 INFO --- [et.reactor-0] localstack.request.aws : AWS secretsmanager.RotateSecret => 400 (ResourceNotFoundException)
And there's zero output in a well-logged lambda, indicating it was not called. The only difference is the last 2 params, and if I manually call that command again (the first one) it works.
If I do a aws secretsmanager describe-secret... I get appropriate output:
{
"ARN": "arn:aws:secretsmanager:us-east-1:000000000000:secret:MySecretKey-MVFUZI",
"Name": "MySecretKey",
"RotationEnabled": true,
"RotationLambdaARN": "arn:aws:lambda:us-east-1:000000000000:function:RotateSecretFunction",
"RotationRules": {
"AutomaticallyAfterDays": 30
},
"LastRotatedDate": "2025-01-19T22:17:41-06:00",
"LastChangedDate": "2025-01-19T22:17:41.089000-06:00",
"LastAccessedDate": "2025-01-19T18:00:00-06:00",
"VersionIdsToStages": {
"03ee0074-71e7-4129-a4ae-a694d2055401": [
"AWSPREVIOUS"
],
"d5c6f636-aa06-4813-a849-dfdfb371cf6f": [
"AWSPENDING",
"AWSCURRENT"
]
},
"CreatedDate": "2025-01-19T22:06:02.182794-06:00"
}
Notice that both fields RotationLambdaARN and RotationRules are set, but subsequent invocations of rotate-secret aren't using them.
What would AWS do? Would it require both fields on every call? Or remember and use the earlier set values?
Expected Behavior
I would expect (if AWS does this) that once set, Secrets Manager would remember RotationLambdaARN and RotationRules fields and utilize their values on subsequent rotate-secret calls w/o having to always specify them.
How are you starting LocalStack?
With the localstack script
Steps To Reproduce
How are you starting localstack (e.g., bin/localstack command, arguments, or docker-compose.yml)
I was using a docker-compose but to get more granular and remove one moving piece I cloned the localstack repo and used python
to run locally, ie install all the dependencies and then run:
localstack --debug start
Client commands (e.g., AWS SDK code snippet, or sequence of "awslocal" commands)
The commands in question are shown in detail above, but here is my start script:
#!/bin/bash
# Set up dummy AWS credentials and region
export AWS_ACCESS_KEY_ID=test
export AWS_SECRET_ACCESS_KEY=test
export AWS_DEFAULT_REGION=us-east-1
export AWS_ENDPOINT_URL=http://localhost:4566
# Create the secret in Secrets Manager
echo ">> Creating Secret in Secrets Manager"
SECRET_ARN=$(aws --endpoint-url=$AWS_ENDPOINT_URL secretsmanager create-secret \
--name MySecretKey \
--secret-string "initialValue" \
--region $AWS_DEFAULT_REGION \
--query "ARN" --output text)
echo "Created Secret ARN: $SECRET_ARN"
sleep 1
aws --endpoint-url=$AWS_ENDPOINT_URL secretsmanager update-secret \
--secret-id MySecretKey \
--secret-string "secondKey" \
--region $AWS_DEFAULT_REGION
# Create an SNS Topic and set raw delivery (JSON only)
echo ">> Creating SNS Topic"
SNS_TOPIC_ARN=$(aws sns create-topic --name SecretKeyRotation --endpoint-url=$AWS_ENDPOINT_URL --query "TopicArn" --output text)
# Create an IAM Role for Lambda
echo ">> Creating IAM Role"
cat <<EOF > trust-policy.json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "lambda.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
EOF
aws --endpoint-url $AWS_ENDPOINT_URL iam create-role \
--role-name MyLambdaRole \
--region $AWS_DEFAULT_REGION \
--assume-role-policy-document file://trust-policy.json
rm trust-policy.json
echo ">> Attaching IAM Policy to Role"
aws --endpoint-url $AWS_ENDPOINT_URL iam attach-role-policy \
--role-name MyLambdaRole \
--region $AWS_DEFAULT_REGION \
--policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
# Grant Lambda permission to publish to SNS
aws --endpoint-url $AWS_ENDPOINT_URL iam put-role-policy \
--role-name MyLambdaRole \
--region $AWS_DEFAULT_REGION \
--policy-name PublishToSNS \
--policy-document "{
\"Version\": \"2012-10-17\",
\"Statement\": [
{
\"Effect\": \"Allow\",
\"Action\": \"sns:Publish\",
\"Resource\": \"$SNS_TOPIC_ARN\"
}
]
}"
# Retrieve Role ARN
ROLE_ARN=$(aws --endpoint-url $AWS_ENDPOINT_URL iam get-role \
--region $AWS_DEFAULT_REGION \
--role-name MyLambdaRole \
--query 'Role.Arn' --output text)
echo "Role ARN: $ROLE_ARN"
# Create the rotation Lambda function
echo ">> Creating Rotation Lambda Function"
ROTATION_LAMBDA_ARN=$(aws lambda create-function \
--endpoint-url $AWS_ENDPOINT_URL \
--region $AWS_DEFAULT_REGION \
--function-name RotateSecretFunction \
--runtime python3.9 \
--role "$ROLE_ARN" \
--handler rotationLambda.lambda_handler \
--zip-file fileb://scripts/rotationLambda.zip \
--query "FunctionArn" --output text)
aws --endpoint-url $AWS_ENDPOINT_URL lambda wait function-active-v2 --function-name RotateSecretFunction
echo "ROTATION_LAMBDA_ARN: $ROTATION_LAMBDA_ARN"
aws lambda add-permission \
--region $AWS_DEFAULT_REGION \
--endpoint-url $AWS_ENDPOINT_URL \
--function-name RotateSecretFunction \
--statement-id "allow-secrets-manager" \
--action lambda:InvokeFunction \
--principal secretsmanager.amazonaws.com \
--source-arn $SECRET_ARN
# Attach the rotation Lambda to the secret (THIS WORKS...NO ERROR)
echo ">> Setting up Rotation Lambda for MySecretKey"
aws secretsmanager rotate-secret \
--region $AWS_DEFAULT_REGION \
--endpoint-url $AWS_ENDPOINT_URL \
--secret-id MySecretKey \
--rotation-lambda-arn $ROTATION_LAMBDA_ARN \
--rotation-rules AutomaticallyAfterDays=30
echo ">> Done!"
Environment
- OS: MacOS
- LocalStack:
LocalStack version:LocalStack CLI 1.1.0
LocalStack Docker image sha: n/a
LocalStack build date: latest? -- cloned the repo today 1/19/25, using main branch
LocalStack build git hash: 4832016Anything else?
No response