Open
Description
Is there an existing issue for this?
- I have searched the existing issues
Current Behavior
I have a lambda set up behind an API Gateway that dumps a message on an SQS queue on every request. All that works fine, including the debugging in VS Code with NodeJS. However, the queue consumer lambda isn't getting hit until the API Lambda spinsdown/timesout, then the consumer will be debugged, but then the API Lambda can't be debugged or restarted until the consumer spinsdown/timesout.
Expected Behavior
I expect both breakpoints to be hit.
How are you starting LocalStack?
With a docker-compose file
Steps To Reproduce
Here is the source code:
CDK:
import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as sqs from 'aws-cdk-lib/aws-sqs';
import * as lambda from "aws-cdk-lib/aws-lambda";
import * as apigw from "aws-cdk-lib/aws-apigateway";
import { SqsEventSource } from 'aws-cdk-lib/aws-lambda-event-sources';
export class TestStack extends cdk.Stack {
constructor(scope: Construct, id: string, props?: cdk.StackProps) {
super(scope, id, props);
const queue = new sqs.Queue(this, 'TestQueue', {
visibilityTimeout: cdk.Duration.seconds(300)
});
const hello = new lambda.Function(this, "HelloHandler", {
runtime: lambda.Runtime.NODEJS_18_X,
code: lambda.Code.fromAsset("lambda"),
timeout: cdk.Duration.minutes(2),
handler: "hello.handler",
environment: {
QUEUE_URL: queue.queueUrl,
QUEUE_NAME: queue.queueName,
}
});
const consumer = new lambda.Function(this, "ConsumerHandler", {
runtime: lambda.Runtime.NODEJS_18_X,
code: lambda.Code.fromAsset("lambda"),
timeout: cdk.Duration.minutes(2),
functionName: "ConsumerHandler",
handler: "consumer.handle"
});
consumer.addEventSource(new SqsEventSource(queue))
new apigw.LambdaRestApi(this, "Endpoint", {
handler: hello,
});
// example resource
queue.grantConsumeMessages(consumer);
queue.grantSendMessages(hello);
}
}hello.ts (API Lambda)
import { APIGatewayProxyEvent, APIGatewayProxyResult, Context } from 'aws-lambda';
import { SQS, SendMessageCommandOutput } from "@aws-sdk/client-sqs"
console.log("Starting API")
if(!process.env.QUEUE_URL) throw new Error("QUEUE_URL not set!");
if(!process.env.QUEUE_NAME) throw new Error("QUEUE_NAME not set!");
const QUEUE_URL = process.env.QUEUE_URL
const QUEUE_NAME = process.env.QUEUE_NAME
export async function handler(event: APIGatewayProxyEvent, context: Context): Promise<APIGatewayProxyResult> {
console.log("API request received:", JSON.stringify(event, undefined, 2));
var sqs = new SQS({endpoint: `http://${process.env.LOCALSTACK_HOSTNAME}:4566`})
const testStr = "This was sent from the API"
var params = {
DelaySeconds: 2,
MessageBody: JSON.stringify(testStr),
QueueUrl: `http://${process.env.LOCALSTACK_HOSTNAME}:4566/000000000000/${QUEUE_NAME}`
};
sqs.sendMessage(params, function (err:any, data: SendMessageCommandOutput | undefined) {
if (err) {
console.log("Error", err);
} else {
console.log("Success", data?.MessageId);
}
});
return {
statusCode: 200,
headers: { "Content-Type": "text/plain" },
body: `Hello, CDK! You've hit ${event.path}\n`
};
}consumer.ts
import { SQSEvent, Context } from 'aws-lambda';
console.log("Starting Consumer")
export async function handle(event: SQSEvent, context: Context): Promise<void> {
console.log("Queue Consumer Received Message:", JSON.stringify(EventTarget));
}This is the error I am seeing:
2023-06-16T15:31:21.409 DEBUG --- [er:$LATEST_0] l.u.c.docker_sdk_client : Starting container localstack-main-lambda-consumerhandler-453acf0f3e14948e56440cc3a90bea07
2023-06-16T15:31:21.470 WARN --- [er:$LATEST_0] l.s.a.i.runtime_environmen : Failed to start runtime environment for ID=453acf0f3e14948e56440cc3a90bea07 with:
Traceback (most recent call last):
File "/opt/code/localstack/.venv/lib/python3.10/site-packages/docker/api/client.py", line 268, in _raise_for_status
response.raise_for_status()
File "/opt/code/localstack/.venv/lib/python3.10/site-packages/requests/models.py", line 1021, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: http+docker://localhost/v1.41/containers/aa40a914a5128dda5cf8077f7244817869c0dcc29098aa18929be6f10a8690fb/start
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/opt/code/localstack/localstack/utils/container_utils/docker_sdk_client.py", line 568, in start_container
container.start()
File "/opt/code/localstack/.venv/lib/python3.10/site-packages/docker/models/containers.py", line 405, in start
return self.client.api.start(self.id, **kwargs)
File "/opt/code/localstack/.venv/lib/python3.10/site-packages/docker/utils/decorators.py", line 19, in wrapped
return f(self, resource_id, *args, **kwargs)
File "/opt/code/localstack/.venv/lib/python3.10/site-packages/docker/api/container.py", line 1126, in start
self._raise_for_status(res)
File "/opt/code/localstack/.venv/lib/python3.10/site-packages/docker/api/client.py", line 270, in _raise_for_status
raise create_api_error_from_http_exception(e) from e
File "/opt/code/localstack/.venv/lib/python3.10/site-packages/docker/errors.py", line 39, in create_api_error_from_http_exception
raise cls(e, response=response, explanation=explanation) from e
docker.errors.APIError: 500 Server Error for http+docker://localhost/v1.41/containers/aa40a914a5128dda5cf8077f7244817869c0dcc29098aa18929be6f10a8690fb/start: Internal Server Error ("driver failed programming external connectivity on endpoint localstack-main-lambda-consumerhandler-453acf0f3e14948e56440cc3a90bea07 (69e7c30a80ea6cf486fe57538a185a1d68d2ccdfd2a89ede0e92f60a18d76027): Bind for 0.0.0.0:9229 failed: port is already allocated")
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/opt/code/localstack/localstack/services/awslambda/invocation/runtime_environment.py", line 172, in start
self.runtime_executor.start(self.get_environment_variables())
File "/opt/code/localstack/localstack/services/awslambda/invocation/docker_runtime_executor.py", line 340, in start
CONTAINER_CLIENT.start_container(self.container_name)
File "/opt/code/localstack/localstack/utils/container_utils/docker_sdk_client.py", line 573, in start_container
raise ContainerException() from e
Environment
start command
DEBUG=1 LAMBDA_DOCKER_FLAGS='-e NODE_OPTIONS=--inspect-brk=0.0.0.0:9229 -p 9229:9229' \
LAMBDA_REMOTE_DOCKER=0 \
localstack start -dAnything else?
By the looks of it, only one lambda can be debugged at a time. Is this intended? Is there a way to debug the whole flow?