Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Large Memory consumption with attachment #82

Open
cloudynetwork opened this issue May 22, 2018 · 7 comments
Open

Large Memory consumption with attachment #82

cloudynetwork opened this issue May 22, 2018 · 7 comments

Comments

@cloudynetwork
Copy link

cloudynetwork commented May 22, 2018

I'm getting JavaScript heap out of memory errors on a 256mb function attempting to process 12mb of attachments. In fact increasing the size showed that it actually used 576mb....

@morenoh149
Copy link
Contributor

any follow up on this? 12mb attachments are not uncommon

@drabaioli
Copy link

I need to set the lambda memory limit up to 1GB in order to process 24MB attachments. Any update?

@morenoh149
Copy link
Contributor

@drabaioli no this project has been pretty silent since it's release. If you really want to use this I'd brush up on the underlying technologies and make the improvements you need yourself.

@drabaioli
Copy link

I found a possible alternative (I haven't tried that yet though) : https://github.com/tedder/aws_lambda_ses_forwarder_python3/blob/master/lambda-ses-forwarder-py3.py

@morenoh149
Copy link
Contributor

I myself ended up paying for email (zoho) because I don’t want to worry about not receiving an email and don’t have time to learn ses and serverless

@arithmetric
Copy link
Owner

AWS SES has a limit of 10 MB on message size that can be sent. This includes all attachments after encoding, so it effectively limits binary files to around 7.5 MB.

See: https://docs.aws.amazon.com/ses/latest/DeveloperGuide/limits.html

When the message size is too large, I see this error message in the Cloudwatch Logs:

sendRawEmail() returned error.
error: 
{ InvalidParameterValue: Message length is more than 10485760 bytes long: '10515066'.
at Request.extractError (/var/runtime/node_modules/aws-sdk/lib/protocol/query.js:47:29)
...

In my testing, this module handled a message with a 7.4 MB binary attachment (before encoding) with 768 MB memory configuration in Lambda and a 6 MB binary attachment (before encoding) with 512 MB memory configuration. I used the Node.js 8.10 runtime with a 30 second timeout.

arithmetric added a commit that referenced this issue Dec 8, 2018
@drabaioli
Copy link

Thanks @arithmetric , I clearly see the limits set by AWS SES. Nonetheless I'm able to forward emails of 24MB. Only problem is that the memory required by the lambda function is over 1GB.
I wonder if it's possible to reduce the amount of memory used by the JS handler.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants