Skip to content

Commit 6e2ad88

Browse files
eahefnawyeahefnawy
authored
Merge pull request serverless#363 from kalinchernev/master
Add example: aws-node-signed-uploads
2 parents b94faa3 + 1cdd1de commit 6e2ad88

File tree

11 files changed

+5443
-22
lines changed

11 files changed

+5443
-22
lines changed

‎README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -75,6 +75,7 @@ serverless install -u https://github.com/serverless/examples/tree/master/folder-
7575
| [Aws Node Serverless Gong](https://github.com/serverless/examples/tree/master/aws-node-serverless-gong) <br/> A simple serverless gong using GitHub webhooks and a Slack app | nodeJS |
7676
| [Aws Node Ses Receive Email Body](https://github.com/serverless/examples/tree/master/aws-node-ses-receive-email-body) <br/> Receive an email, store in S3 bucket, trigger a lambda function. | nodeJS |
7777
| [Aws Node Ses Receive Email Header](https://github.com/serverless/examples/tree/master/aws-node-ses-receive-email-header) <br/> Receive an email, trigger a lambda function to process header. | nodeJS |
78+
| [Aws Node Singned Uploads](https://github.com/serverless/examples/tree/master/aws-node-signed-uploads) <br/> Serverless example for S3 signed uploads | nodeJS |
7879
| [Aws Serve Simple Http Endpoint](https://github.com/serverless/examples/tree/master/aws-node-simple-http-endpoint) <br/> Example demonstrates how to setup a simple HTTP GET endpoint | nodeJS |
7980
| [Aws Node Simple Transcribe S3](https://github.com/serverless/examples/tree/master/aws-node-simple-transcribe-s3) <br/> Example demonstrates how to setup a lambda function to transcribe audio file | nodeJS |
8081
| [Aws Single Page App Via Cloudfront](https://github.com/serverless/examples/tree/master/aws-node-single-page-app-via-cloudfront) <br/> Demonstrating how to deploy a Single Page Application with Serverless | nodeJS |
@@ -176,6 +177,7 @@ serverless install -u https://github.com/author/project -n my-project
176177
| **[Stack Overflow Monitor](https://github.com/picsoung/stackoverflowmonitor)** <br/> Monitor Stack Overflow questions and post them in a Slack channel | [picsoung](http://github.com/picsoung) |
177178
| **[Adoptable Pet Bot](https://github.com/lynnaloo/adoptable-pet-bot)** <br/> Tweets adoptable pets using Serverless (Node.js) and AWS Lambda | [lynnaloo](http://github.com/lynnaloo) |
178179
| **[Aws Mfa Enforce](https://github.com/Chan9390/aws-mfa-enforce)** <br/> Serverless function to automate enforcement of Multi-Factor Authentication (MFA) to all AWS IAM users with access to AWS Management Console. | [Chan9390](http://github.com/Chan9390) |
180+
| **[Aws Node Signed Uploads](https://github.com/kalinchernev/aws-node-signed-uploads)** <br/> Upload files larger than 10MB with AWS Lambda and API Gateway. Can be developed and tested locally. | [kalinchernev](http://github.com/kalinchernev) |
179181
| **[Aws Ses Serverless Example](https://github.com/lakshmantgld/aws-ses-serverless-example)** <br/> AWS SES example in NodeJS using lambda | [lakshmantgld](http://github.com/lakshmantgld) |
180182
| **[Bablebot](https://github.com/abiglobalhealth/babelbot)** <br/> Lambda + API Gateway: Zero-to-chatbot in <10 lines of JS. Built-in integrations for Messenger, Telegram, Kik, Line, Twilio, Skype, and Wechat. Or roll your own! | [abiglobalhealth](http://github.com/abiglobalhealth) |
181183
| **[Bittman](https://github.com/rhlsthrm/bittman)** <br/> A serverless project that follows a stock trading algorithm and uses scheduled functions to save data to DynamoDB and send emails through Mailgun. | [rhlsthrm](http://github.com/rhlsthrm) |

‎aws-node-signed-uploads/.babelrc

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
{
2+
"presets": [
3+
[
4+
"env",
5+
{
6+
"targets": {
7+
"node": "8.10"
8+
},
9+
"modules": false,
10+
"loose": true
11+
}
12+
]
13+
]
14+
}

‎aws-node-signed-uploads/.eslintrc

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
root: true
3+
extends:
4+
- airbnb-base
5+
env:
6+
node: true

‎aws-node-signed-uploads/.gitignore

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
coverage
2+
.serverless
3+
.webpack
4+
node_modules
5+
*.log
6+
config.json

‎aws-node-signed-uploads/README.md

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
# AWS Node Signed Uploads
2+
3+
## Requirements
4+
5+
* Node.js (version 8 is best at the moment)
6+
* npm which comes with Node.js
7+
* yarn
8+
9+
## Introduction
10+
11+
The approach implemented in this service is useful when you want to use [Amazon API Gateway](https://aws.amazon.com/api-gateway/) and you want to solve the 10MB payload limit.
12+
13+
The service is based on the [serverless](https://serverless.com/) framework. The service is uploading objects to a specific S3 bucket using a [pre-signed URL](http://docs.aws.amazon.com/AmazonS3/latest/dev/PresignedUrlUploadObject.html). Implemented in node.js runtime using [getSignedUrl](http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#getSignedUrl-property) method.
14+
15+
The package is targeting the latest runtime of AWS Lambda. ([8.10](https://aws.amazon.com/blogs/compute/node-js-8-10-runtime-now-available-in-aws-lambda/))
16+
17+
## Settings
18+
19+
If you prefer to use a different region or stage, change these:
20+
21+
```sh
22+
$ export AWS_STAGE=
23+
$ export AWS_REGION=
24+
```
25+
26+
Defaults are `dev` and `eu-central-1`.
27+
28+
Change name of upload bucket:
29+
30+
```yaml
31+
bucketName: testBucket
32+
```
33+
34+
### File name to sign
35+
36+
The file you want to upload is signed via `x-amz-meta-filekey` header.
37+
38+
### How to use
39+
40+
Get dependencies with `yarn` or `npm install`. The following examples will assume the usage of `yarn`.
41+
42+
Issue a `GET` request to get the signed URL:
43+
44+
```sh
45+
curl --request GET \
46+
--url https://{serviceID}.execute-api.{region}.amazonaws.com/dev/upload \
47+
--header 'x-amz-meta-filekey: the-road-to-graphql.pdf'
48+
```
49+
50+
If your bucket is called `foo`, and you upload `the-road-to-graphql`, after receiving the signed URL, issue a `PUT` request with the information you have signed:
51+
52+
```sh
53+
curl --request PUT \
54+
--url 'https://foo.s3.eu-central-1.amazonaws.com/the-road-to-graphql.pdf?X-Amz-SignedHeaders=host&X-Amz-Signature=the-signature&X-Amz-Security-Token=the-token&X-Amz-Expires=30&X-Amz-Date=20181210T113015Z&X-Amz-Credential=something10%2Feu-central-1%2Fs3%2Faws4_request&X-Amz-Algorithm=AWS4-HMAC-SHA256' \
55+
--data 'somemething-awesome'
56+
```
57+
58+
### Integrations
59+
60+
Here's a short list of possible integrations I found making a quick Google search:
61+
62+
* [Using pre-signed URLs to upload a file to a private S3 bucket](https://sanderknape.com/2017/08/using-pre-signed-urls-upload-file-private-s3-bucket/)
63+
* [react-s3-uploader](https://www.npmjs.com/package/react-s3-uploader)
64+
65+
### Develop locally
66+
67+
Starting a local dev server and its endpoint for receiving uploads:
68+
69+
```bash
70+
$ yarn start
71+
```
72+
73+
### Linter
74+
75+
Starting the linter tasks:
76+
77+
```bash
78+
$ yarn lint
79+
```
80+
81+
### Deployment
82+
83+
[Setup your AWS credentials](https://serverless.com/framework/docs/providers/aws/guide/credentials/).
84+
85+
Run the following the fire the deployment:
86+
87+
```bash
88+
$ yarn deploy
89+
```

‎aws-node-signed-uploads/package.json

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
{
2+
"private": true,
3+
"name": "aws-node-singned-uploads",
4+
"description": "Serverless example for S3 signed uploads",
5+
"version": "1.0.0",
6+
"scripts": {
7+
"deploy": "serverless deploy -v",
8+
"lint": "eslint .",
9+
"start": "serverless offline start"
10+
},
11+
"devDependencies": {
12+
"aws-sdk": "2.223.1",
13+
"babel-core": "6.26.0",
14+
"babel-loader": "7.1.4",
15+
"babel-preset-env": "1.6.1",
16+
"eslint": "4.19.1",
17+
"eslint-config-airbnb-base": "12.1.0",
18+
"eslint-plugin-import": "2.10.0",
19+
"serverless": "1.26.1",
20+
"serverless-offline": "3.20.1",
21+
"serverless-webpack": "5.1.1",
22+
"webpack": "4.5.0"
23+
},
24+
"author": "Kalin Chernev <kalin.chernev@gmail.com>",
25+
"license": "MIT"
26+
}

‎aws-node-signed-uploads/serverless.yml

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
service: aws-node-singned-uploads
2+
3+
plugins:
4+
- serverless-webpack
5+
- serverless-offline #serverless-offline needs to be last in the list
6+
7+
custom:
8+
bucketName: testbucket123notaken
9+
webpack:
10+
webpackConfig: 'webpack.config.js'
11+
includeModules: true
12+
packager: 'yarn'
13+
serverless-offline:
14+
port: 4000
15+
16+
provider:
17+
name: aws
18+
runtime: nodejs8.10
19+
stage: ${opt:stage, env:AWS_STAGE, 'dev'}
20+
region: ${opt:region, env:AWS_REGION, 'eu-central-1'}
21+
environment:
22+
REGION: ${self:provider.region}
23+
BUCKET: { Ref: Uploads }
24+
versionFunctions: false
25+
iamRoleStatements:
26+
- Effect: "Allow"
27+
Action:
28+
- "s3:*"
29+
Resource: "*"
30+
31+
functions:
32+
upsert-objects:
33+
handler: src/upload.handler
34+
name: ${self:provider.stage}-${self:service}-upload
35+
memorySize: 128
36+
events:
37+
- http:
38+
path: upload
39+
method: get
40+
cors: true
41+
42+
resources:
43+
Resources:
44+
Uploads:
45+
Type: AWS::S3::Bucket
46+
Properties:
47+
BucketName: ${self:custom.bucketName}
48+
CorsConfiguration:
49+
CorsRules:
50+
- AllowedHeaders:
51+
- "Authorization"
52+
AllowedMethods:
53+
- GET
54+
AllowedOrigins:
55+
- "*"
56+
- AllowedHeaders:
57+
- "*"
58+
AllowedMethods:
59+
- PUT
60+
AllowedOrigins:
61+
- "*"

‎aws-node-signed-uploads/src/upload.js

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
import AWS from 'aws-sdk'; // eslint-disable-line import/no-extraneous-dependencies
2+
3+
export const handler = async event => {
4+
const { REGION: region, BUCKET: bucket } = process.env;
5+
6+
if (!region || !bucket) {
7+
throw new Error('REGION and BUCKET environment variables are required!');
8+
}
9+
10+
const S3 = new AWS.S3({ signatureVersion: 'v4', region });
11+
12+
const file =
13+
event.headers && event.headers['x-amz-meta-filekey']
14+
? event.headers['x-amz-meta-filekey']
15+
: undefined;
16+
17+
if (!file) {
18+
return {
19+
statusCode: 400,
20+
body: JSON.stringify({
21+
message: 'Missing x-amz-meta-filekey in the header of the request.',
22+
}),
23+
};
24+
}
25+
26+
const params = {
27+
Bucket: bucket,
28+
Key: file,
29+
Expires: 30,
30+
};
31+
32+
try {
33+
const url = await S3.getSignedUrl('putObject', params);
34+
35+
return {
36+
statusCode: 200,
37+
headers: {
38+
'Access-Control-Allow-Origin': '*', // Required for CORS support to work
39+
'Access-Control-Allow-Credentials': true, // Required for cookies, authorization headers with HTTPS
40+
},
41+
body: JSON.stringify(url),
42+
};
43+
} catch (error) {
44+
return {
45+
statusCode: 400,
46+
body: JSON.stringify(error),
47+
};
48+
}
49+
};
50+
51+
export default handler;

‎aws-node-signed-uploads/webpack.config.js

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
const slsw = require('serverless-webpack');
2+
const path = require('path');
3+
4+
module.exports = {
5+
entry: slsw.lib.entries,
6+
target: 'node',
7+
externals: [{ 'aws-sdk': true }],
8+
module: {
9+
rules: [
10+
{
11+
test: /\.js$/,
12+
include: __dirname,
13+
exclude: /node_modules/,
14+
use: [{ loader: 'babel-loader' }],
15+
},
16+
],
17+
},
18+
output: {
19+
libraryTarget: 'commonjs',
20+
path: path.join(__dirname, '.webpack'),
21+
filename: '[name].js',
22+
},
23+
};

0 commit comments

Comments
 (0)