Skip to content

Commit 2c1a79c

Browse files
author
qqfr2507
committed
Merge branch 'cloudtrail-config'
2 parents f2ed593 + 72334f7 commit 2c1a79c

File tree

3 files changed

+155
-9
lines changed

3 files changed

+155
-9
lines changed

content/integrations/aws.html

Lines changed: 155 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -2,23 +2,30 @@
22
title: Datadog-AWS Cloudwatch Integration
33
sidebar:
44
nav:
5+
- header: AWS integration
6+
- text: Configure CloudWatch
7+
href: "#cloudwatch"
8+
- text: Configure CloudTrail
9+
href: "#cloudtrail"
10+
- text: Troubleshooting
11+
href: "#troubleshooting"
512
- header: Integrations
613
- text: Back to Overview
714
href: "/integrations/"
815
---
916

17+
18+
### <a name="cloudwatch"></a>Configure CloudWatch
19+
1020
The recommended way to configure Cloudwatch in Datadog is to create a
1121
new user via the <a target="_blank" href="https://console.aws.amazon.com/iam/home#s=Home">IAM Console</a>
12-
and grant that user (or group of user) Amazon EC2 and Cloudwatch *read-only*
13-
access.
22+
and grant that user (or group of user) **Amazon EC2, Cloudwatch and CloudTrail *read-only* access**.
1423

1524
These can be set via policy templates in the console.
1625

1726
Alternatively they can set via Amazon's API according to the following
1827
specifications:
1928

20-
## Amazon Read-Only Access
21-
2229
{
2330
"Statement": [
2431
{
@@ -30,6 +37,8 @@
3037
"cloudformation:GetTemplate",
3138
"cloudfront:Get*",
3239
"cloudfront:List*",
40+
"cloudtrail:DescribeTrails",
41+
"cloudtrail:GetTrailStatus",
3342
"cloudwatch:Describe*",
3443
"cloudwatch:Get*",
3544
"cloudwatch:List*",
@@ -71,18 +80,149 @@
7180
}
7281

7382

74-
<h4>Do you believe you're seeing a discrepancy between your data in Cloudwatch and Datadog?</h4>
83+
### <a name="cloudtrail"></a>CloudTrail integration
84+
85+
AWS CloudTrail records AWS API calls for your account in log files. Datadog can read these files and create events in your stream. Here is an example of a CloudTrail event:
86+
87+
<img src="/static/images/cloudtrail_event.png" style="width:100%; border:1px solid #777777"/>
88+
89+
#### How to configure CloudTrail?
90+
91+
First make sure that you have configured CloudWatch and that the user you created for Datadog has the **AWS CloudTrail read-only access**. <a href="#cloudwatch">See above explanation</a>.
92+
93+
CloudTrail has to be configured on a per-region basis. Make sure you complete the two steps below for **all regions** that you want Datadog to collect CloudTrail data from.
94+
95+
1. <a href="https://console.aws.amazon.com/cloudtrail">Go to your CloudTrail console</a> to enable it. Then select the S3 bucket you wish to use as follows:
96+
97+
<img src="/static/images/cloudtrail_config.png" style="width:100%; border:1px solid #777777"/>
98+
99+
2. Your user must have access to the S3 bucket you have selected. To grant your user read-only access to your bucket, you would paste the following policy in the IAM console:
100+
101+
{
102+
"Statement": [
103+
{
104+
"Action": [
105+
"s3:ListBucket",
106+
"s3:GetBucketLocation",
107+
"s3:GetObject"
108+
],
109+
"Effect": "Allow",
110+
"Resource": [
111+
"arn:aws:s3:::your-s3-bucket-name",
112+
"arn:aws:s3:::your-s3-bucket-name/*"
113+
]
114+
}
115+
]
116+
}
117+
118+
#### What events are collected?
119+
120+
Below is the list of events that Datadog will collect from CloudTrail and display in your event stream. If you would like to see other events that are not mentionned here, please reach out to <a href="/help">our support team</a>.
121+
122+
**EC2 Actions**<br/>
123+
AttachVolume<br/>
124+
AuthorizeSecurityGroup<br/>
125+
CreateSecurityGroup<br/>
126+
CreateVolume<br/>
127+
CreateTags<br/>
128+
DeleteVolume<br/>
129+
DeleteTags<br/>
130+
DetachVolume<br/>
131+
RebootInstances<br/>
132+
RevokeSecurityGroupEgress<br/>
133+
RevokeSecurityGroupIngress<br/>
134+
RunInstances<br/>
135+
StartInstances<br/>
136+
StopInstances<br/>
137+
TerminateInstances<br/>
138+
139+
**RDS Actions**<br/>
140+
CreateDBInstance<br/>
141+
RebootDBInstance<br/>
142+
ModifyDBInstance<br/>
143+
DeleteDBInstance<br/>
144+
145+
**IAM Actions**<br/>
146+
AddRoleToInstanceProfile<br/>
147+
AddUserToGroup<br/>
148+
ChangePassword<br/>
149+
CreateAccessKey<br/>
150+
CreateAccountAlias<br/>
151+
CreateGroup<br/>
152+
CreateInstanceProfile<br/>
153+
CreateLoginProfile<br/>
154+
CreateRole<br/>
155+
CreateSAMLProvider<br/>
156+
CreateUser<br/>
157+
CreateVirtualMFADevice<br/>
158+
DeleteAccessKey<br/>
159+
DeleteAccountAlias<br/>
160+
DeleteAccountPasswordPolicy<br/>
161+
DeleteGroup<br/>
162+
DeleteGroupPolicy<br/>
163+
DeleteInstanceProfile<br/>
164+
DeleteLoginProfile<br/>
165+
DeleteRole<br/>
166+
DeleteRolePolicy<br/>
167+
DeleteSAMLProvider<br/>
168+
DeleteServerCertificate<br/>
169+
DeleteSigningCertificate<br/>
170+
DeleteUser<br/>
171+
DeleteUserPolicy<br/>
172+
DeleteVirtualMFADevice<br/>
173+
PutGroupPolicy<br/>
174+
PutRolePolicy<br/>
175+
PutUserPolicy<br/>
176+
RemoveRoleFromInstanceProfile<br/>
177+
RemoveUserFromGroup<br/>
178+
UpdateAccessKey<br/>
179+
UpdateAccountPasswordPolicy<br/>
180+
UpdateAssumeRolePolicy<br/>
181+
UpdateGroup<br/>
182+
UpdateLoginProfile<br/>
183+
UpdateSAMLProvider<br/>
184+
UpdateServerCertificate<br/>
185+
UpdateSigningCertificate<br/>
186+
UpdateUser<br/>
187+
UpdateServerCertificate<br/>
188+
UpdateSigningCertificate<br/>
189+
190+
**VPC Actions**<br/>
191+
AssociateDhcpOptions<br/>
192+
AssociateRouteTable<br/>
193+
AttachVpnGateway<br/>
194+
CreateCustomerGateway<br/>
195+
CreateDhcpOptions<br/>
196+
CreateRouteTable<br/>
197+
CreateVpnConnection<br/>
198+
CreateVpnConnectionRoute<br/>
199+
CreateVpnGateway<br/>
200+
DeleteCustomerGateway<br/>
201+
DeleteDhcpOptions<br/>
202+
DeleteRouteTable<br/>
203+
DeleteVpnConnection<br/>
204+
DeleteVpnConnectionRoute<br/>
205+
DeleteVpnGateway<br/>
206+
DetachVpnGateway<br/>
207+
DisassociateRouteTable<br/>
208+
ReplaceRouteTableAssociation<br/>
209+
210+
211+
### <a name="troubleshooting"></a>Troubleshooting
212+
213+
#### Do you believe you're seeing a discrepancy between your data in Cloudwatch and Datadog?
214+
75215
<p>There are two important distinctions to be aware of:</p>
76216
<ol>
77217
<li>In AWS for counters, a graph that is set to 'sum' '1minute' shows the total number of occurrences
78218
in one minute leading up to that point, i.e. the rate per 1 minute. Datadog is
79-
displaying the raw data from AWS normalized to per second values, regardless of the
219+
displaying the raw data from AWS normalized to per second values, regardless of the
80220
timeframe selected in AWS, which is why you will probably see our value as lower.</li>
81221

82-
<li>Overall, min/max/avg have a different meaning within AWS than in Datadog.
83-
In AWS, average latency,
222+
<li>Overall, min/max/avg have a different meaning within AWS than in Datadog.
223+
In AWS, average latency,
84224
minimum latency, and maximum latency are three distinct metrics that AWS collects.
85-
When Datadog pulls metrics from AWS Cloudwatch, we only get the average latency as a single time
225+
When Datadog pulls metrics from AWS Cloudwatch, we only get the average latency as a single time
86226
series per ELB.
87227
Within Datadog, when you are selecting 'min', 'max', or 'avg', you are
88228
controlling how multiple time series will be combined. For example, requesting
@@ -93,3 +233,9 @@ <h4>Do you believe you're seeing a discrepancy between your data in Cloudwatch a
93233
result.</li>
94234
</ol>
95235

236+
#### Metrics delayed?
237+
238+
<p>When using the AWS integration, we're pulling in metrics via the Cloudwatch API. There is a delay that can occur which is a byproduct of how we’re constrained when crawling the CloudWatch APIs. This is a limitation of AWS Cloudwatch; standard metrics are captured by AWS every 3-5 minutes, may not be exposed by the AWS Cloudwatch APIs before 10 minutes and we are then subject to API throttling when we crawl them, which can result in a slight delay. Overall, we're always looking to cut down on this delay and have a faster priority crawler currently in Beta.</p>
239+
240+
<p>For metrics with zero delay, we recommend installing the Datadog Agent on those hosts. We’ve written a bit about this here (especially in relation to CloudWatch):
241+
<a href="http://www.datadoghq.com/2013/10/dont-fear-the-agent/">www.datadoghq.com/2013/10/dont-fear-the-agent</a></p>
117 KB
Loading
59.1 KB
Loading

0 commit comments

Comments
 (0)