Update Kafka messages docs #31430
Update Kafka messages docs #31430
Conversation
Preview links (active after the
|
Co-authored-by: Rosa Trieu <107086888+rtrieu@users.noreply.github.com>
Co-authored-by: Rosa Trieu <107086888+rtrieu@users.noreply.github.com>
Co-authored-by: Rosa Trieu <107086888+rtrieu@users.noreply.github.com>
Co-authored-by: Rosa Trieu <107086888+rtrieu@users.noreply.github.com>
Co-authored-by: Rosa Trieu <107086888+rtrieu@users.noreply.github.com>
Co-authored-by: Rosa Trieu <107086888+rtrieu@users.noreply.github.com>
Co-authored-by: Rosa Trieu <107086888+rtrieu@users.noreply.github.com>
Co-authored-by: Rosa Trieu <107086888+rtrieu@users.noreply.github.com>
Co-authored-by: Rosa Trieu <107086888+rtrieu@users.noreply.github.com>
Co-authored-by: Rosa Trieu <107086888+rtrieu@users.noreply.github.com>
Co-authored-by: Rosa Trieu <107086888+rtrieu@users.noreply.github.com>
Co-authored-by: Rosa Trieu <107086888+rtrieu@users.noreply.github.com>
content/en/data_streams/messages.md
Outdated
| @@ -2,49 +2,46 @@ | |||
| title: Messages | |||
| --- | |||
|
|
|||
| Use the Messages feature to view Kafka messages directly and troubleshoot your data streams efficiently. You can view messages at specific offsets within Kafka. Inspecting these messages can be crucial when troubleshooting issues related to specific services or data streams. | |||
| The Messages feature allows you to view Kafka messages on specific partitions / offsets, to debug poison payload problems, or to learn more about your data streams. | |||
There was a problem hiding this comment.
Can we start by describing how customers can benefit from the feature? Something like pinpoint poison pills in Kafka by inspecting messages.. Adding this doc for reference if helpful https://docs.google.com/document/d/1TmQMuvyMPDPWzOY1nW9sSLIhN6IbHTLo017DUEp4yiM/edit?tab=t.0
content/en/data_streams/messages.md
Outdated
|
|
||
| If you self-host Kafka, Datadog recommends setting up Messages on one of your Kafka brokers' agents. Otherwise, choose any one agent your producer or consumer services communicate with that has access to your Kafka cluster. | ||
| [Kafka Consumer][6] integration needs to be setup on the topic you want to retrieve messages from. |
There was a problem hiding this comment.
When looking at the integration documentation it's not clear to me what does it mean to be set up on the topic I want to retrieve messages from. If it's there, can we link it to the specific section? If it's not, then worth clarifying here or there - wherever makes more sense
content/en/data_streams/messages.md
Outdated
|
|
||
| Complete the following steps on the same Datadog agent: | ||
| 1. [Run the Agent's status subcommand][10] and look for `kafka_consumer` under the Checks section. |
There was a problem hiding this comment.
Which subcommand did you mean? Service or agent? If agent we can link to https://docs.datadoghq.com/agent/configuration/agent-commands/#agent-information
Should we link to the kafka consumer integration docs - validation and troubleshooting?
There was a problem hiding this comment.
Sure, let's link to the kafka-consumer validation docs.
There was a problem hiding this comment.
I meant the agent status command
content/en/data_streams/messages.md
Outdated
|
|
||
| ##### 2. Enable [remote configuration][3] | ||
| 1. [Run the Agent's status subcommand][10] and check the agent version. | ||
| 2. In the Datadog app, under `integrations`, `View agents`, find the agent running the Kafka Consumer integration, and check its version. |
There was a problem hiding this comment.
I couldn't find "view agents" under integrations. Can you link the view?
There was a problem hiding this comment.
Yeah, linked it. I don't like the fact that it's linking to the the app in us1 always, but maybe there is a way to fix that.
content/en/data_streams/messages.md
Outdated
|
|
||
| ##### 3. Configure Kafka consumer integration | ||
| Ensure [remote configuration][3] is setup for the agent running the Kafka Consumer integration. |
There was a problem hiding this comment.
Are there product specific instructions (existing or new) that we are relying on?
https://docs-staging.datadoghq.com/piotr.wolski/update-kafka-messages-docs/remote_configuration/#product-specific-configuration
There was a problem hiding this comment.
No. Enabling the base remote configuration should be enough
|
|
||
| ##### 1. Create a new role | ||
| ### 1. Create a new role |
There was a problem hiding this comment.
in #4 below - do we only need DSM permissions or all the permissions you mentioned above?
There was a problem hiding this comment.
The logs permissions are part of the standard role, so users should already have them.
|
/merge |
|
View all feedbacks in Devflow UI.
This merge request is not mergeable yet, because of pending checks/missing approvals. It will be added to the queue as soon as checks pass and/or get approvals.
Added to the queue but the mergequeue is not enabled for now.
This pull request was merged directly. |
What does this PR do? What is the motivation?
Merge instructions
Merge readiness:
For Datadog employees:
Your branch name MUST follow the
<name>/<description>convention and include the forward slash (/). Without this format, your pull request will not pass CI, the GitLab pipeline will not run, and you won't get a branch preview. Getting a branch preview makes it easier for us to check any issues with your PR, such as broken links.If your branch doesn't follow this format, rename it or create a new branch and PR.
[6/5/2025] Merge queue has been disabled on the documentation repo. If you have write access to the repo, the PR has been reviewed by a Documentation team member, and all of the required checks have passed, you can use the Squash and Merge button to merge the PR. If you don't have write access, or you need help, reach out in the #documentation channel in Slack.
Additional notes