From e9ad0f86f639b2e0b3e2769ce16761fed87dd56f Mon Sep 17 00:00:00 2001 From: forstisabella <92472883+forstisabella@users.noreply.github.com> Date: Tue, 9 Nov 2021 16:05:54 -0500 Subject: [PATCH 1/4] DOC-359 Adding initial draft of sync health + history doc --- .../storage/data-lakes/sync-history.md | 46 +++++++++++++++++++ 1 file changed, 46 insertions(+) create mode 100644 src/connections/storage/data-lakes/sync-history.md diff --git a/src/connections/storage/data-lakes/sync-history.md b/src/connections/storage/data-lakes/sync-history.md new file mode 100644 index 0000000000..e71f5a16bf --- /dev/null +++ b/src/connections/storage/data-lakes/sync-history.md @@ -0,0 +1,46 @@ +--- +title: Data Lakes Sync History and Sync Health +--- +{% include content/plan-grid.md name="data-lakes" %} + +Segment generates reports about each sync to your data lake so you can monitor sync performance. The sync health page provides information about trends in volume synced to your data lake over the last 30 days and the sync history provides detailed information about the syncs preformed on the data lake over the last two days. The sync data for both reports updates in real time. + +> info "" +> This feature is currently in Public Beta. Segment’s [First-Access and Beta terms](https://segment.com/legal/first-access-beta-preview/) govern the use of this feature. + +## Sync history +The Sync History table shows detailed information about the most recent syncs to the data lake. The sync history table includes the following fields: +* **Sync status:** The status of the sync: either 'Success,' where all rows synced correctly, 'Partial Success,' indicating some rows synced correctly, or 'Failed,' were no rows synced correctly +* **Start time:** The time the sync began +* **Duration:** How long the sync took to complete +* **Synced rows:** The number of rows that were synced with the data lake +* **Notices:** Any notes about the sync: if no data is synced, the notices column will read "No data found" + + + +Selecting a row in the Sync history table opens a tab with more information about the results of the sync and a detailed breakdown of sync duration. + + + +To access the Sync history page, open the **My Destinations** page and select the data lake. + +## Sync health +The Sync Health bar chart provides an overview of the volume of rows that synced to your data lake each day for the last 30 days. + +Selecting a bar from the bar chart opens the Daily Row Volume table, which provides a breakdown of which collections synced, how many rows from each collection were synced, and the percentage of all synced rows that each collection of properties represents. + + + +## Data Lakes Reports FAQ + +##### What is the maximum sync history you can view? +The sync history shows the last 30 days worth of data. + +##### How do Sync History and Sync Health compare? +The Sync History feature shows detailed information about most recent syncs to a data lake (spanning approximately two days,) while the sync health tab shows just the volume synced to the data lake over the last thirty days. + +##### How can I select which collections and properties are synced with the data lake? +You can select what collections and properties are synced with the data lake by updating the settings on the data lakes destination page. + +To access the settings page, select the Data Lakes destination from the My Destinations page. On the Warehouse Overview page, in the schema table, select the data lake you'd like to update. On the Sync History page, select the Settings tab. In the table, select the connections and properties you'd like to sync to the data lake. + From 7040edbbbb0e0f79cec01f907978c7b74f633060 Mon Sep 17 00:00:00 2001 From: forstisabella <92472883+forstisabella@users.noreply.github.com> Date: Wed, 10 Nov 2021 16:27:07 -0500 Subject: [PATCH 2/4] DOC-356 Editing, adding information --- .../storage/data-lakes/sync-history.md | 51 ++++++++++--------- 1 file changed, 28 insertions(+), 23 deletions(-) diff --git a/src/connections/storage/data-lakes/sync-history.md b/src/connections/storage/data-lakes/sync-history.md index e71f5a16bf..9057a80d3a 100644 --- a/src/connections/storage/data-lakes/sync-history.md +++ b/src/connections/storage/data-lakes/sync-history.md @@ -1,46 +1,51 @@ --- -title: Data Lakes Sync History and Sync Health +title: Data Lakes Sync History and Health --- {% include content/plan-grid.md name="data-lakes" %} -Segment generates reports about each sync to your data lake so you can monitor sync performance. The sync health page provides information about trends in volume synced to your data lake over the last 30 days and the sync history provides detailed information about the syncs preformed on the data lake over the last two days. The sync data for both reports updates in real time. +The Segment Data Lakes sync history and health tabs generate reports about data syncs so you can monitor the health and performance of your data lakes. These tools provide monitoring and debugging capabilities within the Data Lakes UI, so you can identify and proactively address data sync or data pipeline failures in real time. > info "" > This feature is currently in Public Beta. Segment’s [First-Access and Beta terms](https://segment.com/legal/first-access-beta-preview/) govern the use of this feature. ## Sync history -The Sync History table shows detailed information about the most recent syncs to the data lake. The sync history table includes the following fields: -* **Sync status:** The status of the sync: either 'Success,' where all rows synced correctly, 'Partial Success,' indicating some rows synced correctly, or 'Failed,' were no rows synced correctly +The sync history table shows detailed information about the 100 most recent syncs to the data lake. The sync history table includes the following fields: +* **Sync status:** The status of the sync: either 'Success,' where all rows synced correctly, 'Partial Success,' indicating some rows synced correctly, or 'Failed,' where no rows synced correctly * **Start time:** The time the sync began * **Duration:** How long the sync took to complete -* **Synced rows:** The number of rows that were synced with the data lake -* **Notices:** Any notes about the sync: if no data is synced, the notices column will read "No data found" +* **Synced rows:** The number of rows that were synced to the data lake +* **Notices:** Any notes or warnings about the sync - +Selecting a row in the sync history table opens a sidebar showing the number of rows that were synced in each collection. -Selecting a row in the Sync history table opens a tab with more information about the results of the sync and a detailed breakdown of sync duration. +To access the sync history page from the Segment app, open the **My Destinations** page and select the data lake. On the data lakes settings page, select the **Sync History** tab. - +> note "" +> Users cannot select when syncs occur. Syncs occur approximately every two hours. -To access the Sync history page, open the **My Destinations** page and select the data lake. +## Health +The health tab provides an overview of the rows that synced to your data lake both today and each day for the last 30 days. -## Sync health -The Sync Health bar chart provides an overview of the volume of rows that synced to your data lake each day for the last 30 days. +The bar chart, 'Daily Synced Rows,' shows an overview of the rows synced for each of the last 30 days. Hovering over a date shows the number of rows that were synced for that day. Selecting a date from the bar chart opens the Daily Row Volume table, which provides a breakdown of which collections synced, how many rows from each collection were synced, and the percentage of all synced rows that each collection represents. -Selecting a bar from the bar chart opens the Daily Row Volume table, which provides a breakdown of which collections synced, how many rows from each collection were synced, and the percentage of all synced rows that each collection of properties represents. +The Daily Row Volume table contains the following information: +* **Collections:** The name for each collection of properties synced to the data lake +* **Rows:** The number of rows synced from each collection +* **% of Total:** The percentage of the total number of rows synced that each collection represents - +Above the Daily Row Volume table is an overview of the total syncs for the day, showing the number of rows synced, the number of collections that synced, and the date. -## Data Lakes Reports FAQ - -##### What is the maximum sync history you can view? -The sync history shows the last 30 days worth of data. +To access the Sync history page from the Segment app, open the **My Destinations** page and select the data lake. On the data lakes settings page, select the **Health** tab. -##### How do Sync History and Sync Health compare? -The Sync History feature shows detailed information about most recent syncs to a data lake (spanning approximately two days,) while the sync health tab shows just the volume synced to the data lake over the last thirty days. +## Data Lakes Reports FAQ +##### How long is a data point available? +The health tab shows an aggregate view of the last 30 days worth of data, while the sync history retains only the last 100 syncs. -##### How can I select which collections and properties are synced with the data lake? -You can select what collections and properties are synced with the data lake by updating the settings on the data lakes destination page. +##### How do sync history and health compare? +The sync history feature shows detailed information about the most recent 100 syncs to a data lake, while the health tab shows just the volume of rows synced to the data lake over the last 30 days. -To access the settings page, select the Data Lakes destination from the My Destinations page. On the Warehouse Overview page, in the schema table, select the data lake you'd like to update. On the Sync History page, select the Settings tab. In the table, select the connections and properties you'd like to sync to the data lake. +##### What timezone is the time and date information in? +All dates and times on the sync history and health pages are in the user's local time. +##### When does the data update? +The sync data for both reports updates in real time. \ No newline at end of file From b2a7852f7eda62e3118cfc5ce11e50cf987e94a1 Mon Sep 17 00:00:00 2001 From: forstisabella <92472883+forstisabella@users.noreply.github.com> Date: Wed, 10 Nov 2021 16:41:50 -0500 Subject: [PATCH 3/4] DOC-359 Copyediting pass [netlify-build] --- .../storage/data-lakes/sync-history.md | 32 +++++++++---------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/src/connections/storage/data-lakes/sync-history.md b/src/connections/storage/data-lakes/sync-history.md index 9057a80d3a..df87e1d3ad 100644 --- a/src/connections/storage/data-lakes/sync-history.md +++ b/src/connections/storage/data-lakes/sync-history.md @@ -3,49 +3,49 @@ title: Data Lakes Sync History and Health --- {% include content/plan-grid.md name="data-lakes" %} -The Segment Data Lakes sync history and health tabs generate reports about data syncs so you can monitor the health and performance of your data lakes. These tools provide monitoring and debugging capabilities within the Data Lakes UI, so you can identify and proactively address data sync or data pipeline failures in real time. +The Segment Data Lakes sync history and health tabs generate real-time information about data syncs so you can monitor the health and performance of your data lakes. These tools provide monitoring and debugging capabilities within the Data Lakes UI, so you can identify and proactively address data sync or data pipeline failures. > info "" > This feature is currently in Public Beta. Segment’s [First-Access and Beta terms](https://segment.com/legal/first-access-beta-preview/) govern the use of this feature. -## Sync history -The sync history table shows detailed information about the 100 most recent syncs to the data lake. The sync history table includes the following fields: -* **Sync status:** The status of the sync: either 'Success,' where all rows synced correctly, 'Partial Success,' indicating some rows synced correctly, or 'Failed,' where no rows synced correctly +## Sync History +The 'Sync History' table shows detailed information about the latest 100 syncs to the data lake. The table includes the following fields: +* **Sync status:** The status of the sync: either 'Success,' indicating that all rows synced correctly, 'Partial Success,' indicating that some rows synced correctly, or 'Failed,' indicating that no rows synced correctly * **Start time:** The time the sync began * **Duration:** How long the sync took to complete -* **Synced rows:** The number of rows that were synced to the data lake +* **Synced rows:** The number of rows that synced to the data lake * **Notices:** Any notes or warnings about the sync -Selecting a row in the sync history table opens a sidebar showing the number of rows that were synced in each collection. +Selecting a row in the Sync History table opens a sidebar showing the number of rows from each collection that synced. -To access the sync history page from the Segment app, open the **My Destinations** page and select the data lake. On the data lakes settings page, select the **Sync History** tab. - -> note "" -> Users cannot select when syncs occur. Syncs occur approximately every two hours. +To access the Sync History page from the Segment app, open the **My Destinations** page and select the data lake. On the data lakes Settings page, select the **Sync History** tab. ## Health The health tab provides an overview of the rows that synced to your data lake both today and each day for the last 30 days. -The bar chart, 'Daily Synced Rows,' shows an overview of the rows synced for each of the last 30 days. Hovering over a date shows the number of rows that were synced for that day. Selecting a date from the bar chart opens the Daily Row Volume table, which provides a breakdown of which collections synced, how many rows from each collection were synced, and the percentage of all synced rows that each collection represents. +The bar chart, 'Daily Synced Rows,' shows an overview of the rows synced for each of the last 30 days. Hovering over a date shows the number of rows that were synced for that day. Selecting a date from the bar chart opens the Daily Row Volume table, which provides a breakdown of which collections synced, how many rows from each collection synced, and the percentage of all synced rows from each collection . The Daily Row Volume table contains the following information: -* **Collections:** The name for each collection of properties synced to the data lake +* **Collections:** The name of each collection of properties synced to the data lake * **Rows:** The number of rows synced from each collection * **% of Total:** The percentage of the total number of rows synced that each collection represents -Above the Daily Row Volume table is an overview of the total syncs for the day, showing the number of rows synced, the number of collections that synced, and the date. +Above the Daily Row Volume table is an overview of the total syncs for the current day, showing the number of rows synced, the number of collections that synced, and the current date. To access the Sync history page from the Segment app, open the **My Destinations** page and select the data lake. On the data lakes settings page, select the **Health** tab. ## Data Lakes Reports FAQ ##### How long is a data point available? -The health tab shows an aggregate view of the last 30 days worth of data, while the sync history retains only the last 100 syncs. +The health tab shows an aggregate view of the last 30 days worth of data, while the sync history retains the last 100 syncs. ##### How do sync history and health compare? -The sync history feature shows detailed information about the most recent 100 syncs to a data lake, while the health tab shows just the volume of rows synced to the data lake over the last 30 days. +The sync history feature shows detailed information about the most recent 100 syncs to a data lake, while the health tab shows just the number of rows synced to the data lake over the last 30 days. ##### What timezone is the time and date information in? All dates and times on the sync history and health pages are in the user's local time. ##### When does the data update? -The sync data for both reports updates in real time. \ No newline at end of file +The sync data for both reports updates in real time. + +##### When do syncs occur? +Syncs occur approximately every two hours. Users cannot choose how frequently the data lake syncs. \ No newline at end of file From 0ecae772c41e1f82ee0c98d87c6cb672b7de52f9 Mon Sep 17 00:00:00 2001 From: forstisabella <92472883+forstisabella@users.noreply.github.com> Date: Fri, 12 Nov 2021 12:44:57 -0500 Subject: [PATCH 4/4] DOC-359 adding sync history/health page to the main.yml file --- src/_data/sidenav/main.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/_data/sidenav/main.yml b/src/_data/sidenav/main.yml index 24908b4d28..214b6ff646 100644 --- a/src/_data/sidenav/main.yml +++ b/src/_data/sidenav/main.yml @@ -188,6 +188,8 @@ sections: title: Set Up Data Lakes - path: /connections/storage/data-lakes/sync-reports title: Sync Reports and Error Reporting + - path: /connections/storage/data-lakes/sync-history + title: Data Lakes Sync History and Health - path: /connections/storage/data-lakes/comparison title: Data Lakes vs. Warehouses - section_title: Data Warehouses