You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/cosmos-db/bulk-executor-graph-dotnet.md
+12-12Lines changed: 12 additions & 12 deletions
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,6 @@
1
1
---
2
-
title: Using the graph BulkExecutor .NET library to perform bulk operations in Azure Cosmos DB Gremlin API
3
-
description: Learn how to use the BulkExecutor library to massively import graph data into an Azure Cosmos DB Gremlin API container.
2
+
title: Using the graph bulk executor .NET library to perform bulk operations in Azure Cosmos DB Gremlin API
3
+
description: Learn how to use the bulk executor library to massively import graph data into an Azure Cosmos DB Gremlin API container.
4
4
author: luisbosquez
5
5
ms.service: cosmos-db
6
6
ms.subservice: cosmosdb-graph
@@ -10,22 +10,22 @@ ms.author: lbosq
10
10
ms.reviewer: sngun
11
11
---
12
12
13
-
# Using the graph BulkExecutor .NET library to perform bulk operations in Azure Cosmos DB Gremlin API
13
+
# Using the graph bulk executor .NET library to perform bulk operations in Azure Cosmos DB Gremlin API
14
14
15
-
This tutorial provides instructions about using Azure CosmosDB's BulkExecutor .NET library to import and update graph objects into an Azure Cosmos DB Gremlin API container. This process makes use of the Graph class in the [BulkExecutor library](https://docs.microsoft.com/azure/cosmos-db/bulk-executor-overview) to create Vertex and Edge objects programmatically to then insert multiple of them per network request. This behavior is configurable through the BulkExecutor library to make optimal use of both database and local memory resources.
15
+
This tutorial provides instructions about using Azure CosmosDB's bulk executor .NET library to import and update graph objects into an Azure Cosmos DB Gremlin API container. This process makes use of the Graph class in the [bulk executor library](https://docs.microsoft.com/azure/cosmos-db/bulk-executor-overview) to create Vertex and Edge objects programmatically to then insert multiple of them per network request. This behavior is configurable through the bulk executor library to make optimal use of both database and local memory resources.
16
16
17
-
As opposed to sending Gremlin queries to a database, where the command is evaluated and then executed one at a time, using the BulkExecutor library will instead require to create and validate the objects locally. After creating the objects, the library allows you to send graph objects to the database service sequentially. Using this method, data ingestion speeds can be increased up to 100x, which makes it an ideal method for initial data migrations or periodical data movement operations. Learn more by visiting the GitHub page of the [Azure Cosmos DB Graph BulkExecutor sample application](https://aka.ms/graph-bulkexecutor-sample).
17
+
As opposed to sending Gremlin queries to a database, where the command is evaluated and then executed one at a time, using the bulk executor library will instead require to create and validate the objects locally. After creating the objects, the library allows you to send graph objects to the database service sequentially. Using this method, data ingestion speeds can be increased up to 100x, which makes it an ideal method for initial data migrations or periodical data movement operations. Learn more by visiting the GitHub page of the [Azure Cosmos DB Graph bulk executor sample application](https://aka.ms/graph-bulkexecutor-sample).
18
18
19
19
## Bulk operations with graph data
20
20
21
-
The [BulkExecutor library](https://docs.microsoft.com/dotnet/api/microsoft.azure.cosmosdb.bulkexecutor.graph?view=azure-dotnet) contains a `Microsoft.Azure.CosmosDB.BulkExecutor.Graph` namespace to provide functionality for creating and importing graph objects.
21
+
The [bulk executor library](https://docs.microsoft.com/dotnet/api/microsoft.azure.cosmosdb.bulkexecutor.graph?view=azure-dotnet) contains a `Microsoft.Azure.CosmosDB.BulkExecutor.Graph` namespace to provide functionality for creating and importing graph objects.
22
22
23
23
The following process outlines how data migration can be used for a Gremlin API container:
24
24
1. Retrieve records from the data source.
25
25
2. Construct `GremlinVertex` and `GremlinEdge` objects from the obtained records and add them into an `IEnumerable` data structure. In this part of the application the logic to detect and add relationships should be implemented, in case the data source is not a graph database.
26
26
3. Use the [Graph BulkImportAsync method](https://docs.microsoft.com/dotnet/api/microsoft.azure.cosmosdb.bulkexecutor.graph.graphbulkexecutor.bulkimportasync?view=azure-dotnet) to insert the graph objects into the collection.
27
27
28
-
This mechanism will improve the data migration efficiency as compared to using a Gremlin client. This improvement is experienced because inserting data with Gremlin will require the application send a query at a time that will need to be validated, evaluated, and then executed to create the data. The BulkExecutor library will handle the validation in the application and send multiple graph objects at a time for each network request.
28
+
This mechanism will improve the data migration efficiency as compared to using a Gremlin client. This improvement is experienced because inserting data with Gremlin will require the application send a query at a time that will need to be validated, evaluated, and then executed to create the data. The bulk executor library will handle the validation in the application and send multiple graph objects at a time for each network request.
29
29
30
30
### Creating Vertices and Edges
31
31
@@ -68,7 +68,7 @@ catch (Exception e)
68
68
}
69
69
```
70
70
71
-
For more information on the parameters of the BulkExecutor library, refer to the [BulkImportData to Azure Cosmos DB topic](https://docs.microsoft.com/azure/cosmos-db/bulk-executor-dot-net#bulk-import-data-to-azure-cosmos-db).
71
+
For more information on the parameters of the bulk executor library, refer to the [BulkImportData to Azure Cosmos DB topic](https://docs.microsoft.com/azure/cosmos-db/bulk-executor-dot-net#bulk-import-data-to-azure-cosmos-db).
72
72
73
73
The payload needs to be instantiated into `GremlinVertex` and `GremlinEdge` objects. Here is how these objects can be created:
> The BulkExecutor utility doesn't automatically check for existing Vertices before adding Edges. This needs to be validated in the application before running the BulkImport tasks.
107
+
> The bulk executor utility doesn't automatically check for existing Vertices before adding Edges. This needs to be validated in the application before running the BulkImport tasks.
* Git. For more information check out the [Git Downloads page](https://git-scm.com/downloads).
116
116
117
117
### Clone the sample application
118
-
In this tutorial, we'll follow through the steps for getting started by using the [Azure Cosmos DB Graph BulkExecutor sample](https://aka.ms/graph-bulkexecutor-sample) hosted on GitHub. This application consists of a .NET solution that randomly generates vertex and edge objects and then executes bulk insertions to the specified graph database account. To get the application, run the `git clone` command below:
118
+
In this tutorial, we'll follow through the steps for getting started by using the [Azure Cosmos DB Graph bulk executor sample](https://aka.ms/graph-bulkexecutor-sample) hosted on GitHub. This application consists of a .NET solution that randomly generates vertex and edge objects and then executes bulk insertions to the specified graph database account. To get the application, run the `git clone` command below:
@@ -126,7 +126,7 @@ This repository contains the GraphBulkExecutor sample with the following files:
126
126
File|Description
127
127
---|---
128
128
`App.config`|This is where the application and database-specific parameters are specified. This file should be modified first to connect to the destination database and collections.
129
-
`Program.cs`| This file contains the logic behind creating the `DocumentClient` collection, handling the cleanups and sending the BulkExecutor requests.
129
+
`Program.cs`| This file contains the logic behind creating the `DocumentClient` collection, handling the cleanups and sending the bulk executor requests.
130
130
`Util.cs`| This file contains a helper class that contains the logic behind generating test data, and checking if the database and collections exist.
131
131
132
132
In the `App.config` file, the following are the configuration values that can be provided:
@@ -151,5 +151,5 @@ Setting|Description
151
151
152
152
## Next steps
153
153
* To learn about Nuget package details and release notes of bulk executor .NET library, see [bulk executor SDK details](sql-api-sdk-bulk-executor-dot-net.md).
154
-
* Check out the [Performance Tips](https://docs.microsoft.com/azure/cosmos-db/bulk-executor-dot-net#performance-tips) to further optimize the usage of BulkExecutor.
154
+
* Check out the [Performance Tips](https://docs.microsoft.com/azure/cosmos-db/bulk-executor-dot-net#performance-tips) to further optimize the usage of bulk executor.
155
155
* Review the [BulkExecutor.Graph Reference article](https://docs.microsoft.com/dotnet/api/microsoft.azure.cosmosdb.bulkexecutor.graph?view=azure-dotnet) for more details about the classes and methods defined in this namespace.
Copy file name to clipboardExpand all lines: articles/cosmos-db/graph-modeling.md
+1-1Lines changed: 1 addition & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -89,7 +89,7 @@ Edge objects have a default direction that is followed by a traversal when using
89
89
90
90
However, traversing in the opposite direction of an edge, using the `in()` function, will always result in a cross-partition query. Learn more about [graph partitioning](graph-partitioning.md). If there's a need to constantly traverse using the `in()` function, it's recommended to add edges in both directions.
91
91
92
-
You can determine the edge direction by using the `.to()` or `.from()` predicates to the `.addE()` Gremlin step. Or by using the [BulkExecutor library for Gremlin API](bulk-executor-graph-dotnet.md).
92
+
You can determine the edge direction by using the `.to()` or `.from()` predicates to the `.addE()` Gremlin step. Or by using the [bulk executor library for Gremlin API](bulk-executor-graph-dotnet.md).
Copy file name to clipboardExpand all lines: articles/cosmos-db/migrate-cosmosdb-data.md
+1-1Lines changed: 1 addition & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -126,7 +126,7 @@ After the prerequisites are completed, you can migrate data with the following s
126
126
127
127
1. First import the data from source to Azure Blob Storage. To increase the speed of migration, it is helpful to parallelize across distinct source partitions. Before starting the migration, the source data set should be partitioned into files with size around 200 MB size.
128
128
129
-
2. The Bulk executor library can scale up, to consume 500,000 RUs in a single client VM. Since the available throughput is 5 million RUs, 10 Ubuntu 16.04 VMs (Standard_D32_v3) should be provisioned in the same region where your Azure Cosmos database is located. You should prepare these VMs with the migration tool and its settings file.
129
+
2. The bulk executor library can scale up, to consume 500,000 RUs in a single client VM. Since the available throughput is 5 million RUs, 10 Ubuntu 16.04 VMs (Standard_D32_v3) should be provisioned in the same region where your Azure Cosmos database is located. You should prepare these VMs with the migration tool and its settings file.
130
130
131
131
3. Run the queue step on one of the client virtual machines. This step creates the tracking collection, which scans the ADLS container and creates a progress-tracking document for each of the source data set’s partition files.
The SQL API Async Java SDK differs from the SQL API Java SDK by providing asynchronous operations with support of the [Netty library](https://netty.io/). The pre-existing [SQL API Java SDK](sql-api-sdk-java.md) does not support asynchronous operations.
Copy file name to clipboardExpand all lines: articles/cosmos-db/sql-api-sdk-bulk-executor-dot-net.md
+5-5Lines changed: 5 additions & 5 deletions
Original file line number
Diff line number
Diff line change
@@ -31,7 +31,7 @@ ms.author: ramkris
31
31
|---|---|
32
32
|**Description**| The .Net bulk executor library allows client applications to perform bulk operations on Azure Cosmos DB accounts. This library provides BulkImport, BulkUpdate, and BulkDelete namespaces. The BulkImport module can bulk ingest documents in an optimized way such that the throughput provisioned for a collection is consumed to its maximum extent. The BulkUpdate module can bulk update existing data in Azure Cosmos containers as patches. The BulkDelete module can bulk delete documents in an optimized way such that the throughput provisioned for a collection is consumed to its maximum extent.|
|**BulkExecutor library in GitHub**|[GitHub](https://github.com/Azure/azure-cosmosdb-bulkexecutor-dotnet-getting-started)|
34
+
|**Bulk executor library in GitHub**|[GitHub](https://github.com/Azure/azure-cosmosdb-bulkexecutor-dotnet-getting-started)|
35
35
|**API documentation**|[.NET API reference documentation](https://docs.microsoft.com/dotnet/api/microsoft.azure.cosmosdb.bulkexecutor?view=azure-dotnet)|
36
36
|**Get started**|[Get started with the bulk executor library .NET SDK](bulk-executor-dot-net.md)|
37
37
|**Current supported framework**| Microsoft .NET Framework 4.5.2, 4.6.1 and .NET Standard 2.0 |
@@ -56,7 +56,7 @@ ms.author: ramkris
56
56
57
57
### <aname="2.0.0-preview"/>2.0.0-preview
58
58
59
-
* Added .NET Standard 2.0 as one of the supported target frameworks to make the BulkExecutor library work with .NET Core applications.
59
+
* Added .NET Standard 2.0 as one of the supported target frameworks to make the bulk executor library work with .NET Core applications.
60
60
61
61
### <aname="1.6.0"/>1.6.0
62
62
@@ -76,19 +76,19 @@ ms.author: ramkris
76
76
77
77
### <aname="1.3.0"/>1.3.0
78
78
79
-
* Fixed an issue, which caused a formatting issue in the user agent used by BulkExecutor.
79
+
* Fixed an issue, which caused a formatting issue in the user agent used by bulk executor.
80
80
81
81
### <aname="1.2.0"/>1.2.0
82
82
83
-
* Made improvement to BulkExecutor import and update APIs to transparently adapt to elastic scaling of Cosmos container when storage exceeds current capacity without throwing exceptions.
83
+
* Made improvement to bulk executor import and update APIs to transparently adapt to elastic scaling of Cosmos container when storage exceeds current capacity without throwing exceptions.
84
84
85
85
### <aname="1.1.2"/>1.1.2
86
86
87
87
* Bumped up the DocumentDB .NET SDK dependency to version 2.1.3.
88
88
89
89
### <aname="1.1.1"/>1.1.1
90
90
91
-
* Fixed an issue, which caused BulkExecutor to throw JSRT error while importing to fixed collections.
91
+
* Fixed an issue, which caused bulk executor to throw JSRT error while importing to fixed collections.
Copy file name to clipboardExpand all lines: articles/cosmos-db/sql-api-sdk-bulk-executor-java.md
+1-1Lines changed: 1 addition & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -31,7 +31,7 @@ ms.author: ramkris
31
31
|---|---|
32
32
|**Description**|The bulk executor library allows client applications to perform bulk operations in Azure Cosmos DB accounts. bulk executor library provides BulkImport, and BulkUpdate namespaces. The BulkImport module can bulk ingest documents in an optimized way such that the throughput provisioned for a collection is consumed to its maximum extent. The BulkUpdate module can bulk update existing data in Azure Cosmos containers as patches.|
0 commit comments