Delta Lake Gen2 for a MongoDB migration

Question

Which Azure pipeline and Data Storage you would prefer for a MongoDB migration?

I know there exists the functionality of an Azure Migration Service where you can shift MongoDB data directly to an Azure CosmosDB. Azure Migration Services seems to be available only for specific licenses. Using Cosmos DB it is also necessary to take care of costs. Another possibility is to use Stitch to shift MongoDB directly into Azure.

Since we don't want to use an additional tool, we want to use Azure Data Factory to shift the MongoDB data into an Azure Data Storage. We want to use the Data Lake Storage Gen2, as it combines the advantages of the Blob Storage and the Data Lake Storage Gen1.

Which pipeline you would prefer? Any experiences with storing the MongoDB data in Azure Data Lake Storage Gen2?

Answer 1

Please see the following Azure Data Factory document pertaining to Pipelines and Activities, which details the source and target data endpoints that are currently supported.

• Copy data to or from Azure Data Lake Storage Gen1 using Azure Data Factory

• Copy and transform data in Azure Data Lake Storage Gen2 using Azure Data Factory

• Copy data from MongoDB using Azure Data Factory

Using the MongoDB connector as a source and Azure Data Lake Storage Gen2 as a sink, you can then perform any transformation and finally, migrate the data to Azure Cosmos DB...if desired.

• Copy and transform data in Azure Cosmos DB (SQL API) by using Azure Data Factory
• Copy data to or from Azure Cosmos DB's API for MongoDB by using Azure Data Factory

If you experience any issues with the migration of data to Azure Cosmos DB, if that is the goal of the migration, then consider the following direct migration paths: Options to migrate your on-premises or cloud data to Azure Cosmos DB