Seed Data to Cosmos DB in Azure Devops

Question

I have ARM template creating Cosmos DB, database and collections through pipeline. Since there are multiple applications using the database, I want to seed initial data for testing, I was looking for the Cosmos DB import tasks in devops and found this https://marketplace.visualstudio.com/items?itemName=winvision-bv.winvisionbv-cosmosdb-tasks, but right now mongo API is not supported. its not able to import the data from json file which I have it in storage account.

My question is, Is there any other way I can add data from json file to cosmos DB through devops like powershell/api?

Answer 1

My question is, Is there any other way I can add data from json file to cosmos DB through devops like powershell/api?

The answer is yes.

We could try to use the Azure powershell task to execute the following powershell scripts:

param([string]$cosmosDbName
     ,[string]$resourceGroup
     ,[string]$databaseName
     ,[string[]]$collectionNames
     ,[string]$principalUser
     ,[string]$principalPassword
     ,[string]$principalTennant)

Write-Output "Loggin in with Service Principal $servicePrincipal"
az login --service-principal -u $principalUser -p $principalPassword -t $principalTennant

Write-Output "Check if database exists: $databaseName"
if ((az cosmosdb database exists -d $databaseName -n $cosmosDbName -g $resourceGroup) -ne "true")
{
  Write-Output "Creating database: $databaseName"
  az cosmosdb database create -d $databaseName -n $cosmosDbName -g $resourceGroup
}

foreach ($collectionName in $collectionNames)
{
  Write-Output "Check if collection exists: $collectionName"
  if ((az cosmosdb collection exists -c $collectionName -d $databaseName -n $cosmosDbName -g $resourceGroup) -ne "true")
  {
    Write-Output "Creating collection: $collectionName"
    az cosmosdb collection create -c $collectionName -d $databaseName -n $cosmosDbName -g $resourceGroup
  }
}

Write-Output "List Collections"
az cosmosdb collection list -d $databaseName -n $cosmosDbName -g $resourceGroup

Then press the three dots after Script Arguments and add the parameters defined in the PowerShell script (put all parameters in Variables):

You could check this great document for some more details.

Answer 2

I was running into the same scenario where I needed to maintain configuration documents in source control and update various instances of Cosmos as they changed. What I ended up doing is writing a python script to read a directory structure, one folder for every collection I needed to update, and then read every json file in the folder and upsert it into Cosmos. From there I ran the python script as part of a multi-stage pipeline in Azure DevOps.

This is a link to my proof of concept code https://github.com/tanzencoder/CosmosDBSeedDataExample.

And this is a link to the Python task for the pipelines https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/utility/python-script?view=azure-devops