Azure Cognitive Search indexer adds to existing non Key field

92 Views Asked by At

I have two different databases (Azure SQL) inside the same resource-group that i want to index to the same search index on Azure Cognitive Search.

Firstly i have a indexer to datasource A to populate the base model. Now i'm struggling with datasource B and populate the subfield based on that subfields Id (not the KEY Id of document).

Example index model:

{
  "@odata.context": "https://<source>.search.windows.net/indexes('index')/$metadata#docs(*)/$entity",
  "Id": "1",
  "Name": "Johnny",
  "Phone": 12345,
  "Companies": [
    {
      "Id": "2",
      "Department": null <-- I want to populate this
    }
  ]
}

My input datasource returns:

Id Company
2 [{"Id":2,"Department":"IKEA"}]
3 [{"Id":3,"Department":"Spotify"}]

I have also tried just to return as JSON without the Id field but added it when i got an error that my indexer had key field "Id" missing from datasource.

So my problem is that i need to map from the datasource, that key "Id" in this case are Company/Id.

I have tried ShaperSkillset:

    {
      "@odata.type": "#Microsoft.Skills.Util.ShaperSkill",
      "name": "department-name-mapping",
      "context": "/document",
      "inputs": [
        {
          "name": "Id",
          "source": "/document/Company/Id"
        },
        {
          "name": "Name",
          "source": "/document/Company/Department"
        }
      ],
      "outputs": [
        {
          "name": "output",
          "targetName": "Company"
        }
      ]
    }

And added to indexer:

...
"skillsetName": "department-name-mapping",
...
  "outputFieldMappings": [
    {
      "sourceFieldName": "/document/Company",
      "targetFieldName": "Companies"
    }
  ]
...

The result are that id adds a new document with Id of the Company Id, instead of populating existing documents Companies field.

Expected:

{
  "@odata.context": "https://<source>.search.windows.net/indexes('index')/$metadata#docs(*)/$entity",
  "Id": "1",
  "Name": "Johnny",
  "Phone": 12345,
  "Companies": [
    {
      "Id": "2",
      "Department": "IKEA"
    }
  ]
}

Actual:

{
 "@odata.context": "https://<source>.search.windows.net/indexes('index')/$metadata#docs(*)/$entity",
 "value": [
  {
   "Id": "1",
   "Name": "Johnny",
   "Phone": 12345,
   "Companies": [
     {
       "Id": "2",
       "Department": null
     }
   ]
  },
  {
   "Id": "2",
   "Name": null,
   "Phone": null,
   "Companies": null
  }
 ]
}

Thankful for help,

UPDATE - Solution Figured out to solve this i had to fetch the data from my datasource as the objects structure in the document.

Example:

SELECT e.Id, e.Name, e.Phone,
   (
      SELECT d.Id, d.Department
      FROM Department as d
      WHERE e.DepartmentId = d.Id FOR JSON PATH
   ) as Companies
FROM Employee as e

Now it automatically maps to a ComplexType(Collection) in search index (use WHITOUT_ARRAY_WRAPPER for non collection complexType).

0

There are 0 best solutions below