Multilingual data modeling on MongoDB

Question

I am trying to model my objects on MonogoDB and not sure how to proceed. I am building a Product catalog that will be:

• No frequent changes to product catalog. A bulk operation may be done weekly / fortnight.
• Product information is in multiple languages ( English, Spanish , French ) new language may be added anytime.

Here is what I am trying to do: I need to model my product catalog to capture the multilingual functionality. Assume I have:

product : { 
 _id:xxx,
 sku:"23456",
 name:"Name",
 description: "Product details", 
 tags:["x1","x2"]}... 
}

Surely, name,description, tags and possible images will change according to language. So, how do I model it?

1. I can have a seperate collection for each language eg: enProducts,esProducts etc

2. Have JSON representation in the product itself with the individual languages like:

   product :{
      id: xxx,
      en: {
            name: "Name",
            description: "product details.."
          },
      es: {
            name: "Name",
            description: "product details.."
          },
      ...   
   }
   

Or is there any other solution? Need help of MongoDB modeling experts here :)

Answer 1

this way will be the best:

product :{
       id: xxx,
       en: {
             name: "Name",
             description: "product details.."
           },
       es: {
             name: "Name",
             description: "product details.."
           },
       ...

  }

just because you have to search for only one product and after you could choose any language.

Answer 2

Both solutions are normally standard for this, the first being standard in RDBMS techs as well (or file based translations being another method that is not possible here).

As for which is best right here, I am leaning towards the second considering your use.

Some of the reasons would be:

• One single document load for all translations and product data, no JOINs
• Making for a single contiguous read of your disk
• Allowing for atomic updating and adding of new languages and changes etc to a single product

But creating some downsides:

• Updating could (probably will) create fragmentation which can be remedied to some extent (not completely) by powerof2sizes
• All your ops will now go to one single part of your hard disk which may actually create a bottle neck however, your scenario is such that you do not update often if at all so this shouldn't be a problem.

As a side note: I am judging that fragmentation might not bee too much of a problem for you. The reason being is that you only really bulk import products, probably from a CSV as such your documents will not probably grow greater than by the power of 2 from their insertion regularly. As such this point might be obsolete.

So overall, if planned right the second option is a good one however, there are some considerations to take into account:

• Could the multiple descriptions/fields push the document past the 16meg limit?
• How to manually pad to the document to efficiently use space and prevent fragmentation?

Those are your biggest concerns if you go with the second option.

Considering that you can fit all of the works of Shakespear into 4MB with room to spare I am actually not sure if you will reach the 16MB limit, if you do it would have to be some considerable text, and maybe storing the images in binary into the document.

Coming back to the first option, your largest concern will be duplication of certain data, i.e. price (France and Spain both have the Euro) unless you use two documents, one to house common data and the other a translation (this will make 4 documents actually but two queries).

Considering that this catalogue will never be updated unless in bulk duplicated data will not matter too much (however, for future reference in the case of expansion I will be cautious) so:

• You can make it have one document per translation and not worry about updating prices atomically across all regions
• You have one disk read without the fragmentation
• No need to manually pad your documents

So both options are readily available but I am leaning towards the second case.

Answer 3

Yet another option is to store your primary data in one language only and to have a separate text-resource translation collection where you map any text resource from your primary language to other target languages (no matter if your text resource comes from the primary data store or is just a translation of a system message on your system).

I.e. make no language specific adjustments to the schema and model at all.

The drawback that I can see is in maintaining the removal of information from the translation collection when the product is removed from the primary store, well, as soon as you guarantee that the same resource is not used elsewhere it is trivial but needs to be programmed :)

Answer 4

For a static list of languages I would go with @Zagorulkin Dmitry solution, as it is easy to query.

For a dynamic list of languages, I would rather not change the schema and allow easy management of the data.

The down side is that querying is less trivial.

  {
    "product": {
      "id": "xxx",
      "languageDependentData": [
        {
          "language": "en",
          "name": "Name",
          "description": "product details.."
        },
        {
          "language": "es",
          "name": "Name",
          "description": "product details.."
        }
      ]
    }
  }

Answer 5

I use following pattern for key and values that should be indexed in key:

 {
"id":"ObjectId",
"key":"error1"
"values":[{
             "lang":"en", 
             "value":"Error Message 1"
          },
          {
             "lang":"fa", 
             "value":"متن خطای شماره 1"
          }] 
}

and Use This Code in C#

object = coleccion.find({"key": "error1"});

view this link Model One-to-Many Relationships with Embedded Documents!

Answer 6

Another option would be to just keep the values different per language. Would probably make maintaining the schema much easier as well:

product : { 
 _id:xxx,
 sku: {
   und: "23456"
 },
 name: {
   en: "Fork",
   de: "Gabel"
 },
 description: {
   en: "A metal thingy with four spikes",
   de: "Eine Dinge aus metal der vier spitze hat"
 }  
}

und would be short for "undefined", i.e. the same for all languages, and could be used as a fallback - or you always use "en" as fallback if you'd prefer that.

The above example is roughly how Drupal CMS manages languages (albeit translated from SQL to Mongo).

Answer 7

What about this approach:

product: {
  id: 1,
  name: 'Original Name',
  description: 'Original Description',
  price: 33,
  date: '2019-03-13',
  translations: {
    es: {
      name: 'Nombre Original',
      description: 'Descripción Original',
    }
  }
}

If the user selects some language different to the default and the key translations exists in the object, you only need to merge it, and if any key has no translation, the original remains.

Another advantage is if you need to remove the translation feature or add/remove some language, you only need to change or remove the translation key and not having to refactor the entire schema.