Django Elasticsearch DSL TransportError for Mapper of Different Types

Question

I'm working on a project using Python(3.6), Django(2.1), ElasticSearch(5.1.1) and Elasticsearch-dsl(5.4.0) in which I need to implement search functionality.

Here's what I have tried:

From models.py:

class searchdatamodel(models.Model):
    id = models.IntegerField(null=False, primary_key=True)
    company_name = models.TextField(blank=True, null=True)
    city = models.TextField(blank=True, null=True)
    state = models.TextField(blank=True, null=True)
    zip_codes = models.TextField(blank=True, null=True)
    street_address = models.TextField(blank=True, null=True)
    street_address_zip = models.TextField(blank=True, null=True)
    county = models.TextField(blank=True, null=True)
    phone_number = models.DecimalField(max_digits=65535, decimal_places=65535, blank=True, null=True)
    fax_number = models.DecimalField(max_digits=65535, decimal_places=65535, blank=True, null=True)
    web_address = models.TextField(blank=True, null=True)
    last_name = models.TextField(blank=True, null=True)
    first_name = models.TextField(blank=True, null=True)
    contact_title = models.TextField(blank=True, null=True)
    contact_gender = models.TextField(blank=True, null=True)
    actual_employee_size = models.IntegerField(blank=True, null=True)
    employee_size_range = models.TextField(blank=True, null=True)
    actual_sales_volume = models.IntegerField(blank=True, null=True)
    sales_volume_range = models.TextField(blank=True, null=True)
    primary_sic = models.IntegerField(blank=True, null=True)
    primary_sic_description = models.TextField(blank=True, null=True)
    secondary_sic_1 = models.IntegerField(blank=True, null=True)
    secondary_sic_description_1 = models.TextField(blank=True, null=True)
    secondary_sic_2 = models.IntegerField(blank=True, null=True)
    secondary_sic_description_2 = models.TextField(blank=True, null=True)
    credit_alpha_score = models.TextField(blank=True, null=True)
    credit_numeric_score = models.IntegerField(blank=True, null=True)
    headquarters_branch = models.TextField(blank=True, null=True)
    square_footage = models.TextField(blank=True, null=True)
    registry_date = models.IntegerField(blank=True, null=True)

    class Meta:
        managed = False
        db_table = 'searchdatamodel'

    # Implement indexing for SearchDataModel model
    def indexing(self):
        SearchDataIndex.init()
        obj = SearchDataIndex(
            id=self.id,
            company_name=self.company_name,
            city=self.city,
            state=self.state,
            zip_code=self.zip_codes,
            street_address=self.street_address,
            street_address_zip=self.street_address_zip,
            county=self.county,
            phone_number=self.phone_number,
            fax_number=self.fax_number,
            web_address=self.web_address,
            last_name=self.last_name,
            first_name=self.first_name,
            contact_title=self.contact_title,
            contact_gender=self.contact_gender,
            actual_employee_size=self.actual_employee_size,
            actual_sales_volume=self.actual_sales_volume,
            primary_sic=self.primary_sic,
            primary_sic_description=self.primary_sic_description,
            registry_date=self.registry_date
        )
        obj.save()
        return obj.to_dict(include_meta=True)

From serach.py:

class SearchDataIndex(DocType):
    id = Integer()
    company_name = Text()
    city = Text()
    state = Text()
    zip_codes = Text()
    street_address = Text()
    street_address_zip = Integer()
    county = Text()
    phone_number = Text()
    fax_number = Text()
    web_address = Text()
    last_name = Text()
    first_name = Text()
    contact_title = Text()
    contact_gender = Text()
    actual_employee_size = Integer()
    actual_sales_volume = Text()
    primary_sic = Text()
    primary_sic_description = Text()
    registry_date = Date()

    class Meta:
        index = 'data-search'

    # A method for bulk indexing
    def bulk_indexing():
        SearchDataIndex.init()
        es = Elasticsearch()
        bulk(client=es, actions=(b.indexing() for b in
                        models.searchdatamodel.objects.all().iterator()))

When I tried to run the bulk_indexing function, it returns an error like below:

File "/Users/abdul/PycharmProjects/Dmitry/DVirEnv/lib/python3.6/site-packages/elasticsearch/connection/base.py", line 125, in _raise_error raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)

elasticsearch.exceptions.RequestError: TransportError(400, 'illegal_argument_exception', 'mapper [zip_codes] of different type, current_type [integer], merged_type [text]')

I have tried by changing the zip_code type to IntegerField in model but no success.

What can be wrong here?

Thanks in advance!

Answer 1

Unfortunately, you cannot just change the type like that. You'll have to re-map the index, which is simply deleting the index and then running the mapping function again.

To delete the index, run

es = Elasticsearch()
es.indices.delete(index='data-search')

And then map the index by running the bulk indexer again.

Note: Any existing data will maintain the old datatype, so if you need to save data but change the data type, that's a whole different can of worms involving making a new version of the index, and reading old data into that.