I am trying to do an excel import of product data where the excel will have data which is already existing in the database. So it turns out to be updating the existing product data from the values coming from excel. Each product data has relations like Tags, Categories, etc. We assume Tags to be a master data which is unique. So each product can link to tags. I used the Doctrine ORM and using the persist and flush combination. But it takes a lot of time and memory to execute. Sometimes it exceeds memory limit when product entities are larger in number. Since Tags are shared, there are many number of SELECT statements executes during the persist. I tried to apply flush() and clear() during the operation.
Can someone suggest some solution on this. Can we use cache to eliminate frequent statements for data fetch if already selected ie shared among entities.