Typeorm expo is slow during insert / update

77 Views Asked by At

I am having some trouble, when trying to syncing my webservice data with my internal expo sqlite database. My main issue which i have are related to performance issues of the sqlite DB i think.

My entities don't have Relations or so,

It takes ages to insert a single new record into my db (almost 2 or 3 minutes), and i can't figure out why.

hope some one is able to help me out ;D before i throw my laptop out of my window.

This is my syncing code: my datasource settings

const config: DataSourceOptions = {
        type: "expo",
        database: 'otr.db',
        migrations: [CreateDriverTable1639438000000,CreateAtions1701436776460,CreateOrderLineTable1639437586071, CreateCustomerTable1639438500000],
        driver: SQLite,
        synchronize: false,
        logging: true,
        entities: entities,
        subscribers: [],
        migrationsRun: true,
    }

ApiService.tsx


 mapToEntity(data){
   //map all json data to Entity data..
   return { ..... } as T
}
 async sync() {
        try {

            const entities = await this.findAll();

            if (!this.repository || !this.repository.syncEntities) {
                console.log("No repository found");
                return;
            }

            const mappedEntities = await Promise.all(entities.map(async (data) => {
                try {
                    return await this.mapToEntity(data);
                } catch (e) {
                    console.log(e);
                    // Handle the error if needed
                    return null; // or throw the error
                }
            }));

            await this.repository.syncEntities(mappedEntities);
            // await Promise.all(mappedEntities.map(async (entity) => {
            //     await this.repository?.syncEnity(entity);
            // }))
            console.log("Synced")
        } catch (error) {
            console.error('Error during synchronization:', error);
        }

baseRepository.tsx

    public async syncEntities(dataArray: T[]) {
        await this.connection.transaction(async (manager) => {
            const repo = manager.getRepository<T>(this.ormRepository.target as EntityTarget<T>);
            const batchInsert = dataArray.map(async (data) => {
                const entity: T = await repo.findOne({ where: { id: data.id } });
                if (!entity) {
                    return this.create(data,repo);
                } else {
                    Object.assign(entity, data);
                    return this.update(entity,repo);
                }
            });

            await Promise.all(batchInsert);;
        });
    }
/**
     * Updates the entity in the database.
     * @param entity 
     * @returns 
     */
    public async update(entity: T, repo?: Repository<T>): Promise<T> {
        try {
            if (repo != null) {
                await repo.update({ id: entity.id }, entity);
            } else
                await this.ormRepository.update({ id: entity.id }, entity);
            return entity;
        } catch (error) {
            console.log(error);
            console.log(`Error updating ${entity}`, error)
        }
    }
    /**
     * Creates a new entity in the database.
     * Using the ORM, we create a new entity and insert it into the database.
     * @param entity 
     * @returns 
     */

    public async create(entity: T, repo?: Repository<T>): Promise<T> {
        try {
            let e;
            if (repo != null) {
                e = repo.create(entity);
                await repo.insert(e); //we know it's a new entity, so we insert it
            } else {
                e = this.ormRepository.create(entity);
                await this.ormRepository.insert(e); //we know it's a new entity, so we insert it
            }
            return e;
        } catch (error) {
            console.log(`Error creating ${entity.id}`, error.message)
        }
    }

i have tried wrapping it my selections into a transaction. but insert is very slow.. also tried without async await, but yeah that did't help. Testing it with a single resource of my webservice such that i know that it is'nt a entity issue..

0

There are 0 best solutions below