Re-using prepared statements for Sqlite in Golang

140 Views Asked by At

I'm using sqlx and modernc.org/sqlite (https://pkg.go.dev/modernc.org/sqlite) to insert 500K rows in a Sqlite database.

The performance is already reasonable, execution time is below a minute; but I'm hoping to improve it as I'm inserting thousand batches of 500K rows (which takes me a day on a m7g.2xlarge EC2 instance)

According to https://jmoiron.github.io/sqlx/, a NamedStmt is a representation of a prepared statement with support for named parameters.

I have a *sqlx.NamedStmt that is created by

    db.prepInsertEbsSnapshotStmt, err = db.tx.PrepareNamed(sql)

which I assumed would prepare the statement. Yet calling the call graph (generated by pprof) indicated that calling the statement below will still generate a call to the (*conn).prepareV2

db.prepInsertEbsSnapshotStmt.Exec(snap)

enter image description here

My question is : Is there a way to re-use a prepared statement using sqlx & modernc.org/sqlite as it seems that the SQL statement is prepared each time a SQL statement is executed which defeats the purpose of prepared statement

0

There are 0 best solutions below