How can I manage DB switchover for write queries?

65 Views Asked by At

Spring Boot version 3.2
Java version 17
org.apache.tomcat.jdbc.pool.DataSource(tomcat-jdbc-10.1.16)

We are defining the DataSource like this

@Bean
public DataSource dataSource() {
    DataSource ds = new DataSource();
    ds.setDriverClassName(driverMapping.get(dbType));
    ds.setUrl(dbUrl);
    ds.setUsername(dbUsername);
    ds.setPassword(decryptPassword(dbPassword));
    ds.setInitialSize(initialSize);
    ds.setMaxActive(maxActive);
    ds.setMaxIdle(maxIdle);
    ds.setMinIdle(minIdle);
    ds.setTestWhileIdle(testWhileIdle);
    ds.setMinEvictableIdleTimeMillis(minEvictableIdleTimeMillis);
    ds.setTimeBetweenEvictionRunsMillis(timeBetweenEvicRunsMillis);
    ds.setTestOnBorrow(testOnBorrow);
    if (StringUtils.isNotBlank(validationQuery)) {
        ds.setValidationQuery(validationQuery);
    } else if (dbType.equalsIgnoreCase(Constants.Database.ORACLE)) {
        ds.setValidationQuery("SELECT 1 FROM DUAL");
    } else {
        ds.setValidationQuery("SELECT 1");
    }
    ds.setValidationInterval(validationInterval);

    ds.setRemoveAbandoned(removeAbandoned);
    ds.setRemoveAbandonedTimeout(removeAbandonedTimeOut);
    ds.setMaxAge(maxAge);
    ds.setLogValidationErrors(logValidationErrors);

    return ds;
}

Now come to real problem, we have a HA setup where DB also comes under a HA environment. In this case master will have all rights read/write/delete, but slave only allowed to do read operation. In some scenario, DB switchover happened and master goes down.

@Value("${db.min-idle:5}")

Now idle connection are 5, and we are assuming when db fail-over happened, some update query sent to db and it fails with the error

ERROR | ERROR: cannot execute UPDATE in a read-only transaction

We are thinking idle connection holding a DB connection with master node and it connects with a read-only db (master node which became slave after DB fail-over) for an update operation which throws the exception.

Here Datasource will validate idle connection before assigning it to application, but with SELECT query only, so if a new connection is assigned from the pool, it may be pointing to master only which is become slave now and read-only. Now all update db calls will fail.

Can this issue be resolved through Spring Boot/DataSource with some better logic?

Note - Its Container deployment so i can not create two separate DataSource. We are connecting DB from Springboot application through k8s service object.

0

There are 0 best solutions below