Golang application continues to connect to RDS PostgreSQL even after credential rotation

32 Views Asked by At

I have a Golang application that connects to an RDS PostgreSQL database. Initially, I set up the application to use specific credentials to establish the connection, and everything worked as expected. However, when I rotated the credentials (changed the username/password), I noticed that the application continued to connect to the database without any issues. It was only after restarting the application that I encountered an authentication error.

I expected the application to fail authentication immediately after the credentials were rotated, but it seems to be caching the old credentials somehow. Why is this happening, and how can I ensure that my application always uses the latest credentials without encountering authentication errors?

func (pg Pg) PrepareDBConnection() error {
    
    //If able to connect db do nothing
    if err := pg.DB.Ping(); err == nil {
        return nil
    }

    fmt.Println("unable to ping db, opening new connection")
    db, err := pg.config.GetDB()
    if err != nil {
        fmt.Println("unable create db connection", err)
        return err
    }
    pg.DB = db
    return nil
}

func (c *Config) GetDB() (*sql.DB, error) {
    dbSecret, err := c.SSM.GetDBSecret()
    if err != nil {
        fmt.Println("Error while fetch creds: ", err)
        return nil, err
    }
    dsn, err := c.ConnectionString(dbSecret)
    if err != nil {
        return nil, err
    }
    u, err := url.Parse(*dsn)
    if err != nil {
        return nil, err
    }
    // configure as postgres
    u.Scheme = scheme
    db, err := sql.Open(u.Scheme, u.String())
    if err != nil {
        return nil, err
    }
    if err := db.Ping(); err != nil {
        return nil, err
    }
    db.SetMaxOpenConns(10)
    db.SetMaxIdleConns(10)
    return db, nil
}

My idea is to use PrepareDBConnection to see connection is alive or not. If there is any issue, I will fetch new credentials from secret manager and create a new db connection.

I expected pg.DB.Ping() within PrepareDBConnection to fail after credential rotation. But it doesn't, and the application continues without any authentication errors until restarted.

0

There are 0 best solutions below